I've had a lot of success with Axure RP Pro. They're up to version 6.0 as of this writing, and it shows. With literally thousands of current users, Axure seems to be the prototyping tool to beat.
Some of the best aspects of Axure include the ability to model realistic interactions such as conditional events, drag-and-drop activities, animation, and other jQuery-like behaviors. Designing a prototype is fairly quick and easy and requires little or no coding if you're not up to it. On the other hand, Axure provides support for more advanced designs using conditional logic, variables, and event chaining, so you can usually create whatever you need.
One of the often-perceived drawbacks of Axure is that, although it presents the prototype in a browser, the resultant code is junky and not usable for production purposes. Yes, it generates pretty ugly HTML, CSS, and JavaScript, all of which can be edited in the usual way, but I don't recommend doing so. There are two ways of thinking about this:
1) Prototypes are explorations. They are temporary constructions, highly mutable, and should be easily created and revised without any concern for code cleanliness or adherence to standards. Or...
2) Prototypes are the foundations for the finished product and should eventually evolve into production-ready code. While they may still be exploratory in nature, prototypes should be developed so as to minimize the need for refactoring.
Personally, I subscribe to the first school of thought. Prototypes are meant to explore the nature of a problem, devise possible solutions, and iterate through alternatives. Since I'm no genius, that usually means trying things out, making mistakes, and trying again until I get it right. Having coded prototypes by hand, I can testify that they don't lend themselves to a lot of adventurous exploration: Being too adventurous runs the risk of messing up perfectly good code, and getting to "good" can take a while. So Axure's code junkiness doesn't bother me as long as I can quickly tear though design alternatives. Just don't let anybody use the resultant code.
Axure's not perfect, however. While it was clearly aimed at the UX profession, it was also pretty clearly not designed by a UX person. That's slowly changing, and I'm happy to see that the soon-to-be-released v6.0 is better, experience-wise, than v5.6 was.
Finally, I'd like to comment on the other means for rapidly prototyping a web app by saying that
they're all valid. However, what you end up using will depend on the nature of your organization.
- Paper prototyping: In my case (working at Texas Instruments with thousands of employees worldwide and about 14,000 different products), paper prototyping won't cut it. It might be useful for some level of interaction design, but a paper prototype usually won't contain enough specification to carry itself through the development process or the testing cycle, nor will it do well if you have to deal with an offshore development team.
- Balsamiq: It's OK, but you have to struggle a bit to make it engaging. Since your question was about prototyping a Web app, you'll want a tool that lends itself well to creating snappy, realistic user interactions. I've used Balsamiq extensively in the past, but I could never count on doing anything other than static presentations with it (or, actually, semi-static presentations, since you can link pages together). Again, that may be good enough for your situation, but from the question you asked, Balsamiq doesn't feel like a suitable match.
- Keynote/Powerpoint/Visio/Illustrator: Useful for quick prototyping of static or semi-static designs, and (except for the lack of deep interactivity) relatively suitable for the entire development process as long as you make a good effort to document your specifications. The real drawback to these sort of prototypes is that you have to make the extra effort to tell how an interaction behaves, rather than just showing it, as with an Axure prototype or a hand-coded artifact. This is the way I used to present all my designs, and it invariably required a lot of hand waving and sound effects to convey the nature of the interaction.
- Photoshop: Very useful (sometimes deceptively so) for creating full-blown comps of the finished vision. With a little deft manipulation you can even get screens to "link" to each other, but I tend to stay away from Photoshop for prototyping purposes unless I really need the "wow" factor to sell something. (But if I need that level of fidelity just to sell an idea, maybe it's not a great idea to begin with.) Plus, once a client sees a Photoshop rendering, there's no going back--you'll have to stay in Photoshop for the rest of the prototyping process. Good luck with that.
Anyway, that's about it. If you're looking for a prototyping tool to rapidly iterate through Web app design ideas, you're going to want to use something that:
- Enables you to create highly interactive, realistic designs
- Gives you the flexibility to employ advanced effects
- Creates explicit specifications that can be used throughout the development process
For me, that's Axure. At least until something better comes along.