LP Login

Think Big. Move Fast.

I recently posted about how usability testing can slow down launch but speed up success. But usability testing is just one of many elements of user experience research, with others including the ethnographic field studies made popular by Ideo, the A:B testing becoming standard for web 2.0, customer feedback, focus groups etc. With so many tools at your disposal, which user experience research tools should you use and when?

Jakob Nielsen recently posted about this topic, and concluded that it depends on what phase of product management you are in. For startups, my summary of his work is below:

Ideation: At the very beginning of the process you are looking at new ideas and opportunities. In this phase, aside form the founders vision, ethnographic field studies, focus groups, diary studies, surveys and data mining of webwide behavior can all be useful. Most startups will not have access to proprietary user data of existing products to identify additional opportunities.

Pre-launch: Once you’ve settled on a product idea and are working towards (beta) launch, you want to improve design and functionality as much as possible to minimize risk and maximize the likelihood for a successful launch. In this phase rely on tools such as cardsorting, paper prototype and usability studies, participatory design, desirability studies and field studies (including closed alpha launches to “friendlies”) to improve the user experience.

Post-launch: Once you’ve pushed the product out you will have live data that you can use to compare the product both to itself and to its competition. In this phase, usability benchmarking, online assessments, customer emails, surveys and A/B testing will be your primary tools

Nielsen provides some additional frameworks to differentiate when to use different forms of user experience research in his post. The site is a good resource about user experience in general.

  • Amy Jo Kim

    great stuff — thanks for posting!

  • http://www.perkler.com Justin Barrie

    Jeremy – Ideo are fantastic at the ‘building’ end of design. Another great user experience company to check out is run by a mate of mine in Redwood Shores – Cheskin. Their thinking is fantastic and they have been a bit of an inspiration for our approach to user experience.


  • http://www.signalfive.com Tim Jaeger


    One tool I find indispensible is OmniGraffle Pro. It is mac-based (hope that isn’t too bad for all the PC users out there..), but there are tons of custom stencils you can make make for it that scale as vectors (or vectorially, whichever..).

    I wrote a post on it, you can check it out:


  • Alison Murdock

    How many start-ups do user testing beyond the immediate families of the 3-10 employees? How many people do they test and how much do they spend – time- and money wise?

  • http://lsvp.wordpress.com jeremyliew


    This varies widely, but it is a good idea to do user testing on non employees who are in the target market and not familiar with the product.

    In general, doing usability testing with 5 people is enough to uncover most usability problems (http://www.useit.com/alertbox/20000319.html). This can be done in a day or two and you can recruit testers via craigslist or a similar resource for $20/hr and do the testing via webex and a telephone so it should be cheap.

    Of course, you have to be willing to listen to the feedback (vs attributing it to “user error” – there is no such thing as user error in usability testing!) and make changes to address issues that come up. That may take considerably more time and money

  • Pingback: Quora()