Self-evaluation
That’s what user testing is. Now, ask yourself the following questions:
•Do you conduct user tests?
•Are they a regular part of your practice?
•Would you like to do more of them?
•What’s keeping you from doing more of them?
I ask these questions often. It’s amazing how few of us do user testing with any consistency, myself included. Everyone wishes they did more of it. That’s both a problem and an opportunity.
User testing in an agile world
The agile mantra is “fail fast, fail early”. The faster you fail, the faster you’ll converge on the right solution. This equates to a lot of tight iterations. Agile teams traditionally have two-week sprints, with the goal of releasing a running (read: testable) build at the end of each sprint.
Great, right? The problem is that this leaves very little time to validate a design, summarize the results, and do just-in-time design for the next iteration. Recruiting can take a week in itself, to say nothing of the testing.
And that’s not tenable. At most, you’ll have a few days to get some actionable insights before the next iteration starts. How might we solve this problem?
Let’s make a few assumptions:
•Five iterations from the start to the end of the design process.
•Five participants in each user test (25 participants for all iterations).
•Four designs in flight simultaneously (five iterations each, 100 participants in total).
One way to solve the problem of getting out in front is to validate multiple iterations before any software is built. Not every design needs a live-code prototype to validate it. Sometimes, a clickable Balsamiq PDF is enough. Now, we’ve shifted the problem. The number of design iterations (and the number of test participants) is the same as before, but you can get a lot further before engineering starts building anything. You just need a lot of participants, fast.
User testing at conferences
Unless you’re lucky enough to design a product that millions of people use, recruiting can be a challenge. Since I design software for system administrators, the best place to get qualitative feedback in a matter of days is at an IT conference.
The basic steps are:
1. Pick a conference
2. Write some studies
3. Set up your booth
4. Analyze the results (in real time)
5. Iterate on the design
6. Rinse and repeat
Obviously, you’ll need help, so bring some volunteers with you. Also, don’t expect to nail this the first time you try it. Give yourself a chance to make mistakes and learn from them.
Conferences:
the best place to conduct a lot of user tests in a very short amount
of time. (Image: LeWeb2)
The number of times you can iterate depends on what you’re learning. If you’re learning a lot, keep going. If you’re running into tool limitations, it might be time to stop and have your development team build you a live-code prototype.
Bonus: if you have software development skills, you might be able to build a prototype yourself. Better yet, bring some developers with you.
Disclaimer: I’ve done conference-based user testing twice, and haven’t entirely nailed these steps (even though we’ve made great strides in the right direction). It might take a few tries to get it right.