Preparation for a User Day
What Do We Log?
For a successful test, it is important to note down as much as possible. Record the steps the tester takes as completely as possible.
Note down all the tester's remarks and your own ideas in the margins.
During your observations, do not try to filter or compile the information, as you will lose time and information that could prove valuable at a later time.
The following table lists typical problem areas in applications. Both the test user and the observer should watch out for such problems during testing.
|
Typical problem areas
|
Examples
|
|---|---|
|
Task suitability
|
How well can you solve the
task using the program? Do you get the desired result? Are there unnecessary requirements? Are flexible processing strategies supported? |
|
Self-explanation
|
Can you learn the application
without a long introduction? Can you figure out the functions by trying them out? Is there enough online help? |
|
Efficiency
|
Are the results worth the
cost? Does the application contain complicated steps? Is the application clear? |
|
Structure
|
Does the organization of
the application suit the task? Are there bothersome dialogs or an unnecessary context change? |
|
Transparency
|
Is the application clear? Can you see where you are? Can you see what you can do? |
This table tells you what things you should log when observing the user because they help in dealing with the typical problems described above:
|
Typical user problems
|
Examples
|
|---|---|
|
Comprehension problems
|
Does the user understand
the task statement? Is it immediately clear what the task requires of the user? Does the tester understand the contents of the screen? |
|
User works in an unexpected order
|
What steps does the user
perform when solving the task? What program parts/pushbuttons are activated and in which order? |
|
Errors and problems
|
Where do misunderstandings/incorrect
transactions occur in the solution? Record any incorrect/inconvenient steps. |
|
Assistance
|
Where did the observer have
to provide assistance? What assistance did he or she give? |
|
Exceptionally long breaks or processing times
|
Which program parts require
intensive thought? Which phases seem to be difficult or complicated? |
Users working with the application for a long time will probably react in different ways. Note down the user's feedback about the application in addition to the objective observations above.
|
General observations
|
Examples
|
|---|---|
|
User satisfaction
|
Does the application make
a good impression? Does the user feel frustrated when using the program? If yes, in which program parts? Which parts are particularly good? |
|
User comments
|
How does the user react
to the program and its steps? Note down any negative or positive points. Where does the program not correspond to the user's requirements? Note signals such things as sighing - non-verbal communication also indicates that the user finds the problem too complicated and demanding. |
|
Design proposals
|
Does the user have ideas
about improvements to the program design? Are they based on interaction with the product or ideas from other products? Is there a theme or principle that underlines the idea? |
Source: User Day Toolkit