Usability Testing

I need:

• a computer w/mouse
• Morae software installed on computer
• a VM to test on
• camcorder w/tapes to record sessions (use built-in laptop camera?)
• tripod
• compensation for testers ($50 each, MT t-shirt)
• release form

Control Panel Usability Testing Tasks/Goals:

*Vary question order to avoid testing bias.

1. You want to add a user to the database.

    Name: Dan Smith
    Email: dsmith@mindtouch.com
    Password: (blank)
    Role: Viewer
    Auth: Local
    Groups: (none)

2. A user has been abusing his privledges on the wiki and you wish to ban him until December 31, 2008. Where would you go to do this? *provide name

3. You want to add users to the group titled “Sales Team.” The usernames are: mjones, sharms, jscott.

4. The company wants to add a new database of users to the wiki and therefore needs to add another authentication provider called LDAP, without making any changes to the default settings. They also want to make this the default provider.

5. Your Deki needs to reflect the look and feel of your company’s brand so you would like to change the logo and template skin CSS. The designer has sent you the logo and CSS files to add to the site. *provide JPG and CSS files

6. You’re not sure what the exact username or email address is for a particular user, however you know that his username starts with “J” since his first name is Joe. He recently left the company on September 16, 2008 so you think this may have been the last time that he was active on the site. You need to access his account so you can change his role as well as remove him from a group and add him to another.

    New role: Guest
    Remove from: Developers
    Add to: Inactive Accounts

7. The company is expanding and has added a couple of new departments: Sales, Marketing, and Designers. You’d like to create groups with these names as well as add the users from the group “New Hires” to only the Marketing group.

8. Since you’ll be using the control panel on a regular basis, you would like to customize your quick links to include the actions you most frequently use. (Let the tester choose whatever they’d like for this).

9. Your company has just purchased MindTouch Deki Enterprise Edition and would like to activate it. The license key is X. *get dummy license key

10. You accidently deleted a page and would like to get it back so others can view it again. The title of the page is X. *create page

First Impression Questions:

1. What site is this? How can you tell?

2. Who do you think this site is for? How can you tell?

3. What do you think you will find at this site? How can you tell?

4. What are your initial impressions of the site?

Questions at end:

1. How would you describe finding what you were looking for on this site?

2. What did you like best?

3. What did you like least?

4. What would be the first thing you'd do to improve the site?

5. If you were to rate this site on a scale of 1 to 10 on the overall experience, what would you give it?

Suggestions:

* In your script, remind participants that the usability test is an evaluation of the product and not of their performance, that all problems they find are helpful and that their feedback is valuable, and let participants know their data will be aggregated with the rest of the participant data and they will not be identified.

* Ask open ended questions
o What are you thinking right now?
o What are you trying to do?
o Is there anything else you might try?
o Where would you go?
o What did you expect to happen?
o You seemed surprised or frustrated…?
o Exactly how did that differ from what you expected to happen?
o Would you expect that information to be provided?
o Please keep talking…

* Provide open-ended hints only when asked: “Do you see something that will help you?”
* Don't offer approval or disapproval with words, facial expressions or body language.

* Occasionally, a participant will fail to complete or will outright quit trying to complete a task. Indirect hints or encouragement such as “is there anything on the screen to help you?” may be used to encourage the participant to explore, but at some point he or she should be allowed to fail.

If a participant fails a task but needs the information from that task to continue, a recommended technique is to count the failure but have the participant try the required portion of the first task again. Doing this lets you understand better how long it takes participants to “get” a particular interaction. You can then gage how easy or hard it is to learn to perform the task and more about where they might be confused by your product.

Provide a more direct hint only as a very last resort.

After a session:

* Reset your machine, clear data and save the participant’s work, if appropriate.
* Ask "Why?" at least five times -- ie. Did the user struggle? Yes. Why? Because they couldn't find the link. Why? Because it was not big enough to see or in the right place. Etc...