Assignment Video: User Test Plan

Loading...

From the course by University of Minnesota

Evaluating User Interfaces

114 ratings

University of Minnesota

Evaluating User Interfaces

114 ratings

Course 4 of 5 in the Specialization User Interface Design

In this course you will learn and practice several techniques for user interface evaluation. First we start with techniques that can be applied alone or in a design team, including action analysis, walkthroughs, and heuristic evaluation. Then we move on to user testing, including learning from a series of usability tests carried out in a real usability lab, and techniques to carry out your own tests even without a lab. Finally, we wrap up the discussion of evaluation--and of UI Design in the specialization as a whole--by looking at the question of how to set and measure usability goals, and in turn, when a design is usable enough to release it.

From the lesson

Evaluation with Users (Part 2)

We continue our discussion of with-user evaluation techniques (Part 2 of 3)

Ethics in User Evaluation10:07

Drafting a User Test Plan17:44

Field Study Case Example: ShareTable Deployment22:18

Assignment Video: User Test Plan7:14

Meet the Instructors

Loren Terveen
Professor
Computer Science and Engineering
Haiyi Zhu
Assistant Professor
Computer Science and Engineering
Lana Yarosh
Assistant Professor
Computer Science and Engineering
Dr. Brent Hecht
Assistant Professor
Computer Science and Engineering
Joseph A Konstan
Distinguished McKnight Professor and Distinguished University Teaching Professor
Computer Science and Engineering

Hello, and welcome back.

Today, I'll be giving you a little more detail about the assignment that

you'll be doing for this module, the user test plan.

So, first of all,

if it's been awhile since you've watched the course 4 videos, before doing this

assignment you may want to review three videos that are most relevant to it.

The user testing goals, formative and summative.

Just to remind yourself what these two terms mean.

The most relevant video is definitely drafting a user test plan

because it walks you through all of the categories that you actually have to

fill out in your user test plan assignment.

And I also think that the usability lab examples

that we recorded can really be helpful here because they provide concrete

examples of tasks that somebody might give to use with a particular system.

So, in terms of the actual assignment,

you'll be provided with a short description of a situating context for

your user test plan, and you should write a two to three page user test plan.

Now, the key is you want to have all of these sections, because the peer

graders will actually be asked to review that all of these sections are present.

So, first, you want to specify very concretely the objectives of your study,

and I recommend separating these into formative and summative,

I think most cases, you want to have some formative and some summative.

You want to talk about the users like who you will actually recruit for this study.

How many users of each kind.

I think it's important to be specific, so don't just say just general population.

How many people, and what does general population mean?

Is my grandma general population?

Am I general population?

Is, I don't know, my three year old nephew general population?

So you actually want to specify exactly who you want to recruit or

your exclusion criteria maybe.

And just kind of think through that aspect of it carefully.

The setting, where will the study actual take place,

what kinds of tools will be provided to the user, methods and

metrics, so will you ask the user to do something in particular way?

What will you actually measure as they're doing this.

The task.

So again, what is the specific task that you are asking the user to accomplish?

So if you're building some sort of an interactive system,

this is likely some sort of a common task that you might

expect them to do with your system in the real world.

And lastly, researcher roles.

How many researchers are actually needed to run the study and

what their roles might be.

And so the reason we're asking you to do this assignment is because this is

actually a very common task in user experience roles.

So there's some aspect of a system that you may be asked to test.

And the very first thing you need to do is you need to draft out this plan

that will articulate all these aspects of it, all these objectives.

So that you know what will actually happen in your user study.

As we of course know user studies cost money and they cost time so

you want to make sure that you use that time well.

Now in terms of the components I'd say the key things, the things that will probably

take the most time for you to think about and write are the methods and metrics.

So what are actually measuring.

How are you actually going to set up your study in the tasks?

What are the specific tasks that the user's going to be asked to do with

the system?

Now let me kind of go through quickly the situating context that you'll be using for

this assignment.

So I will ask you to pretend that you've been hired as a consultant

to evaluate a new web-based public library catalog.

So I'm not sure if your county or your state or

your country has these kind of public library catalogs.

I will provide you with one that you can kind of use as kind of an inspiration as

you think about this.

But let's imagine that a new web-based public library catalog is being designed

to replace the old web-based library catalog.

And what the designers are hoping for is that the new one will be easier to use,

that it will make it easier to explore and find you books to read.

It'll be faster to finding a specific book if somebody already knows exactly what

they want to find.

The design team also wants to know just in general,

what can be improved about their design.

Where they should go with it next and kind of what are the areas of weakness and

strength there?

So your plan is to draft the user study plan to accomplish these goals.

So including the focused statement of objectives, users, settings, methods,

tasks, metrics, and researcher roles.

So all of these things that I've talked about on the previous slide and, as I

mentioned, there was a slide dedicated to each of these in the drafting and

user test plan video.

So, as an inspiration, if your library doesn't have an online catalog,

you can use my county's online library catalog.

I'm sure all of you can get access to it.

It's the Hennepin County Library, hclib.org.

So, take a look there, just to see,

kind of what generally goes in these and how are they structured.

In terms of tips for succeeding on this assignment.

So, the first thing I say is, take a look at the sample user test plan.

So, I've done this assignment myself, I did it for

a different context, so just to keep it different enough.

But you can see what sections are included and

how much detail roughly is expected for each of these sections.

I think you can just model it on that one.

And if you are providing about the same level of detail you should

get a good grade on this assignment.

You want to clearly highlight each of the required sections to make it easier for

their peers to grade it.

So you don't want to just lump everything, your objectives and your metrics and

your methods and your users into a single paragraph.

You really want to separate them on the page.

Again because you're being evaluated on whether you have each of these sections so

make it very easy for

the person who's grading to tell that yes in fact he's met all the requirements.

Be specific in each of the sections.

So like I mentioned before,

don't just say that your users are coming from the general population,

that's kind of my pet peeve, you really want to be specific about who they are.

In terms of the setting, you might want to say,

is this something that you're expecting to conduct in a lab or do you think that this

would be a study that should be conducted in the actual library?

Where is it going to be done?

Now you won't actually have to run this user study, you just have to plan it.

So feel free to get as creative as you want and

assume you have the resources you need.

Even if you personally don't have access to a usability lab,

you can assume that maybe for the study you wouldn't have access to it.

So if you plan the setting to be a usability lab, how would you set it up?

What would be available there?

Or vice versa.

Even if you don't currently, if you can't travel to Minnesota to go to Hennepin

county library to check out how people would use the new library catalog,

just assume that you would have access to that context if you needed it.

So I think you can be creative and kind of think about the best case scenario and

just having all the resources you want to do the creative usability study that you

want to do.

But I also want to say just focus on the stated goals.

The situated context kind of context kind of tells you what the design

team is looking for.

Focus on evaluating those things.

You don't have to evaluate every possible aspect of a library catalog.

So if you go to the Hennepin county website you'll notice that one of

the things they also do is list a bunch of events and kind things like that,

that the library is hosting.

That's not mentioned in the situating context so

don't worry about evaluating it.

Just worry about the things that are explicitly called out there just so

this doesn't take that long.

Overall, I'd expect that you should take about three hours on this assignment.

That's kind of what I would aim for.

And of course, that's on average.

So some people may take longer.

Some people may take a little bit shorter to do it.

So I'm really looking forward to seeing what kind of user test plans you

come up with.

And I hope to see you in the next video.

Explore our Catalog

Join for free and get personalized recommendations, updates and offers.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2019 Coursera Inc. All rights reserved.

Download on the App Store

Get it on Google Play