In this course you will learn and practice several techniques for user interface evaluation. First we start with techniques that can be applied alone or in a design team, including action analysis, walkthroughs, and heuristic evaluation. Then we move on to user testing, including learning from a series of usability tests carried out in a real usability lab, and techniques to carry out your own tests even without a lab. Finally, we wrap up the discussion of evaluation--and of UI Design in the specialization as a whole--by looking at the question of how to set and measure usability goals, and in turn, when a design is usable enough to release it.
Assignment Video: User Test Plan
Loading...
From the course by University of Minnesota
Evaluating User Interfaces
103 ratings
From the lesson
Evaluation with Users (Part 2)
We continue our discussion of with-user evaluation techniques (Part 2 of 3)
Meet the Instructors
Loren TerveenProfessor
Computer Science and Engineering
Haiyi ZhuAssistant Professor
Computer Science and Engineering
Lana YaroshAssistant Professor
Computer Science and Engineering
Dr. Brent HechtAssistant Professor
Computer Science and Engineering
Joseph A KonstanDistinguished McKnight Professor and Distinguished University Teaching Professor
Computer Science and Engineering
Hello, and welcome back.
Today, I'll be giving you a little more detail about the assignment that
you'll be doing for this module, the user test plan.
So, first of all,
if it's been awhile since you've watched the course 4 videos, before doing this
assignment you may want to review three videos that are most relevant to it.
The user testing goals, formative and summative.
Just to remind yourself what these two terms mean.
The most relevant video is definitely drafting a user test plan
because it walks you through all of the categories that you actually have to
fill out in your user test plan assignment.
And I also think that the usability lab examples
that we recorded can really be helpful here because they provide concrete
examples of tasks that somebody might give to use with a particular system.
So, in terms of the actual assignment,
you'll be provided with a short description of a situating context for
your user test plan, and you should write a two to three page user test plan.
Now, the key is you want to have all of these sections, because the peer
graders will actually be asked to review that all of these sections are present.
So, first, you want to specify very concretely the objectives of your study,
and I recommend separating these into formative and summative,
I think most cases, you want to have some formative and some summative.
You want to talk about the users like who you will actually recruit for this study.
How many users of each kind.
I think it's important to be specific, so don't just say just general population.
How many people, and what does general population mean?
Is my grandma general population?
Am I general population?
Is, I don't know, my three year old nephew general population?
So you actually want to specify exactly who you want to recruit or
your exclusion criteria maybe.
And just kind of think through that aspect of it carefully.
The setting, where will the study actual take place,
what kinds of tools will be provided to the user, methods and
metrics, so will you ask the user to do something in particular way?
What will you actually measure as they're doing this.
The task.
So again, what is the specific task that you are asking the user to accomplish?
So if you're building some sort of an interactive system,
this is likely some sort of a common task that you might
expect them to do with your system in the real world.
And lastly, researcher roles.
How many researchers are actually needed to run the study and
what their roles might be.
And so the reason we're asking you to do this assignment is because this is
actually a very common task in user experience roles.
So there's some aspect of a system that you may be asked to test.
And the very first thing you need to do is you need to draft out this plan
that will articulate all these aspects of it, all these objectives.
So that you know what will actually happen in your user study.
As we of course know user studies cost money and they cost time so
you want to make sure that you use that time well.
Now in terms of the components I'd say the key things, the things that will probably
take the most time for you to think about and write are the methods and metrics.
So what are actually measuring.
How are you actually going to set up your study in the tasks?
What are the specific tasks that the user's going to be asked to do with
the system?
Now let me kind of go through quickly the situating context that you'll be using for
this assignment.
So I will ask you to pretend that you've been hired as a consultant
to evaluate a new web-based public library catalog.
So I'm not sure if your county or your state or
your country has these kind of public library catalogs.
I will provide you with one that you can kind of use as kind of an inspiration as
you think about this.
But let's imagine that a new web-based public library catalog is being designed
to replace the old web-based library catalog.
And what the designers are hoping for is that the new one will be easier to use,
that it will make it easier to explore and find you books to read.
It'll be faster to finding a specific book if somebody already knows exactly what
they want to find.
The design team also wants to know just in general,
what can be improved about their design.
Where they should go with it next and kind of what are the areas of weakness and
strength there?
So your plan is to draft the user study plan to accomplish these goals.
So including the focused statement of objectives, users, settings, methods,
tasks, metrics, and researcher roles.
So all of these things that I've talked about on the previous slide and, as I
mentioned, there was a slide dedicated to each of these in the drafting and
user test plan video.
So, as an inspiration, if your library doesn't have an online catalog,
you can use my county's online library catalog.
I'm sure all of you can get access to it.
It's the Hennepin County Library, hclib.org.
So, take a look there, just to see,
kind of what generally goes in these and how are they structured.
In terms of tips for succeeding on this assignment.
So, the first thing I say is, take a look at the sample user test plan.
So, I've done this assignment myself, I did it for
a different context, so just to keep it different enough.
But you can see what sections are included and
how much detail roughly is expected for each of these sections.
I think you can just model it on that one.
And if you are providing about the same level of detail you should
get a good grade on this assignment.
You want to clearly highlight each of the required sections to make it easier for
their peers to grade it.
So you don't want to just lump everything, your objectives and your metrics and
your methods and your users into a single paragraph.
You really want to separate them on the page.
Again because you're being evaluated on whether you have each of these sections so
make it very easy for
the person who's grading to tell that yes in fact he's met all the requirements.
Be specific in each of the sections.
So like I mentioned before,
don't just say that your users are coming from the general population,
that's kind of my pet peeve, you really want to be specific about who they are.
In terms of the setting, you might want to say,
is this something that you're expecting to conduct in a lab or do you think that this
would be a study that should be conducted in the actual library?
Where is it going to be done?
Now you won't actually have to run this user study, you just have to plan it.
So feel free to get as creative as you want and
assume you have the resources you need.
Even if you personally don't have access to a usability lab,
you can assume that maybe for the study you wouldn't have access to it.
So if you plan the setting to be a usability lab, how would you set it up?
What would be available there?
Or vice versa.
Even if you don't currently, if you can't travel to Minnesota to go to Hennepin
county library to check out how people would use the new library catalog,
just assume that you would have access to that context if you needed it.
So I think you can be creative and kind of think about the best case scenario and
just having all the resources you want to do the creative usability study that you
want to do.
But I also want to say just focus on the stated goals.
The situated context kind of context kind of tells you what the design
team is looking for.
Focus on evaluating those things.
You don't have to evaluate every possible aspect of a library catalog.
So if you go to the Hennepin county website you'll notice that one of
the things they also do is list a bunch of events and kind things like that,
that the library is hosting.
That's not mentioned in the situating context so
don't worry about evaluating it.
Just worry about the things that are explicitly called out there just so
this doesn't take that long.
Overall, I'd expect that you should take about three hours on this assignment.
That's kind of what I would aim for.
And of course, that's on average.
So some people may take longer.
Some people may take a little bit shorter to do it.
So I'm really looking forward to seeing what kind of user test plans you
come up with.
And I hope to see you in the next video.
Explore our Catalog
Join for free and get personalized recommendations, updates and offers.
Coursera provides universal access to the world’s best education,
partnering with top universities and organizations to offer courses online.
© 2018 Coursera Inc. All rights reserved.
