Hello, I’m David Travis and in this episode
of the UX Tea Break I answer the question: “What does good look like when it comes to
Discovery research?” So this question comes from Brett Parker.
Brett asked this: “I’m a product owner and I don’t know a great deal about user research.
I have a user researcher on the team who’s doing discovery research for us. He seems
competent but how do I know if his work is any good?” I thought this was an interesting question
because often most of us have got some kind of background in quantitative research: enough
that we can have what I like to call “quantitative literacy”. So for example if I showed you
the results of a survey you would ask me questions such as, well, you know, “How were the people
sampled for this survey? Were they representative?” You’d also know, if I gave you the data in
Excel, how to create an average and a standard deviation and so on. You kind of… most people
know about the basics of quantitative research because we’re taught it at school. But when it comes to qualitative research
that’s something that often people find a lot harder to judge. And I thought I would
give you four criteria that I would use to decide if the piece of Discovery research
that you’re looking at is any good. First of all, I should define some terms.
By ‘Discovery’ research I mean research that’s carried out early on in design in order to
understand the user needs for a particular product or system. Secondly, I’m talking about
digital products and services. I’m not talking about things like, I don’t know, toothpaste.
But if you’re doing research in that kind of area, then… well actually some of these
things might be useful but it’s not really about that. And I’m also not talking about
qualitative research generally. I want to emphasize I’m focusing on Discovery. So I think there are four criteria you could
use, Brett, to decide whether or not the research is any good. So the first criteria is to look
at the research and ask, “Did it use small samples?” And by small samples I mean somewhere
between five and… I don’t know… 20 users. In contrast to say a research study that might
use hundreds of users. And the reason we want small samples in qualitative research is because
we want to go in depth. We really want to understand the range of experiences that people
have so that we can fully understand the space that we’re designing for. We’re not interested
in creating a kind of an average like you would be if you were doing quantitative research.
So that’s the first one: it uses small samples. Second and kind of related to that is that
the research focuses on behavior. What I mean is what we’re interested in is what people
do rather than what they say. So we’re not so interested in opinions. It’s not that we’re
not interested at all in people’s opinions and what they say it’s just that it’s very
risky to design digital products or services based on people’s opinions. Instead we want
to see what they actually do. So that means in the research that you’re trying to evaluate
look at how long the researchers spent with the participant where the participant was
actually doing something — the meaningful activity I like to call it. If the research
session was about an hour in length I’d like to think that the participant was doing stuff
— demonstrating the way they go about doing the meaningful activity — for about half
an hour or so. So about half the time, as a kind of a rough rule of thumb. Related to that is the third principle which
is that the best kind of discovery research is done one-to-one rather than in groups.
The reason you want to run one-to-one research is you want to get this in-depth understanding
of people and if you run people as groups it’s much harder to get that understanding.
Because often what people will do is they’ll coalesce around some norm or description of
ways they behave. And you won’t understand the differences as much as you would do if
you went one-to-one. Then the fourth criteria is that the research
that you carry out is carried out in context. So by ‘in context’ I mean it’s carried out
in people’s homes or places of work, as opposed to say, remote research mediated over the
internet, or pop-up research in a cafe, or lab-based research in a studio. So if your
users do this meaningful activity in their homes, that’s where the Discovery research
should take place. If it happens at work, that’s where the Discovery research takes
place. If it happens in their car, that’s where the Discovery research should take place.
So the purpose of that is to make sure that you’re in the user’s context. And the reason
that matters is because first of all people forget things that they remember when they’re
in context. They forget that they do things in a certain way. They forget about other
devices they may use or other applications they may use to achieve their goals when you
see them out of context. But in context, when you can ask them to do stuff, those things
act as triggers to remind them of exactly what it is that they do in that particular
situation. So those four criteria: first of all, small
samples. Secondly, it should be behavior-based research rather than opinion-based research.
Thirdly, it should be one-to-one. Fourthly, it should be in context. If you can tick all
of those boxes it’s going to be very hard to do poor quality Discovery research. All
of those four are really good indicators that the research has been done well. Finally, I should say that I’m not saying
that every piece of Discovery research needs to hit all four of those criteria. It could
well be the team have a very specific question that you could go out and answer by speaking
to some people in a cafe or somewhere local. By all means go ahead and do that. It’s just
that that shouldn’t comprise the majority of the Discovery research. The majority of
the Discovery research should hit those criteria that I’ve listed. Well, I’m sure somebody disagrees with that
so let me know in the comments. What resonated with you? What things did you disagree on?
What other criteria would you use to decide that Discovery research is any good? And of
course, as always, if you liked the video please give it a thumbs up. That way, YouTube
recommend it to other people. And subscribe to the channel.
Ultimate Blogger Theme By Buywptemplates