When it comes to user research, you won’t get a more honest and unique insight then when observing and interviewing users in their own natural environment
But of course this comes with its own set of unknowns.
It’s a cliché so overused that you be forgiven for not paying attention to it, but ‘expect the unexpected’ has to be at the forefront of any user researcher when they’re out in the field. The unpredictable nature of human interaction, as buffeted by the winds of chance, circumstance or an overly-friendly labrador, can challenge even the most hardy individual.
One such seasoned and ‘battle hardened’ veteran is Steve Portigal, a user research consultant who has spent the last 15 years facing all the challenges that being in the field research. He’s also been collecting his very favourite ‘war stories’ in a newly published book ‘Doorbells, Danger, and Dead Batteries’. A fascinating collection of bizarre, funny and occasionally heartbreaking tales from the user research frontline.
I caught up with Steve to discuss the challenges, joys and horrors of user research in the field as well as his new book.
Could you tell me a little about your background, and what drove your interest in user research?
I discovered HCI (Human-Computer Interaction) at the tail end of my computer science undergraduate program and I loved the idea of something less theoretical and more about the people that use computers. This was before the web, and before we had the term UX. I ended up working in Silicon Valley at a traditional product design firm where services like designing software, or learning about users were just starting to emerge. I really fell in love with the post-fieldwork part of the process, something I was ‘permitted’ to contribute to before I developed any actual fieldwork skills.
What are the fundamental skills needed to be a good researcher?
At a fundamental level, there’s an inherent curiosity about people (which may be more of a leaning than a skill). There’s the ability to deeply listen. We need patience. We need to think quickly on our feet, to be in the moment and creative in how we ask questions.
Your book ‘Doorbells, Danger and Dead Batteries’ is a collection of war stories from the world of user research, collected from real life researchers. Are there common themes between the stories?
That’s how the book is structured, actually. Each of the 11 chapters considers a particular theme, something that is of particular challenge to researchers. Some of these are obvious (such as participants we have problems with, or the potential to end up in a dangerous situation) and some are less obvious (encountering not-safe-for-work content in the field, or dealing with our own emotions as well as those of our participants). The chapters begin with an essay by me, include a handful of different stories about the theme, and then wrap up with takeaways that researchers can use to develop their own practice.
You say that improving research skills is about coming to grips with our own ‘flawed humanity’ – how so?
Pulling off a research program is an enormous logistics exercise. Coordinating materials, participants, times, locations, stakeholders, incentives, recording equipment, and on and on. When you work inside a commercial enterprise, there’s a desire to optimize processes, create spreadsheets, build checklists, summarize objectives, deliver key takeaways, etc. But the truth we may forget is that research is an activity that researchers perform with participants. People who make mistakes, people who come to the session with something else going on in their lives, people who have emotions, people who have different verbal abilities, people who have different expectations of the session, and on and on it goes.
While the optimizing efforts are important, these will always be person-to-person interactions. You can fight that and always be frustrated or stressed or disappointed, or you can embrace that as part of the joy of doing research and a source for richer learning. Learning to overcome the pull of frustration and to find a way to actually embrace it is where we can personally grow in our practices.
What if you’re presented with a participant who you have an inkling isn’t being truthful – how do you deal with that?
I love the word “inkling” because it captures that feeling so well. I can think of two different untruthful situations. The first is when you feel they have misrepresented something in order to be in the study. One story I’m hoping to get written up someday involved someone bringing neighbours into the home to pose as their children so that they could complete the session. Typically, you don’t actually know if someone isn’t telling the truth, but you start to feel uncomfortable because something just isn’t right.
Maybe there’s a misunderstanding? It causes me all sorts of self-doubt and I usually just to set it aside and leave the “resolution” until later.
The other situation is when someone is telling a story with an excess of grandiosity. I have developed a strong sense of when I am convinced that person is making up the story; I still don’t have any way to prove it, but those moments when that happens are key moments when something truthful is being revealed.
It’s not about judging the participant’s ethical decisions; it’s about a deeper perspective into what that person is about. It reveals more about the dynamics of the interview and the emotional content of the subject. I don’t care why people lie to be part of the interview – it’s likely about the money – but I do care why they might lie in a discussion about their behaviour, beliefs, expectations, values, etc.
Do you prefer fieldwork or a more controlled environment? What are the disadvantages of either?
Fieldwork requires a bigger effort. A more controlled effort is going to be centralized, either your facility or a third party environment where you can set up and have people come in, one after another. It’s going to be more efficient. And maybe a little soul-crushing?
I believe it has a real impact on the people we learn from that we go to them. That right there says something about our footing, we’re on their territory. I love the serendipity that affords; we see a lot of the details of their context and that gives a great perspective on how and why the things we are interested in are happening.
What have been the most difficult conditions under which you’ve conducted user research?
The worst situations are less about participants than what’s happening with the team. I’ve found myself in collaborations with other consultants where the communication was poor (to be kind about it). In those situations, where my best efforts to create alignment around objectives and to set up a protocol for the research that we’d all agree on are thwarted, it’s very challenging to do the work.
I once travelled to another state for a kickoff meeting, where despite repeated requests, they would not share the agenda at all. At one point, I was driven to a suburban area and told, basically upon pulling into the driveway, that I’d be going into the house to interview someone! They dropped me off, with no research plan, no objectives, no preparation, and then pulled away!
Do you have any tips for being fully prepared for any eventuality – or is that even possible?
The way you phrase the question suggests you know the answer! You can’t possibly prepare for everything (my favorite illustration of this is one of the earliest war stories, where Dan Soltzberg kneels in cat urine). But what you can do is acknowledge the limitations of preparation and be prepared for the world to surprise you. That can help you deal with those surprises as just part of the journey.
Why hasn’t user research adopted failure analysis into its practices, the way that other industries have?
Leadership maybe isn’t that interested in the frayed edges around the user research practice, in terms of what may be revealed. And researchers struggle with credibility – “You only talked to HOW many people?” “Well, did anyone say they wanted [feature] with [situation] and [context] when they were someone with [criteria]?” – so it may be professionally risky to have these conversations. It’s up to leadership to create a safe place for these discussions. I hope that the stories in the book are a big first step and that researchers – and their colleagues – will take the next step.
Do you have a favourite war story?
That’s like asking someone who their favourite child is! I love them all! They all have something to teach us; some have an ‘obvious’ lesson and some just reinforce the important fact that, hey, shit happens.
My other answer would be that I love the most recent war story the most… it’s been exciting to see some new stories come in since the book came out. There are more stories out there and I’m very excited to keep collecting and sharing them. The latest are on my site.
Main image: ‘Doorbells, Danger, and Dead Batteries’ book cover, courtesy of Rosenfeld Media.
The post Interview: Steve Portigal on user research war stories in the field appeared first on UserZoom.
