How to Combat Bad Surveys and the Replication Crisis Feat. Sarah Wittman
Sarah Wittman is a talented and thoughtful research leader who’s led teams in-house and in-agency. She’s always thinking about how to best position her team for successful work, which has led her to think really deeply about what makes some surveys so bad, and how to deal with the replication crisis.
Read on to see how Sarah deals with these difficult problems! Or watch her series and Thought videos at joinlearners.com.
Sarah’s role is Director, Design Pathfinders at RBC, where she focuses on scaling the impact of design research, and supports design researchers embedded in cross-functional teams. A lifelong reader and collector of facts, she loves the opportunity to find parallels between disparate industries and situations.
- Survey Writing
- Team Management
- Research Strategy
Let’s take a look at some of Sarah’s work!
Most Popular — Survey Writing
What it’s about: Surveys are a powerful tool—but why are there so many BAD surveys out there, and what makes them fall short?
In this talk, Sarah explains her concrete strategies on how to write a survey, and how to have confidence in your results. Learn how to apply the practices you already use in qualitative work to your surveys, and find out how the quality of your respondents may be impacting your results.
“Respondents are used to seeing really bad surveys, and being invited to really bad surveys. And what happens then? Respondents are less likely to actually fill out any surveys in the future because they have such bad experiences ”
“The researchers who don’t have a mentor are seeing examples only of bad surveys, and very few examples of good surveys that they can learn from and figure out how to ask questions better so they can make a meaningful impact.”
Trending in User Research — Escaping Your Rut
What it’s about: This Thought is all about recognizing and escaping your ruts. What happens when you lose that spark of joy in your work? What kind of thinking emerges, and how do you recognize the issue? Sarah grapples with these questions and offers a deceptively simple solution.
“Try playing a bit more. Try experimenting a bit more—because otherwise, where are you growing—and where’s the fun in that?”
Learners’ Choice — The Replication Crisis
What it’s about: Over the past decade, academics in social psychology have been wrestling with a problem: when they tried to reproduce studies that had been published in academic journals, a worrying number weren’t seeing the same results. This is not merely an academic issue.
In business, there is rarely the chance to repeat a study, but there is little reason to believe UX research is immune to this problem. The UX Research industry can learn from what was uncovered in social psychology, and create guardrails to increase our confidence that our work will stand the test of time.
“As researchers, we need to understand that sometimes the things that we understood from the beginning aren’t going to hold up through iteration.”
“I’d like you to imagine that somebody exceptionally generous has just come to you. They’ve got a lot of time, maybe they’ve got some money. And they’ve offered to repeat the last 28 of your studies—very generous of them…A little bit scary mabye for you? What are they going to find? Maybe they’re not gonna find the same things you will.”
Thanks for reading! We’d really appreciate you sharing this blog using the buttons below! If you want to discover more trailblazing user researchers and designers, check out 5 Stories From The Design and UX Research Frontiers.
Alec is a co-founder and the CEO at Learners. He’s worked as a User Researcher at Sciencescape, TWG, and North—before his time as the head of Product at Finaeo and then CEO at UXR Collective. Alec and the other Co-Founders of UXR Collective have long noticed how isolation in the UXR community made growth and connection difficult.
It’s that isolation that ultimately drove Alec and the Learners team to work towards a world with more accessible learning opportunities for everyone.