Carol Rossi on Meaningful Data Collection

Alex Smith: Design Leader Insights is brought to you by Fuego UX. Fuego UX is a leading UX research, strategy, and design consultancy. Hey, Carol. Thank you so much for joining the show today. 

‍Carol Rossi: Hey, thanks, Alex. 

Alex Smith: For sure. And yeah, just as we get started, can you give us a little bit of insight into your journey in UX research? 

Carol Rossi: Yeah. So my journey started, in actually in like large enterprise companies when I was even in grad school, even before grad school. And I quickly moved on to kind of some other smaller companies. The internet became a thing. It wasn't a thing when I first started working. And I realized that I like more of a startup environment. I worked at GeoCities, which usually when I say that people go,oOh, I had a GeoCities website when I was eight or whatever. But I was actually the sole researcher there. And then after the, you know, we had the boom, then we had the bust. I kind of took a little bit of a break from working in tech and did some other stuff for a while, came back to tech, worked for myself for a while doing mostly like contract usability studies. And then at the end of 2008, I thought, I wonder what it would be like to have like one tech job again. Joined Kelly Blue Book to start the UX research team there. I was there a couple of years, then I joined Edmunds and started the UX research team there and I ended up being there for quite a while. And then in 2018, that was, I was still living in LA, 2018, I moved to the Bay Area and joined NerdWallet and took over the UX research team at that point. And now I lead a consultancy focused on helping companies, usually who don't have research leaders, get the most out of customer insights. 

Alex Smith: I want to start with continuous discovery. How should teams be thinking about that? 

Carol Rossi: Well, I think if you're a career researcher, you should be thinking about it and not just sticking your head in the sand and saying, like, make it go away. I think that, you know, this is something designers, PMs, engineers, content strategists, like having direct conversations with customers is something that's been happening for a long time. It just became sort of more interesting recently because of, you know, Teresa Torres's book. But I think that for career researchers, the tendency is to just say, let's not have them do that. Let's get more jobs for us. But I think there's a place for these things to exist in parallel. And I've actually had a lot of success at Edmunds, at NerdWallet, even at Kelly Blue Book for a little while, training non-career researchers to collect their own insights. And I've seen it work really well where, you know, we help them do it better. And so we do some kind of coaching and training to help people eliminate bias from their questions and understand what to do with this mound of interview data that they have afterwards and know, help them know how to make those insights actionable and then how to measure the impact of that. And then rinse and repeat, like, what did you learn from this experience and do it again? So I think that there's an opportunity for us to do some training and coaching for our peers and help them do this work, you know, for themselves. 

Alex Smith: I have a, I have a question though. How, how do we ensure, because there's a lot of methodology that's needed that they're talking to the right customers or the right prospective customers or the correct users. How do you, how do you help teams do that? 

Carol Rossi: It's a really important point and it's actually something that I think can get glossed over in the rush to talk to people. And so I think there are a few things. One is that, you know, in B2C companies, it can be easier to find customers because your customers are consumers. I think with people that are, if we talk about that, really like B2B, very refined kind of customer set. I think that one thing that I've seen work really well, and this is, there are pros and cons to all of these solutions, but is to go through the trouble of developing the relationships with the people that have direct contact with the customers. So customer support sales, you know, BizDev, whoever those people are in the company, spending time developing relationships with them. It's going to make it a lot easier to get some people to talk, you know, get some people that you can talk to. I think another thing that I've seen work really well is just to have a panel of customers that are willing to be called on, like, and make it very specific. Like, we're not going to come to you more than like for an hour a month or something, but we do, you know, hope that you're going to give us that time. Like, but I think it's really also important to think about concerns around privacy and making sure and security and making sure that you're not skimming over, like just putting a prompt on your website or in your app or asking your customer support team to just reach out to somebody without ensuring that their data is going to be managed well. So what are you going to do with that information? And for B2C, it's the same concern, right? You want to make sure you're not violating any laws. And you also want to make sure that you're not, especially with sensitive topics like FinTech or even, you know, big purchases that people are making, like they're going to be telling you a lot of information about themselves. What are you doing with that data without overdoing it? That doesn't mean we don't talk to people until we have the perfect system in place. It means we think about what kind of infrastructure we need, what kind of tools we need a little bit ahead of time. But I think there are things we can do to mitigate some of those problems rather than just throwing our hands up saying we can't find the right people. I also think it depends on what you're doing, like If what you're trying to get is very upfront, like, should we build this thing? You really need more people that are more closely aligned with your target audience. If what you're trying to do is find out, do people understand, or can they, can they use something that's maybe, especially in a B2C environment, more general experience of like usability, there might be possible to let go of some of those really stringent requirements about who your target audiences to get feedback, but it just depends on the situation. 

Alex Smith: I think a lot of product people are like, yeah, I'm going to go talk to some users. Get some insights. Which is great. It's a good starting point. How do you actually institutionalize that? Like, make it a repeatable, sustainable, non-biased process that is driving some sort of KPI where they see the value in it in future product iterations?

Carol Rossi: So, I've seen this work really well and I've seen it work really badly. And the places where I've seen it work really well are the environments where there is a regular cadence of like some kind of training. And so what we instituted at NerdWallet, what we did at Edmunds, what I've seen work in other places is this kind of like, maybe it's, you know, a few hours of like half a day at the most of like, how do you set up a study for success? How do you plan a study? And there's a template. People are, you know, we kind of talk about how do you ask questions and how do you set up tasks that are not biased and what do you need to think about in terms of the way that you interact with people if you're doing this live? Or if it's going to be unmoderated and you have a tool for that, how do you set that up in a way that you're going to get, you know, so like setting things up for success and then actually running a study, whether it's live interaction or setting it up in the unmoderated tool is another kind of session. And then a third session around distilling all of that data into actionable insights. And the team, the way that I've seen it work best is that, you know, every quarter or every whatever period, depending on what the hiring cadence is, any new people coming in on who have that role of product manager, designer, content strategist, anybody that might be on the front line of having conversations with customers goes through that program. It's like a couple of weeks, you know, three afternoons or whatever. And then there's coaching. Then it's like, oh, let's see how this is actually working when you're doing it in the real world. And gives feedback on that and questions that people have once they're actually out there. I've seen it work really well in that way. I've seen it work really badly where some version of that isn't done. 

Alex Smith: Yeah, that makes sense. Carol, what advice do you have for new researchers that are entering the field today? 

Carol Rossi: Yeah, I think there are a few things we need. One thing is that we need to embrace AI tools and figure out how we can use them to help the research mean better. And of course, that's changing constantly. So, you know, staying up to date on that is a task on its own, right? We need to be mixed methods researchers. I think gone are the days when we can say I’m qual, or I’m quant, or whatever, because the industry is looking for generalists. And so of course, we might be more prone towards one method or one approach than another, but we need to be embracing that. We need to be looking at across at different kinds of data and be able to either partner with people in analytics, you know, behavioral science, other teams, or even know how to dig into some of that on our own. And I actually think if I were starting now, I would do a much deeper dive into behavioral science and take that arm as another focus area, because I think that it's, you know, companies are looking for more quant, like then maybe they were during the design thinking era of the 2000. And so I think the more we can bring to the table that shows that we can do all of these things at some level. And again, it's always like you know, T shaped right. I know something about all of these, maybe I specialize in one or another. But yeah, I think it's really important to show breadth. Or impact. I think another thing that's really, really important that I would encourage anybody to think about is from the very beginning, think through how you're demonstrating in the work that you're doing. And when I think about impact, it's, you know, at a few different levels, it's product impact. Like the things that we often think about as impact. What kind of rift are we seeing? What kind of revenue? What kind of customer satisfaction? You know, lift all those things. It's also organizational impact. How well are we bringing everybody along with us? Are we training other people to do that research? And are they seeing a benefit to working with us as researchers? 

Alex Smith:Thank you so much. Where can people go to learn more?  

Carol Rossi: Yeah, I mean, very simple. You can go to my website, carolrossi.com or find me on LinkedIn. 

Alex Smith: Awesome. Well, thank you so much for joining the show today.

Carol Rossi: Thank you.