Faster Feedback | Rob Holland from Feedback Loop

DISCLAIMER: Below is an AI generated transcript. There could be a few typos but it should be at least 90% accurate. Watch video or listen to the podcast for the full experience!

Rob Holland 0:00
We really believe that it’s important and critical to bring in the real voice of the customer, the voice the consumer into those decision making processes. But early on in the middle, or towards the back end of the innovation cycle.

Alexander Ferguson 0:19
Welcome to UpTech Report. This is our applied tech series UpTech Report is sponsored by TeraLeap. Learn how to leverage the power of video at teraleap.io. Today, I’m very excited to be joined by my guest, Rob Holland, based in Chicago, and he’s the CEO at Feedbackloop. Welcome, Rob. Good to have you on man.

Rob Holland 0:35
Great to be here.

Alexander Ferguson 0:37
Now, what you guys provide is a research technology platform for rapid, quick consumer feedback. And your aim, if I understand correctly, is to provide reliable consumer insight for those product managers and researchers out there. Your mission statement that I pulled from your site says to help companies thrive by learning faster and innovating smarter. Tell me, what does that mean?

Rob Holland 1:02
That’s a great question. So you know, the way we think about it is the world is just changing. So fast, and the consumer is changing so fast, and it’s really, really hard, it’s getting harder and harder for companies to just keep up and stay connected to their consumers and their customers. So one of the big things in that we found is a real benefit. And a real need is there’s so many decisions that have to be made faster, and they have to be made. And they’re really, they have just be implemented very quickly. The data revolution, so much is made on gut feel, or there’s an abstraction from the end customer. So we want to be able to provide the right type of research the right type of consumer insight to product owners, to researchers, to other business owners, so that they can make decisions faster.

Alexander Ferguson 1:53
Let me ask you before, I mean, people always needing insight, right, they were always needing to be able to for leaders to be able to make the right choices with the right data they had before, what were they doing? Like? What is the typical approach? And how are you guys approaching this to solve this problem.

Rob Holland 2:09
So I think there’s two things going on what typically, what they, what companies would do is, they’d go to research experts to actually help them solve a problem. And those researchers would run studies or programs, or go to research agencies to actually conduct studies. So these would take weeks, and they would cost a lot of money. So it would be really you’d have a full a few bullets, and you’d be able to use them really judiciously. Otherwise, what you’re doing is what we’d love to call is dot voting. And we’ve all been in the in the conference rooms when we’re doing a brainstorming session, and everybody puts a bunch of things on the whiteboard, and then you get a bunch of sticky notes. And you can just everyone goes and votes on what the best idea is, and you go and many, many big decisions are made that way. We really believe that it’s important and critical to bring in the real voice of the customer, the voice the consumer into those decision making processes, but early on in the middle or towards the back end of the innovation cycle.

Alexander Ferguson 3:06
Are you saying that, you know, the traditional approach would take six 812 months to do the research, you finally get it back. And then you can use those points really quickly. But if you want to make faster decisions, there is another option. So like, if you have a meeting on Friday, you could have solutions the next week, or answers insights the next week.

Rob Holland 3:25
Absolutely. And that is it would take days weeks, it would actually take weeks or months typically to get studies done. And that is too slow. Right. So you’re basically you’re flying blind because you have to you have to make the decisions by time you get the research back, the decision point is past. And so the luxury of time is just becoming less and less. And so it’s really important to get that information. If you have if you have a have to make a decision next week, you can run a study and within two to three days, get all the information you need back come in and formed and at least make a more informed decision on how to move forward.

Alexander Ferguson 4:01
So let’s let’s take an example here use case we’ll talk about where we are right now. COVID pandemic, being able to make decisions based off of the available data faster is going to be important. How can you How could you apply in this type of scenario what you guys do.

Rob Holland 4:20
So a real good example one of our our customers, we work with the Ad Council, and the Ad Council works with a whole bunch of brands. And one of the big initiatives that ad councils is working on is how to get the population today comfortable with vaccinations. So they’re working with all these different brands who are trying to put together all these programs and ad councils helping to bring that together and synthesize it. So they’re testing different types of messages down to what’s the message, what’s the tone, what’s the language that’s used for different consumer segments because not one message resonates with everybody. And they’re able to put out a number of concepts and run those tests in very rapid order. get immediate feedback that they can then go down a path work or stop and change course in real time and do that in a really urgent environment where an initiative has to get taken out

Alexander Ferguson 5:14
against doing something in the old way, would take weeks and weeks and months. But it doesn’t have to be that way. In this

Rob Holland 5:23
Yeah, either would take weeks and weeks or months. Or it would also the alternative was you’d be in a room and everybody be brainstorming ideas and your dot vote. And then it’s you what you’re doing is you’re getting you’re getting experts opinions, but you’re not really getting the voice of the customer, the voice of the consumer that you’re really going after, and bringing that direct voice into technology today is so much it’s possible. So the the the ability to provide that is is game changing.

Alexander Ferguson 5:52
For the typical companies you work with is definitely the b2c area not so much in the b2b because yeah, it’s when it’s you have a lot of consumers, a lot of people you’re needing to be able to get insights from helped me understand another potential use case, I feel like you work with all size companies as well like this one small 100 person shops to like what’s the range.

Rob Holland 6:16
So we work from from startups that are trying to go direct to consumer, for example, all the way up through fortune 100, fortune 50 customers. So we really span it, we really cut cover the span of all different needs, because the universal need is consistent. If you’re working with consumers, even if you’re a b2b, b2c company or but if your end end, endgame is consumers, there’s just an insatiable amount of questions to be had. We do work in healthcare, we do work in financial services, consumer finance, we’re doing work with insurance, we’re doing work in media, you think about streaming changes in media consumption, we’re doing work with CPG. And then a whole host of other industries as well.

Alexander Ferguson 7:05
That this concept, you’ve referenced it twice now I’ve just yeah, on a on a whiteboard, just dotting the opinions of what everybody has, and then you make a decision is some could argue that your guts or innovative ideas is better than consumer feedback. What would How would you reply and respond to someone with that with that type of feeling?

Rob Holland 7:25
I would say that’s a fair point. So why not test both, because it’s easy enough to do both and then see if there’s an inconsistency. And if there is, then test again, to understand why and if there isn’t, go.

Alexander Ferguson 7:38
It’s like more data and more more insight doesn’t necessarily a bad doesn’t mean a bad thing. To compare your gut reaction.

Rob Holland 7:45
That’s right, it fit if as long as it’s fit, we want it to be fit for use, which means it’s appropriate, it’s scaled, and it is it is sufficient for the need at hand. And there are many cases where there’s sophisticated research that’s very complex. And that is, you know, takes time. This is total, appropriate use cases for that. But many, many, many decisions are being made without any input any data. And we found a way to really bring the voice of the customer into those decisions and do it in an iterative way so that you can move forward and just go and progress.

Alexander Ferguson 8:22
Let’s talk about your actual technology platform itself. How does it work? How and how is that different from that, again, you’re going to research organization or company that goes off and does it versus a platform that someone themselves can use a research analyst.

Rob Holland 8:38
So we our platform is usable, designed to be used by researchers or business owners with their product, product people or marketers or even in innovation and insights. So we kind of have a business arm which we tend to call product. And the consumer insights are research arm so but it’s really meant to democratize on all ends. And we try to make it fast, easy and reliable so that it can be used by the businesses with the sanctions of the researchers. Or it can be used by researchers more quickly to support and keep up with the business needs that are out there. So that’s the underlying principle. What we’ve tended to do is when you there are multiple steps to create a study and we do really we’re focused in on doing agile surveys, consumer surveys. So that’s really what our platform sweetspot is at this point. And so what we’ve done is we’ve decomposed the different steps that you would take to, to to run a project from creating a test hypothesis to designing a questionnaire to figuring out what type of audience to source and response to creating the survey to running the survey to aggregating the data, to publishing the data in a dynamic way quantitatively and qualitatively. And then being able to distribute that information. So we’ve taken all those steps and integrated them into one technology platform that can run it in To end and do that in very rapid, rapid fashion. So we can create a study and kick it off, go find 300 500 people, get them to respond process through that and turn it all back usually inside three days, sometimes less.

Alexander Ferguson 10:15
Three days or less being able to do all that. What’s the how we understand that the history and how is it? The company changed over the years? And where is it headed going from here?

Rob Holland 10:28
It’s a great question. So we, we’ve been on this journey for about five years, our founder, Thor, Urmston, he grew up in the game, and he was a coder for gaming. And so talk about rapid development, agile development, that’s where the whole agile product development really began. And just getting rapid feedback, deploying and adjusting in real time to make decisions on when to take product. That was his idea. And the research, typical research just couldn’t you couldn’t even play in that space. So he figured that that’s, you know, a big need with digital transformation, changes in consumer technology, the generational shifts that are just going on these mega trends, that are just changing the way people react to technology. And the way data is used just everywhere, he saw a need here that there’s a greater demand for insights than the current supply chains, projects, really technical, heavy duty studies can provide, there was this gap in the market, and he saw that is a need unmet need. So we started with that as our DNA. And then what we’ve done is we’ve just started to evolve it and bring it out to where it’s not just teams that are sort of on their own, doing their own business teams doing their own thing, whether they’re product teams or marketers working on messaging, the researchers started to connect to that as well, particularly as we started gaining traction. And they wanted to understand well, what is this? Is this sort of Rogue research, so to speak? How do you think about that, and it became clear that for our best customers, we started to see the research teams in the business teams were collaborating, and the research teams were able to embrace the platforms in the in the whole agile research concept and, and really engage it. And when we started making that connection, it really opened up the whole the whole business for us to start thinking about transforming the way research is done, not just on the outer edges, but really at its core, how we

Alexander Ferguson 12:30
understand it typically for the research department and the business side, do they deny not generally get along or talk enough? Or or is there enough integration?

Rob Holland 12:40
You know, there’s there’s a whole spectrum here, but there’s a natural tension, if you think about it, so the businesses are trying that their tasks, which is getting their jobs done, and trying to move as quickly as they can, researchers, that’s a really important function for them. They’re, they’re smart, they’re experts, they can help provide really valuable inputs in direction helps shape direction, when it’s done well. And they also have a fiduciary responsibility to make sure that the businesses aren’t just running wild. And that does happen. I’m a line operator, I’ve had the opportunity to run a little wild, I’ve also been on the research side. And so like there’s a bit of a natural tension. In many cases, the the traditional conventional research philosophies and approaches tend to be a little bit more conservative, a little bit more thorough, a little bit more, just precise. And sometimes you just need about right or good enough, or you need to have some information to guide you to keep moving and you can’t wait. And so this is tension of can’t keep up flying blind business of flying blind researchers can’t keep up that creates a tension. And what we’re trying to do is create a platform, what we are doing is creating a platform. And as part of a bigger movement towards agile research where we can bridge those gaps and create positive feedback loops within those organizations, as well as feedback loops between businesses and their consumers.

Alexander Ferguson 14:12
Are you moving towards a almost a collaboration platform for researchers and business folks to be able to be in the same place? Is that Is that how it division? Is

Rob Holland 14:21
it? That’s actually what we do? So yes, and I think that’s really going forward, really building deep, more more complex or sophisticated, agile research, collaboration tools to allow business and the research side to come together and to really make the best decisions they can for their customers and their businesses.

Alexander Ferguson 14:44
What a researchers will start with them from what do they really care about? And, like, what are they looking for?

Rob Holland 14:51
They really want to make sure that when a decision is being when data is being presented to help inform a decision, that is is quality data and it’s really not false positives. Really, that’s the key, it just comes down to that at the end of the day, that’s the primary objective. So they want to use the best information that can possibly be brought together to make a business decision. And best doesn’t necessarily mean deepest, biggest, most thorough, it’s the it is really the right amount of data. To make the decision at hand, given the need for speed, the actual table stakes of that decision that are one way door to date way doors is is basically like to talk about, if they’re reversible, if they’re irreversible, if there is a certain amount of investment behind it or not, you know that so there’s really you have to kind of gauge it, and what and we’re trying to create more and more tools and platforms so that there’s an easy way to collaborate and align on that and get the right use our tool for those those purposes, which there are many of where there’s nothing right now. So people are voting and flying blind, or or they’re not moving fast enough, because they’re waiting for something

Alexander Ferguson 16:08
so fast, you know, this, this is the hard question or not, but how do you fight against false positives or being able to ensure the accuracy of data when when you’re being able to produce it at this speed?

Rob Holland 16:19
Well, that’s that is a really good question. And so, you know, what we think about is we are looking at at our surveys typically will collect information with between 100 to 300, maybe 400 people, right? So so we are using sample sizes that are statistically relevant for the types of surveys we’re doing, we keep our we keep our questionnaire is pretty tight. Usually it’s around 10 questions, or last weekend to crop qualitative and quantitative, so that there’s a balance there. And we have a research team that that also works and supports our customers, and can help think about survey design and quality we and we bake that in, we also go very much after quality inputs. So we have a lot of quality checks in the data, we get in the respondents to make sure that there are viable respondents and they’re appropriate, and we do a lot of machine learning to kind of clean the information and bring it back. So you know, we really are we take that that responsibility is critical to what we do. And it’s table stakes. So where we’re focused on that, quite

Alexander Ferguson 17:24
you’re utilizing sounds like with technology, both of this the platform for for gathering and bringing it in and using machine learning to to be able to process it faster. But so it’s using technology as a platform, but it’s always monitored and and insured by people, real people inside your company that is working, and you’re helping them even put the right questions in the first place. I get that right.

Rob Holland 17:49
Do that. Yes, we do. And we also have an open back end. So that if the if the the end user, whether it’s a business user or researcher wants to go in and dig, dig in, and they they’re not comfortable with some of the responses, they can pull those out. And they can do their own cleaning and editing, and put their final quality checks, because they all have different levels of tolerance. And so we provide that flexibility as well. Our business leaders

Alexander Ferguson 18:17
out there. What advice or word of wisdom would you be giving them in light of today’s environment, a world we’re in when it comes to being able to have agile data and insights fast? What advice would you be giving?

Rob Holland 18:33
Wow. So I think the the most basic thing I would say is these needs existed. Because of the mega trends that I talked about digital transformation, generational change technology, consumer technology is just it’s just changing the way people live. And that’s been happening for an accelerating for a long time. Now that now that we’re in the COVID, and post COVID world, it’s literally like a tsunami that’s hit. That’s that every single human being in the face of the earth has some change in their life. So there are definitely going to be changes and they’re not linear. And they’re not necessarily obvious and there’ll be unintended consequences or cumulative effect changes as well. So for the foreseeable future, anyone who’s working with consumers needs to make sure in the in the it’s critical to make sure that you’re watching and you’re and validating your core assumptions about your customers and your product and your and your offering and their needs. And then you’re looking for an identifying for new opportunities. I think of it as like imagine you’re on a surfboard and you’re trying to run your business as if you’re on a surfboard, the water underneath you is unstable, you’re trying to stay on it. You’re watching over your back to make sure you don’t get crushed by a wave and you’re keeping your eyes open for that next wave that’s coming in to make that quick move you need have tools, business processes and capabilities to enable you to do that. And agile research is a massive enabler. If you’re making decisions flying blind, if you’re not voting, if you’re not getting direct feedback from consumers and your customers to make decisions that affect them, you’re missing out, you’re missing out on risks, and you’re missing out on opportunities. And the capability is here now to be able to enable you to do that I wouldn’t accept flying blind anymore.

Alexander Ferguson 20:31
Going forward, how? What do you see it looking like how frequently should you see a business leader doing these agile research? insights? I mean, how often is it? Is it monthly? Is it weekly? I mean, what does it look like?

Rob Holland 20:51
It is really situation specific, we have a great example. In in, I’ll give you a great customer example. So it is as much as you need it to be, but I’ll find that now give you an analogy as well. So um, so one of our best one of our customers, Farmers Insurance, they’re 130 year old plus company, typically go through agents to get insurance. That whole insurance world’s been turned upside down with direct to consumer, you’re probably familiar with lemonade, and they created an app. And the goal, the traditional insurers were kind of caught by that in some way. Right. And so farmers spooled up an innovation team called the toggle team, and they went out the innovation and they create a toggle, which is their app. And they they basically had to go direct to consumer and break 135 years of history. And they had to do it in rapid order. And so they we partnered with them. And we were fortunate to partner with them in it in a in a matter of six months. They ran over 145 tests, they talked to hundreds of 1000s of customers and consumers and they were able to start to really just understand things from what should we call our brand name to different language we’re using going after a millennial and Gen Z, how do we go direct to consumer? How do we change language in the contracts? How do we make it easy for someone to understand, and they cut that development cycle down by over they said they saved over over a year’s worth of development time. And they made really important decisions that they would run rapid order. So sometimes you’re running multiple tests in a week. And sometimes you’re you know, you’re coming out of a brainstorming session, you come up with 30 tests, 30 ideas, you test those ideas, you get it down. So it really does flow. And the way I think about it is imagine if you were going you when you’re going on a journey today in a car. So now we all go on Apple Maps or Google Maps, you plug in your your path to get it all set up, and you’re going and you get on the road and suddenly it starts raining, raining really hard. You have your map, you know where you’re going. But you need suddenly you need more feedback, what do you do, you turn on your windshield wipers. And you get rapid, instantaneous, not perfect feedback. But enough to keep your eyes on the road to keep the window clear to make sure you don’t crash into anybody run anybody over or go off the beaten path. So you have this intermediate feedback, that’s helping you get this bigger journey. And that’s really what what I think the real power of this is. So if whenever you need you have the ability to do what you can, as frequently or infrequently as the business requirements are you have a tool at your disposal, where you can just turn it on and go, you don’t need to do big heavy studies. You don’t need to go outsource with a bunch of a bunch of third parties, you don’t need a subject matter expert that’s so technical, that you can’t move forward, you remove the roadblocks to be able to have that ability to be as agile as you need to be to ride that surfboard. And make sure you find that next wave and don’t get crashed by something underneath you because you don’t see it.

Alexander Ferguson 24:15
I love that analogy. By the way. What is the most exciting feature? Or what can you share on your roadmap of where you’re headed from here that you’d want to share?

Rob Holland 24:27
You know, it’s actually that’s it. I’ll talk about something that we actually discussed today at my executive team meeting we had a team come in and what we’ve done is use some really interesting machine learning and data science to really take natural language processing and sentiment type of analysis to another level. We call it we’ve introduced a feature called broad search. Typically when you do search, what we typically in our surveys, there’ll be a number of quantitative questions where you pick like one, rank one to five or yes nose, and then you’ll get some open ended questions where you get, we get responses and and you do that with two 300 people, you get a lot of responses. And so one of the big challenges of doing consumer survey work is to kind of call through all those, all those, those responses and figure out what’s going on. So we have a search capability in our platform, that’s a literal search. If you type in, you know, type in a word, we would talk about travel example. So it was like safety. And you get, you get some some sort of response to anything that says safety in it, right, or maybe another word or two that’s familiar with safety. Now we’ve created this thing called broad search, where you type in safety and it pulls themes out, and it brings whole responses back. And it’s much broader than safety. And we’re using algorithms that really score the words and put numerical values on words and they kind of nest those words next to each other, so that you can get whole responses back and it’s been game changing already, because it just helps you synthesize through the analysis of that to get to the real essence very, very quickly. And something that could take hours could take seconds now and that’s a great example of how were automating and reducing friction and using technology to help us routinize and reduce time so that we can be faster, easier and reliable.

Alexander Ferguson 26:38
The usage of natural language understanding is impressive where it’s being integrated and this is a beautiful use case for its its power, where a business leader can real time search through that three 400 results that came back from different people submitting and get not just exact words but sentiment and the larger concepts and getting those answers where you would normally in the past have to actually listen to each one to be able to pull those insights out.

Rob Holland 27:10
That’s absolutely right and we’re just getting started I see the ability to integrate video more aggressively and read faces and be able to bring snippets of that and then search and really bring platform text search across many of the different studies you’re running and create libraries of benchmarks across multiple studies and and really start to get the take advantage of learning what’s going on and pick up broader themes and that’s really what we’re excited about going over time.

Alexander Ferguson 27:41
Now your your past rob you you’ve been in the business and consumer world for a while I think I see you also at Oracle and and Nielsen company. What How have you seen research and insights change over the years and what do you see is kind of that the future on how and how technology is going to be intersecting both helping it or maybe even hurting it?

Rob Holland 28:06
Yeah, I’ve been I’ve been around for a while. So um, my sorted out in the, in the in the world of consumer product goods with the Clarkes company and, and then also in consumer retail with Safeway, before I went to Nielsen where we started getting into the consumer research side, and then data logics Oracle, which is all about AD tech, again, driving advertising to consumers, and then martech with Consumer personalization. So I’ve been hanging around the consumer my whole career, you know, I can give you a really Stark example of something when I started out, which is, you know, in the 80s, late 80s, at the Clorox company, which is a great company doing amazing things. And, again, COVID, their name of their products have just taken on a whole nother another dimension. But I’m, early on in my career, the company was getting ready to launch it was launched Clark’s detergent, which is gonna go up against tide, and was a massive big bet. And back then we used to do research by running test markets in many markets, where we would take over a whole city produce the product, put it into the shelves, run advertising, tests, you know, all kinds of testing, and these were multi million dollar investments that took months and months and months. Meanwhile, the competition’s watching everything you’re doing, and they’re creating their response plans, and then and then you go on your launch. And it was it was a really painful launch for the Clorox company at that time, and it cost 10s of millions of dollars, manufacturing plants, and other things then, love Clorox that was 30 years ago, nobody’s there. Now we don’t make those mistakes anymore. And what we’ve done is we’ve de risked it by bringing technology in and making faster decisions, more informed on smaller bets. But that’s just the way the industry work. Even Nielsen’s started out doing test markets in many markets support. And, you know, and they were doing diary type work. And now it’s all online research and it’s mobile. So technology is allowing us allowing everyone in the consumer space to get closer to consumers. And it’s allowing consumers to have a more direct voice into their, into their brands and into the, into the businesses and companies that they work with. And so really creating new methods and new technologies using technology and create new methods and innovating so that we can make those the voices that are out there. And the questions that exist on the demand and the supply side, bringing those together in a more efficient way, just really helps make a more efficient market. And that’s really what’s happening. And, and the, the pace of change is just happening dramatically as well. So you know, it all this is all feeding together. And it’s just part of what’s happening everywhere in life anyway, because of the of those mega themes of consumer technology informed consumers, you know, just expectations are changing across the board. And we have to adapt and business for that, or we run the risk of really either, you know, watching our businesses completely fall apart or being disintermediated rapidly and making bad choices. So we want to help make good choices.

Alexander Ferguson 31:23
We all we like making good choices. Last question for you, Robin, just kind of a fun one. there any current books that you’re reading or read in the past or podcasts or audiobooks that you’re listening to that you would recommend for other business leaders.

Rob Holland 31:39
Failure is not an option by Gene Krantz. Gene was the head of Mission Control for NASA from the beginning of the space program through the early parts of the space shuttle. So he gives you a long bird’s eye view of what went on during the whole path from beginning through getting someone on the moon and beyond. It’s a great expose a on how the whole program evolved from Gemini through mercury through Apollo. And it really talks about each of the missions, what their goals were, what each missions was expected to accomplish, and how it built on and they learned from an iterated, to go from the first steps to getting a rocket out into space to getting someone on the moon and back alive. It’s a phenomenal story. It’s full of courage, ingenuity, persistence, hard work, and just great drive to accomplish the impossible and to do it as quickly and as effectively as, as humanly possible. gene is a person if you’ve seen the movie Apollo 13, with Tom Hanks, Gene played Mission Control leader by Ed Harris. So that gives you a visual of what gene was all about. Anyway, it’s a great long read, it just shows you how hard work iteration and just constant pushing you can achieve great things and overcome seemingly insurmountable obstacles. Love the book, highly recommend it.

Alexander Ferguson 33:12
I love it. Well, thank you so much, Rob for for both your insights to be able to dig into what is this new world of Agile research insight. For those that want to learn more, definitely go to feedbackloop.com you’ll be able to watch some videos that they have there, get or get a demo and go to UpTech report.com to get this full interview if you’ve if you’re watching this from somewhere else. And Rob, this was a pleasure. Thanks for your time.

Rob Holland 33:36
Thank you very much, Alexander. Great to be here.

Alexander Ferguson 33:38
That concludes the audio version of this episode. To see the original and more visit our UpTech Report YouTube channel. If you know a tech company, we should interview you can nominate them at UpTech report.com. Or if you just prefer to listen. Make sure you’re subscribed to this series on Apple podcasts, Spotify or your favorite podcasting app.