It might seem like it would be easier to run a win/loss program in an organization—like Forrester—which already has a culture that embraces customer experience and research. However, while there is an upside to launching a program in such an environment, there are also heightened expectations.
Forrester is one of the most influential research and advisory firms in the world, so it’s no wonder that their Vice President of Customer Experience (CX), Derek O’Grady, likens being the company’s CX leader to being asked to cook pasta in the North End, Boston’s venerated capital of Italian cuisine.
Derek is the lead on the win/loss program because Forrester puts win/loss research under the CX umbrella. “It’s about listening to customers and prospects—what’s working for them, and what isn’t,” Derek says. “Win/loss is a super important vehicle to capture the voice of the customer, which is why we’re really investing the time to do it right.”
In a recent conversation on the Blindspots Podcast, Derek shared some really helpful frameworks and advice for building and managing a successful win/loss program.
Derek looks at voice of the customer (VoC) data as being multidimensional. He recognizes three distinct dimensions, each of which delivers a different kind of value, and each of which supports and complements the others.
CRM data might be considered the first level of VoC. It can be useful, but it’s not always the most reliable source. While Forrester does do a good job of collecting this data, it’s not perfect. (It rarely is.) Most companies struggle with ensuring good consistency and enough detail in their CRM data. More to the point, it’s important to remember that, because it’s self reported, CRM data has the potential to be biased.
A customer survey gives you data straight from the customer, but it does still reflect only very high-level feedback. That’s simply the nature of a survey. It needs to be brief and fairly general because people are busy, and because you’re asking everyone the same questions.
For Derek, the third dimension of VoC data is the more extensive interviews he does in partnership with DoubleCheck. These interviews may start with the same set of questions for everyone, but they offer an opportunity to follow the threads of the conversation—digging more deeply into the “why” behind responses, and also allowing for the customer to drive the conversation to the topics that matter most to them.
In addition, Derek notes the importance of having a third party do the interviews. “A third party doesn’t have a dog in the fight,” he says. “So they can approach interviews without any preconceptions, as a professional win/loss practitioner who has a proven process for establishing hypotheses, an interview guide, and the skills to conduct an agile interview.”
Derek also emphasizes the importance of having a professional level of rigor around the analysis. This is a big part of what makes in-depth win/loss interviews such an effective way to capture deeper insights.
Surveys and in-depth win/loss interviews each have their place in competitive intelligence research, but it’s important to understand the strengths and weaknesses of each method so you can get the most out of your efforts.
Derek points out that the primary difference is the level of depth you’re able to get to, “There are some limitations with surveys because you can ask a combination of open-ended questions, and questions where you ask respondents to rank something on a scale, but the responses lack context.” Surveys are best for quick-hit items like validating a very specific idea or getting initial, high-level information back in a non-invasive way that doesn’t ask for too much of a customer’s time.
Interviews, on the other hand, are for more involved research. “We reserve interviews for bigger, strategic items that we’re really curious about,” Derek says. “It might be a topic that’s causing us some pain, or a topic where we really see some opportunity, but we need in-depth information to capitalize on that opportunity.”
Forrester is a dynamic company that is always growing and evolving. There’s always something happening. Recently, for example, they launched a new product, and have also gone through an acquisition. It’s all very exciting, but in order to ensure the long-term success of all these ventures and initiatives, they need to know what’s resonating. Win/loss research helps them get a clear read on what’s resonating with customers and prospects, not just in terms of sales, but also in terms of retention. This win/loss/churn analysis is something that matters to every part of the organization.
“The first thing we did, before we launched the program itself, was to make sure we had executive team buy-in,” Derek says. “At Forrester, this means we needed really tight alignment with what we call the ‘revenue engine’: product, sales, and marketing.”
Because any new initiative tends to require a little proof-of-concept before people are willing to totally sign on, Derek recommends involving stakeholders early in the process. “Getting stakeholders involved early on is beneficial for two reasons,” he says. “First, it keeps them engaged and informed. Second, it gives them the opportunity to inform the actual study by providing their hypotheses, opinions, and experiences.”
Derek also bolstered the success of the program by carefully balancing leadership involvement with involvement from folks on the front lines. Engaging customer-facing experts who have direct, real-world experience with the issues and behavior you’re researching is a smart way to get a leg up on what’s happening, which will help you refine your research.
Of course, by involving a wide variety of stakeholders at the start, you’re going to wind up with a lot of possible directions you can take. Derek started with about 15 to 20 stakeholders and a lot of different hypotheses. To create an efficient program that could generate a win right out of the gate, he needed to pare things down quite a bit.
Determining which topics and issues to focus on was a process of both categorization and streamlining. “There was some pretty obvious overlap in the hypotheses provided from different sources, so our first step was identifying those commonalities,” Derek explains. “And then we grouped the remaining topics and questions by patterns and themes.”
From there, they were able to identify the strongest topics, and prioritize where they wanted to start their research.
You can prepare the most well-designed and detailed interview guide, but conversations always take on a life of their own. For best results, it’s important to be able to adapt to what’s actually happening instead of stubbornly sticking to the script.
“The conversation is going to meander. There will be certain ‘hotspots’ that the client will be very passionate about. And there will be other things you thought would be important, but they’re not,” Derek says. “Your interviewer has to be very agile in order to identify the gaps and opportunities in a conversation.”
Derek also offers the reminder that you don’t need to test every single hypothesis in every single interview. Some topics will come up organically; others will not come up at all. Take your lead from the interviewee, and you’ll end up with better feedback and insights.
The key to selecting good interview candidates is making sure that you’re selecting people who will reflect reality. Derek has found that sometimes this means basing selection more on a specific person than on a specific organization. There are just some personality types who are more eager to participate in interviews, and more willing to provide feedback that will be useful.
Derek’s team already knows some of the candidates because, in their CX role, they interact directly with a lot of customers. In addition to making his own selections, Derek also taps account, sales, and customer success teams to make recommendations. In particular, he follows up with account teams, asking them to weigh in on which customers they think would be willing to talk, and have maybe already expressed thoughts—either glowing or constructive—that could translate into valuable insights.
Again, it’s important to remain aware of any potential bias that might skew the interview. “We are very conscious of the potential bias in asking our account teams to identify candidates,” Derek says. “We are careful not to stack the deck with only positive comments. That’s not helpful. What you want is a balanced view of reality.”
Forrester’s outreach approach is pretty straightforward. It usually begins with a brief email from Derek, as CX leader, explaining the project and inviting the candidate to participate. Forrester often gets a good response because they are in the unique position of having customers who are very invested in what the company does. “They see the value in what we do, and how we help them succeed in their businesses,” Derek says. “So they are very good about patting us on the back when we’re doing well, and about pointing out where we need to do better.”
As a follow-up, Forrester lets candidates know that DoubleCheck will reach out to schedule the conversation. “It’s important for candidates to know that there will be a third party working on behalf of Forrester,” Derek says. “That way, they can speak more candidly, especially if they have something critical to say.” Ensuring that interviewees can speak their mind is especially important in B2B, where there tends to be more empathy between buyer and seller than in a typical B2C transaction.
“There is an art to communicating with internal stakeholders,” Derek says. “You need to find the right balance. People don’t want to be emailed every day with updates, but they do want to stay informed.”
Derek employs a two-prong strategy to help establish the right cadence and relevance when he shares research findings.
First, he uses a variety of communications channels to reach his different internal audiences. For example, he presents some material at all-hands meeting calls for various groups within the organization. He might then deliver other insights and information via less formal conversations and exercises on Salesforce’s messaging app, Chatter. By changing up the communication channels and formats, he avoids oversaturating his audience.
Second, Derek makes sure to keep the information he delivers simple, focused, and actionable. He understands that everyone is busy, so the content needs to be highly relevant to the reader, and the format needs to be easily consumed. “If you come back with 99 findings, you’re going to lose your audience after the third one,” he says. “But if you come back with 3 to 5 big themes, and you can explain what each one means, that really resonates.”
The “WIM” (What It Means) component is critical. The win/loss interview process is quite elaborate. There are interview recordings, transcripts, and DoubleCheck’s themes and other analysis. Most people within Forrester lack the bandwidth to wade through all the minutia themselves. “People really rely on us to curate the information for them,” Derek says. “It’s our job to deliver the story, and their job to react to the story and figure out how they will apply the insight to their business.”
In terms of the actual information being shared, Derek works to strike a balance between the expected and the unexpected. On the one hand, he takes care to circle back to the original hypotheses that were provided by stakeholders and other contributors. “If people take the time to provide their hypotheses, they will want them acknowledged,” Derek says. “We usually have a slide early in the readout deck that addresses those directly so that people know we listened to what they said.”
At the same time, Derek focuses a lot on unexpected learnings. “We spend most of our time going through what we learned that we didn’t expect going in,” he explains. “We want to identify new insights that are going to change our way of thinking.” For example, loss conversations turned up the unexpected insight that some customers felt that Forrester was unable to meet a specific need when, in fact, Forrester did have the required capability. The fix was easy—the sales team just needed to take a different approach to connect the capability to the need. That, as Derek said, is valuable low-hanging fruit that can make a big difference.
While win rate and other metrics are a reliable and common way to measure program efficacy, there are other clues that you’re headed in the right direction. “The reason I know our win/loss program is working well is that we’ve created demand,” Derek says. “People proactively approach us with an idea they want to test. That wasn’t happening a year ago, but it happens on a weekly basis now, and from all different parts of the business. People are thirsty for our insights.”
While he is very pleased with the success of Forrester’s win/loss program, Derek does have one piece of advice to offer to other program managers—one thing he might do differently if he had the chance. “In theory, I would have knocked out the three dimensions of our win/loss data sources in order,” he says. “Meaning that it would have been great to get each area in order before moving to the next—starting with getting internal CRM data cleaned up; then working on some concise surveys; and finally bringing in the heavy-hitter, in-depth interviews with a third party for more strategic pieces of research.”
Establishing the pieces of a well-rounded win/loss program to feed your VoC practice doesn’t necessarily have to be done in that order, but Derek thinks it might be ideal to have the first two dimensions (CRM data and survey data) in place before you start interviews. This would mean you’d have starter insights to tee up for your third-party interview partner, which can be really helpful when you go to tackle more complex conversations.
At the end of the day, all the different dimensions of VoC data are connected, and they all complement and inform each other. You just need to understand which methods to use when, and commit to working closely with all your different stakeholders so your program is aligned with your organization’s big picture goals.
Join us in insightful conversations with win/loss leaders to help product marketers get a leg up on the competition through win/loss and churn analysis. Subscribe on Apple Podcasts, Spotify, Google, or listen on our website.