2,112 words; 8 min 27 sec reading time

—

3-2-1 Launch?

We all know how exciting a product launch can be, and often, from an idea's conception, teams are galloping towards launch like a horse to stable. Modifications can be time-consuming and costly. "Isn't it time to go to market already?" we ask impatiently.

Maybe.

Yet even the best ideas do better by adding the safeguard of a scientific approach. Launching your offering is hard. Knowing what customers want is harder. The only way to learn is to explore. Ideally, this is done long before you launch. If you invest the time to test your ideas, you will save time in the end. Launching without structured, calculated forethought is a rush towards failure. This article will help you avoid the common foibles faced when going to market through the scientific approach of hypotheses and assumptions. 

What are Hypotheses and Assumptions? 

The Oxford English Dictionary defines Hypotheses and Assumptions like so:

Hypothesis: "A supposition or proposed explanation made on the basis of limited evidence as a starting point for further investigation."

Example: If I cut back on sugar, then I will lose weight. "If…then…" statements are a classic example of hypotheses. Good hypotheses are testable and measurable. 

Assumption: "A thing that is accepted as true or as certain to happen, without proof."

Example: An assumption, essentially, takes a hypothesis as fact. For example: If you cut back on sugar, you will lose weight. While a person may lose weight after cutting back on sugar, the causation is not concrete.

 For Marketing purposes, think of Hypotheses and Assumptions as the questions we pose or statements we make. We then answer, prove, or disprove them to better understand our customers, our offering, and our market. These findings inform our decision-making process in everything from design to investment, thereby exponentially improving our go-to-market odds.

Hypotheses and Assumptions: Then and Now

The approach to Hypotheses and Assumptions has changed over the years. Let's explore their evolution in the following sections:

Then: 

Hypotheses and Assumptions have long been used to improve company offerings and decipher short- and long-term changes. From small tweaks to a complete overhaul, verifying what we believe to be true has led many teams to success. However, the traditional approach to Hypothesis and Assumption testing focuses on the offering rather than the customer. In a vacuum, this approach makes sense: you want the best offering possible for your customer. 

However, once out of the vacuum, this approach loses potency. Even the best offering in the world will fail if no one is there to buy it. The customer is key. As Cindy Alvarez says in her book Lean Customer Development: Building Products Your Customers Will Buy:

"It's a simple formula. Learn what your customers need, and use that knowledge to build exactly what they're willing to pay for." 

This approach reduces risk and illuminates your premium pricing by providing an offering your customer is motivated to buy. It also means you don't waste time investing in unwanted features or costly modifications. Test before you build to discover your product or service's greatest potential.

Assumptions and Hypothesis testing traditionally follow the Lean Startup method created by Eric Ries. It looks like this:

We believe [X feature]

Will result in [Y outcome]

We will have the confidence to proceed when [we see A, B, and C, which are measurable indicators]

Teresa Torres digs even deeper into the idea on her website Product Talk:

Now:

Torres has since updated her thinking to an approach that goes a little deeper to avoid certain pitfalls she's found with the Then approach. Like Alvarez, Torres suggests a more customer-based approach that insists on approving or disproving changes to offerings before they happen, not after. Our takeaway overall is that as customer voices become louder and clearer, customer development must precede your offering's development, and the whole team (customer service, offering development, etc.) must work together, informing one another of their unique viewpoints to create a truly 360-degree view of your offering. 

Now that you understand the basics of Hypotheses and Assumptions, we will delve into how actually to evaluate them and what to avoid while doing so. To illustrate the process, we will use a hypothetical new product launch at Starbucks called the Flying Chai, a chai latte with an added shot of espresso.

The Flying Chai

Step 1: Launch

By Step 1, Starbucks has already broken the rules of the road: don't develop a product before you know there's demand. 

Step 2: The Hypothesis

Following the Lean Startup model, the hypothesis for the Flying Chai would look something like this:

We believe that adding the Flying Chai to our drink offerings

Will result in more Millennial customers

We will have the confidence to proceed when we see sales increase by 3% within three months from launch

Looks pretty good, right?

The Assumption, or rather Assumptions, here are that Millennials like Chai and Espresso and will like them in combination and/or that they will be intrigued by the Flying Chai and come to Starbucks to make a purchase. Other Assumptions could indeed be tacked on but let's keep the list simple.

Now that we have our Assumptions and Hypothesis let's see how the launch went.

Step 3: Analysis 

Sales for the Flying Chai were lackluster. At the 3-month marker, sales were up just 1%. Perhaps Millennials don't like Chai, or maybe they don't like Espresso, or maybe they don't like the combination? Perhaps they don't like Starbucks but love Chai and Espresso?! Where did we go wrong? Why is the Flying Chai floundering?

Where Starbucks Went Wrong

One by One 

Your knee hurts. What should you do to relieve the pain? Perhaps the pain is from overuse, so you rest it. Perhaps the pain is from an instability, so you wear a brace. Perhaps the pain is just from getting older. You take some ibuprofen. You perform all of these remedies at the same time. The pain subsides over the next week, and you are back to your old self. Wonderful, right? Well, sort of. Why did you get better? Was it the rest? The brace? The ibuprofen? Without testing each of these individually, you won't know. Having added all of these solutions at once, you don't know which solution solved your problem.

The same is true when testing your Assumptions and Hypotheses. Starbucks' problem was lower sales than hoped for in their stores with Millennials. Rather than deciphering why that might be, Starbucks launched a product they assumed Millennials would like. They wasted valuable resources to go down a path based on a hunch. Rather than focusing on their customer, they focused on their offering.

The Team Player

The results are back: the Flying Chai didn't bring in Millennials, right? Well, maybe. 

The different teams who worked on the launch might disagree.

Some might argue that 1% is good enough and a 3% threshold was too high.

Others might note that the launch of the Flying Chai happened in Summer when more Millennials were traveling out of the country, and the results only took into account U.S. sales. 

The Marketing team might get blamed by others, stating that the launch wasn't well-publicized to Millennials.

Maybe the product development team gets blamed next when people suggest that perhaps the product itself was sub-par.

Maybe the price point was too high.

Perhaps Millennials do like Chai but not in combination with Espresso.

The ways to negate or agree with the findings are immeasurable. 

Had Starbucks done a little research into their target customer (see our article on creating your Ideal Customer Profile here) before the product was created, they might have more acutely deciphered their customers' wants and needs. Having done this research, they would have found themselves better positioned to meet these wants and needs with the perfect offering. The product development team could have consulted with the customer service team to decipher whether customers like the Chai and espresso combination. The marketing team might have informed the sales associates that mentioning how "unique" or "special" the drink was resonated with Millennials and might lead to higher sales. 

Offerings and services historically have been created by separate teams with little overlap. There's the development team, the engineering team, the marketing team, the sales team, and so on. Yet, in order to develop a truly representative product, all teams need to be on the same page and share their wisdom. The perspective from which each team and team member stand is unique. These positions must be shared. Perhaps sales representatives think they notice that customers aren't buying because of X reason. Yet, instead of sharing this with the team at large, they simply figure they aren't a great salesperson, or the product is inferior. Closing the feedback loop to include all team members makes every faction of your team stronger and ensures your offering is built using every essential resource possible.

Don't Ignore Results

Have you ever gone shopping and walked away with something that you loved but that didn't fit you well? "It's not that tight," you tell yourself, or "I'm definitely going on that diet. 5lbs, and this will fit like a dream". Those pair of pants are still at the back of your closet, aren't they?

In our Starbucks scenario, the same might hold true for the Flying Chai. The more we fall in love with our offerings, the less likely we are to see them for what they might be: ill-fitting. The Flying Chai flopped but still, in love with the product as they are, they continue to offer it, citing false prophecies for its future success and flawed reasoning as to why they will come true.

The time period wasn't long enough to test it. 

People love the Flying Chai; they just didn't know about it.

The 3% rate was too high. 1% is good enough.

We've all fallen in love and been able to look back on how it clouded our judgment. Don't let falling in love with your offering lead you down a wandering path. While you might be the closest to it, those a bit more removed from your offering are actually better equipped to evaluate it. 

Experiment Design: All Hands-On Deck

Building off of The Team Player section above, the team must work as a unit. Before experimentation begins, all hands must be on deck and in agreement. For Starbucks, some of the team thought 1% was too high; others thought 3% was too low. From the get-go, the experiment was doomed because no matter the results, no one agreed on what they meant. Thus, they were open to personal interpretation, ensuring an unhappy resolve.

Tidying Up

Science is based on facts and (ideally) devoid of bias. However, as humans, bias is a pretty sneaky beast, showing up even when we think we've hidden it away. To avoid your and your team's biases, start small and work through the steps. 

1—What are your Assumptions? 

1. Test them individually
2. Test them often

2—How will you test them?

1. Ensure you've agreed as a team what data you will collect and what your indicators will be

3—On whom will they be tested?

1. Who is your participant pool? In our Starbucks scenario, it was Millennials. We would want to define the birth years and then discuss other applicable data.

4—For how long?

1. Again, as a team, decipher what your study looks like length-wise. When will measures be taken? How? For example, we will analyze sales statistics after the new offering has been available for three months. Sales must be up by 3% to consider it a "win." 

5—Accept pushback

1. Again, check-in with your team. Is everyone on board? Why or why not? Create a plan as a team.

6—Run the test, scientifically

1. Do not test multiple hypotheses at once. Test one assumption. Break it down into manageable bites so that we don't have the runner's knee issue.

7—Decipher ahead of time what your results will mean

1. What will an increase in sales by 3% mean? Will you continue the offering? What will 2% mean? Don't forget that as the person behind the offering, you are biased. Ask for help from outside sources, decide on what your results will mean before you review them, and then stick to them.

8—Test and review

1. Stick to your guns, be honest with yourself, and if you can't, hire someone who will. Every methodical step brings you closer to your customer and thus increases sales.

Sources

https://www.producttalk.org/2017/08/experiment-design/

https://www.producttalk.org/2014/11/the-5-components-of-a-good-hypothesis/

https://realstartupbook.com/

‍

© 2021 Inteligems, Inc. All Rights Reserved.