Did you know that scientists have actually been applying an approach to validating their hypothesis for hundreds of years? It's called:
The Scientific Method
The Oxford Dictionary (online) defines the Scientific Method as:
A method of procedure that has characterized natural science since the 17th century, consisting in systematic observation, measurement, and experiment, and the formulation, testing, and modification of hypotheses.
Let's look at how the scientific method was used by Francis Crick and James D. Watson to determine that DNA had a helical structure:
Define a question/problem and study/observe it - Crick & Watson defined their research outcome as finding the structure of DNA
Form an explanatory hypothesis - Crick & Watson hypothesized that DNA had a helical structure
Test the hypothesis by performing an experiment and collecting data in a reproducible manner - Crick & Watson knew that crystallized pure DNA subjected to a process called X-ray diffraction would produce an X shape if indeed the structure of DNA was helical.
Build: Run an X-ray diffraction on crystallized pure DNA
Measure: Measure the resulting shape of the crystallized pure DNA
Learn: The resulting X-ray diffraction was X-shaped, meaning DNA is helical in structure
The Lean Startup Cycle: An example
Let's take a fictional example. Lets imagine that you are the product manager for an accounting software called EasyAccounting. The latest research has thrown up an interesting idea: what if when customers had questions, they could have the help of an actual qualified accountant at any time? This service could be provided to existing customers in addition to the access to the EasyAccounting software.
As a product manager, you state your hypothesis (i.e. your assumption) as "We assume that customers would pay an extra $50 a month to have unlimited access to a real qualified accountant via chat/Skype." Let's see how we can validate this hypothesis using the Lean Cycle.
Idea - Your first idea is to convince an accountant in your company to make himself available to customers on Skype and chat for one week. Your intention is to reach out to customers who have been in touch with the company's support team during the last ten days. These customers will be emailed with an offer for this service.
Build - You spend one day creating a landing page signup form and then ask your colleague in the tech team to give you the emails of all the current customers who have been in touch with support in the last ten days.
Product - The landing page and nicely-designed email represent your product. Some customers will get an email, then visit the landing page where they can then pay $50 for this service. At the end of the week, you will send each paying customer a survey to gauge satisfaction.
Measure - You will implement some tracking on your landing page.
Data - Your data would consist of:
How many customers opened the email?
How many clicked the button / How many visitors to the landing page?
How many purchased the product?
What were the satisfaction levels from the survey?
Learn - You will examine the data and, from this small experiment, determine:
if customers were willing to pay
if they were curious (clicked on the button in the email, but didn't pay)
If they didn't pay, why not? If you have a strong opinion here, fixing that problem would be an idea for your next experiment.
If they did pay, were they happy with the service? What did they particularly like? How did they describe the benefits in their own words? What did they not like? What would they improve?
The Lean Startup Cycle - Not just for startups!
Consider the following two scenarios:
When building a new product, product managers develop a product vision by finding a problem and a set of users who have that problem and then doing product research to further their understanding. After interviewing customers, researching the competition, and sketching potential business models, the product manager will have an idea of what the solution is. The business model, product, and high-level features will not yet have been validated but will have been elicited from interviews and research.
When working on an existing product, the product manager should be clear on the existing product vision. When analyzing the success of the product and potential future improvements, the product manager will know what key activities and metrics they require. Before building complex features that cost a lot of time and money, the product manager validates on a small scale that these features are likely to produce the improvements in metrics that are expected.
In both cases, an experiment-based approach to learning is an ideal way to validate the underlying assumptions.
The story of Dropbox
Drew Houston is the CEO of Dropbox, a service that updates your files automatically across computers. Many of you today areDropbox users, but back when they launched it was not yet known whether customers would understand and be willing to pay for such a service.
Rather than build a full solution, Drew created a video explaining the service to see if the concept resonated with their early adopters. The video was not the most professional, slick marketing video ever seen, yet the results were incredible. Their waiting list went from 5000 users to 75000 users overnight.
This is a great experiment:
Build - Drew could build a video in the space of a few days
Measure - Their waiting list grew by 1500%
Learn - Drew learned that people really identified with the problem, could understand the concept, and it was worthwhile to continue.
Minimum Viable Product
In the dropbox example above, Drew Houston created a video. This video was relatively quick to build, certainly much quicker than building the whole Dropbox system! Drew also got very quick feedback. When we build a solution that enables us to get real feedback (or data) quickly, we refer to this as the "Minimum Viable Product" or "MVP".
In the next lesson, we will learn much more in depth about the different types of MVPs.
