How to run an experiment to test an idea
The Aim
To de-risk your ideas through experimentation before you launch them.
Resources
- An experimentation budget (this can be anything from $10 – $10,000 depending on the size and scope of your project. The rule of thumb is only to test exactly what you need to learn and to maximise your learnings per $$ spent)
- An experiment working group – typically a cross-functional team of 2 – 5 team members who are responsible for designing and leading the experiment and reporting on the experimentation results
Method
There are 4 key steps to a well-run experiment:
- Hypothesise – is where you identify and list your riskiest hypotheses (AKA assumptions) that underpin the success of your idea. These are the assumptions that are too risky to proceed without validating.
Start with assumptions you might have about your customers, in terms of things you’ve assumed your customers will value about your idea and ways you’ve imagined they will behave. (E.g. Our customers will pay more for a quicker service, or Our customers will want to buy our product without seeing it in real life.)
Once you’ve listed all your customer-focused hypotheses, prioritise them, starting with your riskiest hypotheses. If you can’t prove these, you no longer have an idea. You can then design an experiment to test your riskiest customer-focused hypotheses by completing steps 2–4.
Once you have tested assumptions about your customers, move to Feasibility (i.e. can we do this?) and Viability (i.e. should we do this?) assumptions.
- Design & Build – is where you design an experiment to test your hypothesis. First, you need to decide on your ‘Minimum Viable Product’ (MVP). Your MVP is designed to test your customer’s behaviour around your hypothesis with the least amount of energy and resources.
An MVP can be anything, but as a rule of thumb must measure your customer’s behaviour, not their intentions. You can check out some different types of MVPs and learn more HERE.
Before running your experiment, you must also decide on your success metrics (i.e. the metrics you will use to measure your success). These should link back to your hypotheses. For example, if your hypothesis is ‘Our customers will pay more for a quicker service’, your metrics could be ‘the number of customers who purchase X at the higher price’. Again, make sure you align on what success looks like before conducting your experiment (e.g. 70% of customers purchase at the higher price = success).
Once you’ve identified your MVP and aligned on your success metrics, you should consider who you want to include in your experiment and the easiest way to reach them. For instance, you could start with a particular customer group or store for your experiment. However, you should only include customers relevant to the idea you are testing (ideally, living with the pain point you’re trying to solve). You also want to make sure they are easily accessable.
Review THIS video for more information about experimental design considerations.
- Analyse – once you’ve conducted your experiment, you need to analyse your data to determine whether you achieved your success threshold (i.e. whether your hypothesis was supported). If it was not met, you should explore why by seeking qualitative feedback from the customers involved in your experiment. If you did reach your success threshold, you can move to the next step below.
- Iterate – based on customer feedback, you may wish to iterate your idea. If you have achieved your success threshold, you can move on to your next riskiest hypothesis for testing. If you did not meet this threshold, the experiment working group should discuss whether to pivot, persevere or kill the idea based on the experiment data.
The Outcome
Running quick and lean experiments helps de-risk your ideas before moving into implementation. It also helps avoid over-resourcing ideas which aren’t delivering value to your customers. Often, tight timelines and reward structures that focus on delivering commitments mean this step is skipped, as teams are keen to move straight to implementation. However, this can lead to costly mistakes for the business and only add to timelines in an attempt to course-correct, not to mention unhappy customers!
Other ways of creating a highly innovative team
Forge an exceptionally innovative team by harnessing collective intelligence through shifting. Elevate success rates with pre-mortems, reclaim time via zombie campaigns, validate ideas through experiments, and uncover lucrative business prospects, fostering a culture of continuous and transformative growth.
