In this article the authors describe their findings from research into the application of scientific decision-making methods on new venture success in the Lean Start-up context. They find that Lean business founders trained in the scientific method have a higher success rate than those who do not receive such training. They present a case study of two Lean entrepreneurs who applied the scientific method to successfully launch a scooter rental business for commuters.
Ask any start-up founder in recent years what guided them in the early days of their entrepreneurial journey and the Lean Startup method is likely to enter the conversation. The book by serial entrepreneur and software engineer Eric Ries, published in 2011, has become a bible for business neophytes; it has sold more than a million copies and is taught in business schools and accelerators.
But much like the iterative approach it preaches, the Lean Startup Method could itself be made better. And my colleagues and I may have found a way.
The lean method calls for startups to learn about customers’ problems and needs, obtain feedback, and build a minimum viable product to test demand. Rinse and repeat until launch day, and even beyond. Lean is about quick, iterative learning through experimentation and feedback, and quitting or pivoting when the original idea proves to be untenable.
My colleagues and I found that training founders to think like scientists could help reduce the risk of sticking with ideas that don’t ultimately work out. In a randomized controlled trial on 116 early-stage start-ups, we show that entrepreneurs who were taught to formulate hypotheses from theories and rigorously test them on carefully chosen samples of potential customers were more likely to acknowledge that an idea was bad, pivot from non-starters or pitfalls, and generate more revenue than the control group.
Lean Startup method reloaded
We partnered with two startup schools to create a training program based on the Lean Startup method. We then recruited nascent startups – those with only a business model in mind, or who had just begun to work on their idea – to enroll in our training program for free. Due to resource constraints, we selected just 116 of the 164 that applied. They were randomly assigned to either the treatment or the control group.
Over the next four months, both treatment and control groups separately attended the same training program, which consisted of five lectures and five coaching sessions with a mentor. All our participants learned to draw up a “business model canvas”, conduct behavioral customer interviews, create a minimum viable product or service, and test customers’ responses to it with experiments or quasi-experimental data.
The difference between the groups was that we taught the treatment group to use a more scientific approach throughout the process. First, they learned to use first-principle thinking, which allowed them to identify assumptions and leaps of faith they had made, as they re-examined their business idea. Then, they examined the relationships among the components (“value propositions,” “cost structure” and so on) of their business model canvas and got into the habit of assessing the entire model holistically. They were also trained to collect evidence through robustly designed experiments and rigorous data analysis. Finally, we nudged them to articulate decision rules at the start of their experiments or interviews that would help them to stay the course — or change direction.
In short, entrepreneurs in the treatment group were trained to think and act like scientists. This structured approach helped them better mitigate their biases when they searched for and analyzed market signals. It reduced the likelihood of false positives (bad ideas being mistakenly accepted) and false negatives (good ideas getting rejected). Ultimately, scientific-minded founders in our study averted path dependence, whereby a wrong move early in the process has an enduring impact on decisions downstream.
Over the course of the training and for ten months after, we conducted a total of 16 phone interviews (observations) with each startup. For the 44 that dropped out, interviews were conducted up to the point they exited. Compared to the control group, the treated group had more dropouts (24 vs. 20) and more pivots (19 vs. 11). They also earned more revenue: we recorded 85 positive revenue observations in the treatment group compared to 22 in the control group over a one-year period. Average and median revenue reached €7,800 and €1,300 respectively in the treatment group, versus €900 and €500 in the control.
Mobility in Milan
MiMoto, a scooter-sharing service whose three co-founders were in the treatment group of our study, illustrates how the scientific approach can improve entrepreneurial decision making.
Based on their own experience commuting to school, the co-founders originally planned to offer sturdy (i.e., safe) electric scooters for rent to college students in Milan, home to Enjoy, a trailblazing scooter-sharing service that was already catering to working commuters aged 21 and above. They travelled to San Francisco, Mexico City, and Barcelona to study start-ups offering comparable services. The legwork convinced them that their idea of a dockless scooter-sharing service targeting college students was viable.
In late 2015, the founders signed up for our startup training program and were assigned to the group that would be trained in the scientific approach. Following what they learned about unravelling assumptions and testing their business model by way of falsifiable hypotheses, MiMoto’s founders conducted an experiment with 600 university students. Participants were randomly assigned to try one of 10 scooter models and interviewed afterward on their willingness to use a scooter-sharing service, as well as which model they favored.
The findings turned MiMoto’s original business idea on its head. It turned out that most college students were lukewarm about the idea of using a scooter. Their schedules were fairly fixed — dictated by class schedules — and so public transportation was generally just as convenient, and cheaper as well. Relatively few were attracted by the idea of renting scooters.
The survey also showed that MiMoto’s preference for sturdy scooters could have sunk the startup before it was even out of the garage. Participants, especially women, said they didn’t like the sturdy but unwieldy three-wheeled scooters. It dawned on MiMoto’s founders then that this was a key reason (in addition to relatively high prices) that Enjoy, whose entire fleet was made up of three-wheelers, had failed to gain traction. (Enjoy eventually terminated its scooter service in July 2017.)
MiMoto’s experiment did, however, suggest that nimble scooters would appeal to commuters who needed to navigate Milan’s congested roads on short notice. So the founders pivoted in this direction. They generated new hypotheses and tested them. They continued to narrow down the various likely customer demographics and adapted scooter models until they had an MVP and had identified a promising customer source: young professionals who wanted a flexible and fast way to travel around Milan and were willing to pay for it.
The market validation process took longer than the founders had anticipated, pushing back the launch of their service by a full two years to October 2017. But their methodical, scientific efforts have paid off: MiMoto has raised more than €1 million and expanded to Turin and Genoa, with Florence scheduled to come online this year. More than 70,000 users have signed up with the service, whose 600 lightweight scooters are each booked an average of three times a day. Four competitors have emerged, a further measure of MiMoto’s success, and Italian authorities last year added scooter-sharing to the basket of goods used to measure consumer prices.
MiMoto’s founders recently agreed to sell the company to New York-based Helbiz, a fellow environment-friendly mobility startup, contingent on the latter’s successful listing on Nasdaq. The companies have begun to merge their fleets of electric two-wheelers — mopeds, bicycles and e-scooters — putting the combined entity firmly in market-leading position in Italy, with plans to expand to cities in Europe and the United States.
***
The scientific approach and the Lean Startup method have a lot in common. Both value continuous experimentation, discovery, and learning. But if the lean method is the what, the scientific approach is the how. Using genuinely rigorous investigation techniques is likely to produce better results.