When you imagine a startup, the image in your head probably looks like this: a bunch of people working day and night, sipping energy drinks at 2 am, chasing crazy ideas they are convinced will change the world. But that’s pretty much every startup. What sets the ones that eventually win apart from the ones that spend years working insanely hard and then disappear?
There are obviously a myriad of factors, and there isn’t hard data on all of them. But one is undisputed: how fast a startup learns. And by learning I don’t mean reading more books, becoming better at building, or consuming more information. I mean how quickly you can take something you believe about the world, test whether it’s actually true, and change your assumptions based on what you find in the real world.
What the data says
In randomized trials with 759 startups, founders trained to approach their ideas like scientists killed bad ideas earlier, made more focused pivots, and performed better (Camuffo et al., 2024). In another study looking at 35,000 tech startups, those running A/B tests improved performance by 30 to 100 percent within a year (Koning, Hasan, Chatterji, 2022).
What does this mean concretely for a founder?
The interesting thing about both studies is understanding why controlled experimentation helps. The companies got better at figuring out what was true. Running a startup is essentially a truth-seeking exercise. You start off with a rough goal and a pile of assumptions. Your goal should be to verify these assumptions as fast as possible, before your money runs out.
Most of them sound obvious enough that you don’t even think of them as assumptions:
- People in X want Y.
- Problem Y is painful enough that they will pay someone to solve it for them.
- Person with profile Z is the one who actually cares about this problem and makes the buying decision.
The list goes on forever.
This leads to the most basic Y Combinator advice: build something people want. It almost sounds stupid because it’s so obvious. Nevertheless, most startups fail exactly here. They build something people don’t want.
How does this happen? Usually because founders never really verified that people want their product. They verified weaker versions of the statement, missing some crucial points. Someone told them it sounded interesting. People liked the demo. But that is weak evidence. Someone going through the process and buying your product is hard evidence.
Every startup has wrong assumptions. What kills them is carrying those wrong assumptions for too long without realizing it.
Learning speed is different from execution speed
People talk about startups moving fast all the time. But I think this creates some confusion.
You can move incredibly fast in the wrong direction. This will lead you nowhere. You can ship ten features every week that nobody wants. You can make 500 sales calls with the wrong pitch. That’s execution speed.
What matters is whether your execution actually creates new information and how fast you learn from it. Execution speed is still extremely important, but only if it leads to faster learning.
Learning velocity = meaningful belief updates / time
As a mental model, not as a strict formula.
How many important things about your business did you learn this week that you didn’t know last week? And did those things actually change what you are doing?
If the answer is nothing, then I’m not sure how much progress you really made, regardless of how many hours everyone worked.
In turn, your runway doesn’t just measure how much time your company has left before going out of business. It measures how many learning cycles you have left before you have to find truth.
Learning velocity is one of the few advantages startups have
Even a well-funded startup will have less capital and manpower than the long-standing incumbents in the industry. Working harder will not bridge the gap. If you are a 20 person team working 16 hour days, the net hours will still be less than a 1000 person company where people work 8 hour days. So what makes these small companies win over the big ones?
There are many factors, but most importantly they learn faster and in turn move faster. Moving faster means they come up with ideas more quickly, throw out the ones that don’t work, and double down on those that do.
A startup might launch a product, get feedback, and launch a new feature addressing this feedback within a week. An incumbent needs an entire quarter for the same process.
This is already common knowledge among early stage VCs: you bet on the founders, not on the idea. The reason behind this is that a good founder will move fast enough to find the right idea eventually.
So why isn’t everyone doing this? I am not really sure, but I think part of it is psychological. If you are a founder, by definition you have to be ambitious and really believe in your idea. Challenging the core assumptions might be deeply against this attitude.
How I try to do it today
Every week I have a list of assumptions that I am verifying, and a way to measure them, for example a new launch, user interviews, etc.
At the end of the week, I update my assumptions and attach a confidence level to them. The confidence level is something I borrowed from the rationalist community. It’s not strictly necessary, but it helps me not trip over making big statements based on relatively few data points.
For example, an assumption can look like this:
Claims leaders in the workers’ comp industry are willing to buy a standalone product that solves the triage of claims.
An experiment would look like this:
Reach out to 100 potential customers, get 10 conversations, and get 5 of them to move further in the sales process.
If the experiment fails, I have a new data point on my assumption. Depending on how confident I feel about the experiment or assumption, I then double down on it or consider it falsified.
The fastest learner wins
Every startup begins with a model of the world that is at least partially wrong.
- Your customer is probably not who you think they are.
- The customer’s pain points are probably not what you think they are.
- Your sales pitch is probably wrong.
- Something you thought to be unimportant turns out to matter enormously.
This is not a failure. It’s the result of operating under extreme uncertainty.
The winners aren’t the founders who start with all the right assumptions, but the ones who find truth the fastest.
If you liked this article, feel free to message me and I can share the concrete framework I use to challenge my assumptions and the automations behind them.