Master Class: How to use lean analytics – with Ben Yoskovitz
When a startup fails, most founders wonder, “What happened?”
That’s the situation Benjamin Yoskovitz was in a few years ago. “It was clear that what we thought was a solution to a problem that we were attacking was not resonating,” says Ben, a tech entrepreneur, angel investor, and co-author of Lean Analytics: Use Data to Build a Better Startup Faster.
But he didn’t know why it didn’t resonate. “And so you’ve mentally and psychologically and financially committed to something,” he says. “And it’s very hard to…change it completely. Frankly, you’re not even going to know what to change. You’re not even going to say, ‘Okay, this is not quite working, but what have I learned from this experience?’”
In contrast, Localmind, a startup that Ben invested in, tested one small hypothesis before they built anything. Localmind wanted to design a product that would allow someone to ask a location-specific question, and people currently at that location could answer it. But would people actually answer the questions?
“And so they used geotag tweets to test it, so people who were tweeting from Times Square, you would know they’re at Times Square,” says Ben. ”And people answered the questions…so [Localmind] de-risked one element of the business.” They went on to build their product, and were eventually acquired by Air B&B.
In his Mixergy course, Ben shows you how to use data to build a better startup faster. Here are three highlights from the course.
1. There Can Be Only One
But Ben says there’s only one metric that matters. “At any point in time with your business, there’s one metric that you should really be focused on, and it’s the metric that everybody is trying to improve and is thinking about,” he says. “Obviously, it’s not the only thing that you track, but it’s there to create focus. What are my number one problems, and what’s the metric that’s going to help me understand if I’m going in the right direction?”
So how do you figure out which metric matters most?
Choose a stage-appropriate metric
Your number one metric depends on a couple of factors.
“[My book] defines five stages that every startup will go through,” says Ben. “It starts with empathy, it goes to stickiness, it goes to virality, revenue, and scale. You have to understand the stage that you’re at, and you have to understand your business model to…narrow down which is the one metric that matters for your company.”
For instance, when you’re just starting out, you’re probably in the empathy stage. “Empathy is, does anybody even care?” says Ben. “At that stage, oftentimes, it’s qualitative feedback. There might not even be a product, it might just be qualitative feedback from talking to users.”
2. Get a Grip on Reality
“You need a target, you need a goal,” says Ben. “But you know, let’s say churn is 8% right now per month for your business. You know it’s not good. You know it could be better. But what should it be?”
For instance, Ben says WP Engine had a churn rate of 2% per month, and they wanted to reduce it. “When you think about it, that’s a quarter of your customers a year disappearing,” he says, “which can freak you out.”
But WP Engine didn’t really know what their goal should be for churn rate. So they just tried a bunch of things to reduce it, “and they couldn’t really move the needle,” says Ben.
So how do you figure out what your goal should be?
Know what’s standard
Find industry benchmarks.
For instance, WP Engine talked to other people in their industry, like Automattic, the makers of WordPress. “[They] learned that a churn of 2% is quite healthy in the hosting business,” says Ben. “[The founder] realized, ‘I could spend all my energy banging my head against the wall, trying to over-optimize a metric that’s probably not going to move. And even if I improve it a little bit, it probably won’t have a significant impact on the business.’”
And knowing your industry benchmarks can help you focus on things that will have an impact on your business. “You can focus more on acquisition, to…pour more people into the bucket,” says Ben. “The bucket’s a little bit leaky. It’s always going to be leaky. But it’s good enough.”
3. Make Your Metric Do Yoga
Ben points to the example of Buffer. When Buffer first launched, they charged customers to use it. “That is extremely rare for…a traditional enterprise software product,” says Ben. “But if we think about empathy, if you can get people to pay you some money, that’s a pretty good indication that they care about the problem that you’re solving.”
And people did pay. And because 60% of people who signed up were still engaged a month later, the product was also sticky.
But what Buffer needed was virality. In order to scale, they needed as many users as possible. And the paid model was a barrier for user sign-ups.
So what do you do when your original “one metric” conflicts with your current goals?
Embrace change
Keep changing the one metric that matters.
“[The Buffer team] said, ‘Well we know people will pay, but let’s [create a free version] because we need to acquire more people,’” says Ben. “And so then they used freemium as a marketing lever to acquire more customers faster. And conversion at the time was about 2%, which is actually quite good for a freemium business.”
And because they changed their metric, Buffer was able to reach more than a million users. “They’re very much in that scale phase now, which is, let’s just acquire a ton of people and see if they follow the same behaviors of the people that came before, and if they do, you’ve got a business,” says Ben.
Written by April Dykman.