Andrew: Before we get started this interview is sponsored by Fast Customer. You ever have to make a phone call to your bank, but don’t want to wait on hold for half an hour until a real person gets on the phone? Well, with Fast Customer you just do what I do, it’s right here on my iPhone and I’ve had it since it launched. You just type in the person you want to reach, in my case I’m going to go Citibank, credit card support.
They will call them, they will get a person on the phone and then they’ll connect me. I don’t have to press one or press zero or do any of it, or wait on hold. I get to go back to work and they call me when the person is available to talk to me. I do this with Triple A if I needed to, and I did a few weeks ago. I do this with Citibank, with tons of companies. Any company you want to reach, just about, is going to be in there. Go to fastcustomer.com and get the app right now for your iPhone or Android device.
If you’re a company and you don’t want to keep your people on hold forever, maybe you want some insight into their customer satisfaction go to fastcustomer.com and partner with them.
My second sponsor is Shopify. You already know that Shopify is the easiest way to get a store online, and you know that Shopify stores increase sales for their store owners. But they don’t want to reach you. Shopify’s sponsoring Mixergy because they know you’re an influencer and they want to reach your friends.
So if a friend of yours says “I need to sell something online!” I’m hoping you’ll remember to say “Shopify is the number one way to sell online.” Anyone can set up a store right now on shopify.com.
And finally Scott Edward Walker, you know what he specializes in. Working with entrepreneurs like you, especially in the tech world. If you need a lawyer, don’t take my word for it, look at all these great testimonials by entrepreneurs who you know and respect. They all say the same thing that I do. Go to walkercorporatelaw.com.
Here’s the Program.
Hey everyone. I’m Andrew Warner, I’m the founder of Mixergy.com home of the ambitious upstart and a passion project for me where I get to talk to successful entrepreneurs about how they built their businesses and bring some of their best tactics, best learnings as we’re now saying, we’re now using the word “learnings.” Bring them back to you so that you could build your company and hopefully do what Eric Ries is about to do here today. Share what you’ve learned with my audience.
Big question for this interview is how do you build a company using the Lean Startup methodology. That’s what we’re going to learn in this program. Eric Ries used it to take IMVU from a startup with a failed product to one that will generate 50 million dollars in sales this year. He has been building a movement around this methodology and recently published a book on the topic, one that I love and so many of your friends, I’m sure, recommended it and even, not just your friends, but people you admire have. The book is called “How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses.”
Eric, before the interview started I asked you a question that I asked so many of my guests. I say “How do I make this a win for you?” And this glow came about you. Tell my audience what your answer was and where it comes from.
Eric: Well the truth is for the last year I have been trying to convince people to preorder this book. The reason I wrote the book is to take the set of ideas that we’re going to talk about. Not just that are becoming popular in places like Silicone Valley meets TechHub, really take it to a mainstream audience. For those that know about publishing the most important indicator you can have that convinces the mainstream book readers to buy a book is for it to be a best seller.
So I’ve been in sales mode for months. I’m an engineer by trade and I hate selling things. So it’s just really, truly delightful to be able to tell you that I don’t need anything from this interview. We don’t have to sell anything, we don’t have to convince anybody to buy. Because the book, actually if you look at this Sunday’s paper, the book will debut at number two on the best seller list.
So it was a wild ride for me. But the feeling of relief is intense.
Andrew: Congratulations. So what are you doing here? Why aren’t you out on an island somewhere with a martini, or talking to corporate American about how they can pay 20 billion dollars a month to hire you as a consultant who’s going to turn around their business by bringing Lean Startup ideas. What are you doing here on Mixergy, man?
Eric: I’ll tell you the truth which is that I have extremely fond memories of the first interview that we did. Way back before Lean Startup was cool, you had me on to talk about IMVU and the process that we went through. I remember that as one of the best interviews I’ve had the chance to give.
So I’m just excited to bookend the process by coming back on the program.
Andrew: Wow, that means so much to me. Thank you.
Eric: Well, thank you.
Andrew: Let’s get people that kick in the gut that’s going to get them to listen to the whole rest of the interview by identifying with the problem that the Lean Startup methodology helped solve. Give me an example of the problem that many entrepreneurs who don’t use Lean Startup methodology are experiencing.
Eric: The root cause of startup failure is that we still believe that we can plan for the unknown. That’s a very reasonable thing to believe. All of us who were trained in 20th century functionally specialties, in 20th century management were taught planning and forecasting as a basic tool of our work. So if you do better than planned, you get promoted. You do worse than planned you get fired.
Planning and forecasting requires that we have a long and stable operating history from which to extrapolate those plans. Otherwise they’re totally inaccurate. I say this now to audiences all the time “Raise your hand if you feel like the world is getting more and more stable everyday.” No one ever does.
What’s happening is entrepreneurship is all about uncertainty, confusion, new products, and disruptive innovation. When we try to plan out what’s supposed to happen we get the phenomenon I call achieving failure. Which is where you successfully execute a plan, but it’s no good. I have this image, like you’re driving a car off a cliff, but you’re going “We’re getting great gas mileage!”
That’s the problem, and that’s all very abstract this problem about management, but it becomes all too real when entrepreneurs try to put it into practice. In the book I mention briefly, but we can talk about it, this is still a painful memory from the very first startup I tried to build. We had what I call the first half of the movie “The Social Network” experience. So everything in “The Social Network” that happens up until the point where they get rich and get to sue each other, we had all those experiences.
We were working just as hard. We recruited our friends. We did the whole thing. We were literally in a college dorm on an Ivy League campus. We were onto something really big. We had the idea, get this, for college students to create online profiles for the purpose of sharing. Pretty good, right?
Andrew: It worked, yeah.
Eric: Yeah, in 2001 that would have made us way ahead of the curve. But, oh, what the tiniest caveat was we thought we should share the profiles with businesses for recruiting purposes. So what’s funny about that story to me now is if you got back in time, if you could’ve sat with us and said “Guys, you’re almost onto something, but it’s actually about the students sharing the profiles with each other,” and explained social networking to us, we would have said “That is stupid. Come on, we’re trying to build a real business here. We don’t want to just be some dot com eyeballs thing. We’re trying to make a real business selling something to somebody. Somebody pays you money for it.”
We had this very narrow view of what constitute a real business and we were completely convinced that our vision, our strategy, our plan was going to work. We built a great product, we raised a bunch of money, spent the investor’s money successfully. Entrepreneurs raise money, promise the moon, you come back a year later – and what do you know for sure? Just what we did? You know the money got spent. Money always gets spent.
We were always busy. All the product milestones were hit. Our product was revolutionary, and technically very cool, and the tiniest little problem was we didn’t make any money. So that turned out to be a total disaster.
Andrew: So, now in retrospect what do you think…? Did you spend a lot of time building it before showing it to users? Did you spend a lot of time burning through the money and then when you launched it, it flopped? What specifically was the problem there? Because the idea seems pretty sound.
Eric: It’s totally sound, in fact I still think it’s a good idea even to this day. We didn’t understand the process that we would need to turn our good product idea into a company. All we understood was step one, work really hard a build a great product. Step two dot, dot, dot. And step three make a lot of money.
We just assumed that if we built a great product and shipped it, it would automatically great value for customers, it would automatically grow and thrive, and we would get to participate in that…
Andrew: Why wouldn’t it? A lot of times when I interview people I say “What do you do to succeed?” And they say “Well, it all starts with a great product. Once you have that, everything else falls into place.” Then they act like the great product is the be all and end all. What is it? Is it marketing wasn’t there? Is it there wasn’t enough technology to support the product? Is it that people didn’t like the look of it because you didn’t design it right?
Eric: I’ll tell you the truth which is that… I have a saying in the book, I say “If you don’t know who your customer is, then you don’t know what quality means.” So, yes, it actually is true that entrepreneurship is all about creating a great product. It’s just that when we’re sitting in our dorm room, or at the white board in our office imaging the product, what it should be, we have no idea what actually would be a great product.
We have at best a guess, a hypothesis about it that needs to be tested. I’ll tell you concretely what the looks like for us at this recruiting company that we built. We had really cool technology, and we did build this cool thing. We thought it’s a classic chicken and egg business, if you get students in the database, put their resumes in the database that you could sell that asset to companies. They’ll pay you for recruiting. Makes perfect sense, seems very logical at the white board.
So we spent all of our time and energy building this product that would have an algorithmic sorting. Everyone was amazed, it had this very specific workflow built into it that was about how we understood recruiting, what we liked, and had a lot of features that I knew appealed to students to make it easy for them to create a profile.
We launched it with a bang. We spent most of our money on product development and a big product launch. We were hosting parties on campus, had band playing with our t-shirts and stuff. We were very successful in getting the word out to college students that they should create these online profiles.
But then what we had just assumed, that once we had that asset it would be automatically valuable. The companies would then pay us and it turned out that that was just factually incorrect. Once we had it together we couldn’t sell it to anybody. Nobody would buy it. We didn’t have the credibility necessary to sell the company, we didn’t understand how to build a sales process.
But also the product that we had built was not well designed for those specific corporate users. It looked like a college student site, it didn’t look right, it didn’t feel right, it didn’t have the right workflow built into it. So there was no way to transform that product into any kind of a sustainable business.
Andrew: I see. If you had to do it again, today, or if someone else were doing it now and came to you for advice what would you say to them to fix this problem? What’s the one thing that they should do to fix this issue?
Eric: You want to work backwards from “What do we need to make a sustainable business?” to then “What products can we build where we have am minimum viable product? What MVP do we need to test that hypothesis?”
So first of all, whoever pays you money is the customer. So you better understand what that customer demands in order to figure out what to build. Here’s the problem, if you’ve ever worked with a visionary entrepreneur they will be really frustrated when you start talking about customers. They’re going to be like “Listen, customers don’t know what they want.” And they’re right. That’s true.
If you ask customers what they want, they have no idea. There’s this bug in human psychology which is that when asked to answer how we would behave in hypothetical situations, we can’t. We have no idea. You ask someone “What will you want to have for lunch tomorrow?” Statistically speaking they have no idea.
Now imagine how accurately I can answer the question “Do you think you would want to buy this hypothetical product that I’m vaguely describing to you that may or may not work? How much would you pay for that?” It’s like complete garbage.
But imagine I was a scientist, I was a physicist and I came on your program and I said “Science is impossible because electrons don’t know what they want. We can’t ask them questions, and therefore, we’ll never find out the structure.”
You’d be like “What are you talking about? That’s not science, science is you run experiments to determine empirically what people actually do.” That’s the same thing here. We need to devise product experiments that reveal customer behavior in the right way.
So for our recruiting business we absolutely should have built some kind of prototype functionality that would have tested whether employers were interested in buying the product from us right at the beginning. That’s one of the fundamental things we should have been testing.
Andrew: We gave the bad, I’d like to give an example of someone who did this right. And then I broke down your book into a series of tactics that we can discuss here in the interview and give people an understanding of the general idea of the Lean Startup process. Hopefully they’ll go to the book and get even more. But you don’t care, you’re ready now, you’ve done it.
I care. You don’t care because you’re already now a best selling author. I care because I’m so passionate about this audience I want to make sure that they go and do great things. You know what it’s like when you’re out there just spreading a message. Basically what you do is only so meaningful. It’s what people take with your ideas and accomplish that really-
Eric: I said this on stage so many times especially the last two weeks I’ve been on the book tour. I feel so lucky because I get to go onstage and take credit for everybody else’s work. The real work of this movement is not me and my book, it’s what the people do with the ideas. To be honest, when I think about “Why am I doing this?” People always ask me why am I doing this, when I could be off creating the next Facebook, or making a billion dollars, or… people have all kind of random ideas of what I should be doing.
To me, if there’s even one person who watches this interview and it changes how they behave, and makes their company more likely to succeed, if I can save even one person from that humiliation, and embarrassment I felt the moment when the company we were just talking about failed. When it failed that feeling in your gut, when you realize that all the cynics who told you this was a bad idea- Unlike in the movies, they were right and you’re the one who was wrong.
That is a horrible feeling, and if we can spare… from that, how awesome would that be?
Andrew: It really is such a terrible feeling. I remember the first time I ever went to a therapist, one of the first questions he asked me is “How’s work?” He wasn’t asking it in a chit-chatty way, he asked it because he understood, that so much of the way, of our outlook on life, depends on how work is going. Especially if you’re an entrepreneur. Basically your business is you, your idea, is your essence. And if people reject it, they reject your essence, and if your money if going down the drain on this thing, not only are you being rejected everyday, but also you can’t now pay the rent, you can’t look at yourself in the mirror and, you’re struggling there. So let’s help them get through it. Let’s give a quick example and I was thinking maybe we can talk about Aardvark.
Aardvark, couple of guys have an idea, how do they do this right? Give me like the synopsis of it, and then we’ll break down the process that they went through, and other have gone through.
Eric: Got it. The founders of Aardvark were really hard core technologist. They had worked at some of the big famous tech companies like Google. And they noticed this thing, that they called it social search. Which is, when you ask Google, a subjective question, it’s terrible. It’s really good for objective questions. You know, ‘ What’s the population of United States’, ‘What time is it, in Greece right now’, you know, really good.
For questions like, ‘What’s the best bar to get a drink, after the game tonight?’, and it’ll say, ‘Oh, oh.’. Google does not have an answer for that. Because it’s not just subjective, in the sense there is no like one best bar. But it’s also contextual, so that the bar, best bar for you and your kind of person, in a couple of diets, very complicated problem.
They thought, let’s kind of build some kind of search product, that will solve that problem. And given their background, you might’ve said, ‘Oh, they probably going to lock themselves in the basement, spend a year or two building some complicated AI, algorithm, and then launch it on the world. If they had done that, I’m pretty sure they would have been guaranteed failure.
Because it’s one of these products that depends on so much on what is the exact details of people’s behavior, what they want, how do they behave, what kind of searches they do, what is the best algorithm. There is no way to solve that problem with the white board.
I love the Aardvark team, because they spend the first six months, of the company’s life, with a build in commitment, that they weren’t going to wed themselves to any specific bar. They went through a series of prototypes.
They actually incorporated a company they called, ‘ The mechanical Zoo’. And they gave each product an animal name. So that they can be really clear like, ‘This is not the company, this is just our latest test’. They made these really funny, whole bunch of prototypes, you look back on the list of things, you’re like, ‘What were they thinking?’. One was like some kind of notes, ‘Make it easy to make a book mark’ thing. One was about automating, browsing the web for you. And most of them, like in retrospect like, ‘I can’t believe that, that was a good idea.’. And that’s exactly how I felt about Aardvark, last time I heard it.
For Aardvark the idea for the product was, you would IM this, like AI bot, called Aardvark. And he will ask you questions, and he will go through your social network, and IM friends of yours, to see who might know the answer. Ask them the questions to go get you these, it’s like agent. I was, ‘Oh, an agent’, we were always talked about digital agents, that totally didn’t work, and never had done. So I would, I bet if they asked for my opinion, luckily they did not, I would’ve said, ‘ Terrible idea’.
But they decided they were going to do a fast prototype of each product. Faking out as much of it as possible. To build the least amount of technology as possible. To discover if it was a good idea. And they spend months building these prototypes doing these tests. And discovering that customers basically didn’t like it. And they were like, good thing they didn’t build an AI algorithm and all these technology, because customers didn’t want it. One of the founders say to me, something like, ‘The results of our experiment, were unequivocally negative, until Aardvark’.
So because they had all that failure, of all the products that didn’t work, they really understood where it meant, when just a handful of people really gravitated to Aardvark, and are willing to use it, to see that traction.
Andrew: Were they thinking of using artificial intelligence to figure out what the best local bar was? They were, I see.
So if they would’ve spend artificial intelligence, it takes a long time to build. If they would spend time building that out and presented to me, you’re right, it wouldn’t have worked for me.
I want to get a sense of what my friends recommend. I’m much more interested in what book, you want to read, then what books artificial intelligence tells me, is the next book for me to read.
OK. I’ve got that. Let’s break it down. One of the steps is to find your earlier adopters, because these are the guys are going to be forgiven of mistakes. How do I find them? if I have an idea for something, I want these guys that are going to be willing to forgive mistakes, and give me great feed back, who’re going to keep watching me iterate, until I solve the problem.
Where do I find these guys?
Eric: Well, the good news is, you probably already have some in your friends and family network. But it’s really important to remember what an early adopter is. Somebody who has the most acute version of your product problems. If someone is so eager to have their problems solved, they are willing to take a chance on an unknown thing. So most of the people you talk to, are not going to fit the criteria, they’re going to look at the product and like, ‘Why would I want to buy this product from you’. I mean, think back to the, just like when we were in college, most companies are not going to buy a recruiting solution from bunch of college kid. Unless they really, really, really are having a recruiting problem, and they are really eager to get access to this specific kind of students, that we could offer them. There were certain people who were the right target for that.
So the key is just to be talking to as many people as possible, networking as much as possible, what Steve Flank calls ‘getting out of the building’, to screen people to discover who is a good fit for this and who isn’t. And the nice thing about early adopters is that they’re insane. They’re crazy. If you pitch them a solution that really solves their problem they will not let you leave the room without buying it from you. They can be very intense. And that’s the kind of response you want because that’s the kind of person who’s going to be willing to go through the iterations, to be with you through success and failure, to really give you the feedback. It’s about casting a wide net that you can then learn ‘Oh, here’s the kind of person that makes sense as an early adopter’.
Andrew: So if I or somebody were going to create this recruiting social network that you had the idea to create a long time ago, the first thing that you would do is just go to recruiting fairs and see who’s the most desperate person. Make phone calls around to your friends to see who they got pitched on the most aggressively when they were looking for work and then say ‘That’s the first person I’m going to talk to’. And I’m going to say ‘Let me lay it on you. I’ve got this idea for a social network. I will let people connect with each other with the goal being that you can find the best person’. And that becomes the early adopter who you work with.
Eric: Yes. There’s another technique there that was not available to us ten years ago but is now really common which is that we also have the advantage of taking Google AdWords and these really low cost ways of buying customer attention. So another way to find our early adopters is simply to spend a small amount of money on advertising. When we first discovered this technique we spent $5 a day, literally $5 a day. And in those days at 5 cents a click that would buy you 100 clicks a day. So we’re getting 100 people a day who are opting in to ‘Oh, I have that problem’. Then, we call it the magic test: the best way to find your early adopters is simply to put up a web page that says ‘Do you have this problem? I’m going to solve it for you.’ And not really specify, be a little vague on how you’re going to solve it. You’re basically saying ‘I’m going to solve it buy magic’. And then see if people sign up and with those people that sign up then you want to engage in dialog with them. We always say ‘If you can’t sell magic, you definitely can’t sell your product’, because your product is not as good as magic but it still allows you to gather a flow of people who really care about what you’re trying to do.
Andrew: So I read that and I talked to my producer Tristan and I said ‘Tristan, let’s buy a hundred clicks a day from Google to send to one of our courses and then we can figure out how to explain these courses better and get more customers. Anyway, it’s expensive. It’s going to cost us maybe $500-$600 a day just to get 100 clicks right now to one of our pages. I feel like that doesn’t work that much anymore. I want to know what else I could do to find those early adopters. What else can someone in my audience, or what can I do, if we’re looking for early adopters, how do we hunt for them?
Eric: It’s true that Google AdWords is getting more expensive but Google is not the only game in town by any means. The key is in traditional direct marketing it’s all about targeting. You want to get the most efficient yield. For early adopter research it’s not about targeting at all because you don’t know enough to know how to target. So the problem is that everyone uses the same targeting techniques and they always wind up competing with each other, bidding on the same high traffic keyword. What we want to do is find extremely low volume keywords that only a crazy person would put in. Or we can start to do more demographic type ads, Facebook ads. I’ve been hearing really good reports for Stumbleupon ads. These kinds of broad spectrum advertising platforms that really aren’t very well targeted.
We can also borrow a technique from Aardvark. They did this thing called Wizard of Oz testing where you use a technique like Amazon Mechanical Turk to replicate things that the product doesn’t actually do. In there case there was still some AI for matching ‘OK, I ask a question, now which of my friends is the best person to answer?’, that’s pretty hard for a computer to figure out, but not that hard for a human being to do, and they would actually think ‘I thought I was interacting with a computer but there was actually a human being on the other end.’
I know companies that are using Mechanical Turk to recruit early adopters. So you’re actually paying people, who think they are doing a routine task on Mechanical Turk, but actually you’re having them go through a signup club, go through some type of product development experience and all you’re doing is measuring their behavior.
Andrew: I See. What else can we do to find early adopters?
Eric: There’s really no substitute for physically putting yourself in the same place as early adopters. Anyone who’s having trouble with this, that is the most effective thing. Being at trade shows. One company that we talked about in the book actually went to supermarkets because it had to do with groceries and they physically were at supermarkets interviewing customers. So anything that you might do normally as market research. We’re trying to transform from focus groups into an experiment. But the same general techniques still apply. Read Craigslist ads. We used to recruit usability testers from Craigslist and say ‘Do you want to try out a cool new thing that kind of looks like this?” We would pay people to come into our office. Our early version of our product was so bad we couldn’t even pay people to use it. They’d be like, “Oh, keep your 25 bucks.” It was a really horrible technique and be face-to-face with someone and see mind-to-mind, this is someone who either gets it or doesn’t get it.
Andrew: What happens if you go to a trade show… Say, you have a product for real estate brokers. You go to a trade show that’s aimed at real estate brokers, and you start talking to people and saying, “Do you have this problem because I’m working on a solution” and you finally find someone, but you’ve told a hundred other people what your idea is. And now, that idea is out there in the world ready to be stolen by some frustrated real estate broker who wants to go and start a start-up.
Eric: Yeah. That is definitely a risk, but if your speed of iteration is so slow that some random frustrated real estate broker can out iterate you, then you’re doomed anyway.
Andrew: I see. So, you might as well be doomed quickly.
Eric: Might as well be doomed quickly. Really, the truth is it’s actually quite hard to get an idea stolen. I often give entrepreneurs this exercise if they’re worried about this, and it’s literally, take one of your second best ideas, go find the relevant product manager in a company who should be doing that idea and try to get them to steal it. Write them a memo. Send them your business plan. Do whatever it takes to get it stolen, and when you do that you realize that most people face so much inertia, so much politics, so many reasons why they can’t be out executing you that even if they have your idea, it really doesn’t matter. We have to get over that fear.
But the other thing is remember we’re not just trying to talk to other people about our idea in general. Most entrepreneurs are eager to pound their chest and tell everyone how great their idea is. We need to be figuring out how to test our idea with customers. So, actually when of my interview co-founders today runs a fireworks software simulation business. He loves fireworks, and this is really a fun thing for him. It’s a really cool product.
When it was early on, literally one and one of his engineers would be at fireworks conventions trying to sell people their product, and they had a computer right there where they would demo the product. And then, each day they were trying to sell it. At night they would go back to their hotel room and literally change the product based on what they learned that day to see if they could improve the sales rate for the next day. They were actually changing the demo, changing the features, changing the specs, changing the data sheets, everything so that the speed of the iteration was, instead of months and weeks, it would be every single day they would be running a new [??].
Andrew: How do you do that? I find that sometimes people will give me great ideas. I want to change the template of the site in order to use those ideas, and then it takes for frickin’ ever to do it. And I’m on a WordPress site. How do you build a product that you can iterate so quickly?
Eric: It’s about collapsing down that fundamental cycle time involved, the build measure. And oftentimes…
Andrew: What measure? Can you repeat it?
Eric: Build, measure, learn.
Andrew: Build, measure, learn.
Eric: A start-up is a catalyst that turns ideas into products through building. Customers interact with those products that are just a measure generating data. We learn from that data for the next set of ideas. So build, measure, learn. We want to do everything possible to minimize total time through that [??] even if it would otherwise be considered inefficient.
For example, it might be cheaper to outsource your product development to some designer and have them go spend a month cranking on your design, come back, give it to you and now it’s done. That’s very efficient for the designer. It’s very efficient for you, so the whole thing can be achieved. If our goal is to learn, you don’t learn anything while they’re off in a cave for a month. If they leave your product, they’re never going to learn anything, and the next day you’re like, “Oh, I just want to change one more thing.”
Well, now you’ve got to wait a whole other month. What we want to do is whenever possible, collapse down those silo barriers between the function, and we want to say no. Somehow, there’s one designer over here working on something, and a programmer over here and a business guy over there. Whenever possible, put the relevant people together in a room so that they can work at the same time.
Even better, if that room happened to be co-located with customers because if you think about the fireworks example, the fact that the engineers are spending their days selling the product to customers, and then back in their room actually making the changes immediately is what encourages that speed of iteration.
Andrew: OK. So now, we know the product. We have the early adopters. It’s time to build a minimum viable product. You mentioned the Wizard of Oz approach to building the minimum viable product. Basically, in Aardvark’s case, if I were an early user of Aardvark it wasn’t the system that would take my request for a great local bar, it would be one of the founders or someone in there who would take it and then go pass it on to some of my friends, essentially manually.
Eric: Yeah.
Andrew: There’s another example of someone who did that. The guys at Food On The Table who you introduced me to at Noah Kagan’s party, a really fascinating story, and I really didn’t understand how interesting it was until I read the book. Can you tell the audience about how they did this, how they did what you call Concierge MVP?
Eric: Yeah. I really like this story, and it’s not just because the CEO is a friend of mine. He also worked with Matthew. It’s because it’s really educational because his product, Food On The Table, is a menu planning service for families, so if you’re a parent, you’ve got to go shopping every week, you’ve got to figure out what you’re going to buy, you’ve got to come up with recipes that your kids love that are also healthy, that are also cost effective, it’s the proper good life that most of us have. This is a service that would understand what’s on sale at your local grocery store, it would have chefs on staff who are coming up with good, healthy recipes, they would know what your kids like to eat, your dietary restrictions, the menu, the recipes that you yourself like to cook that are easy for you, how much time you have, all this data about you and your family and what’s on sale, and it would give you a shopping list and a recipes list once a week and you pay a subscription service for that to help put food on the table.
Andrew: So, recipes based on what you like and what’s on sale at the supermarket near you.
Eric: Exactly.
Andrew: By the way, if I hear that idea, I think my instinct is, or the average entrepreneurs instinct is to say ‘now I have to go and partner with as many supermarkets as possible so they can feed me data, then I need to go and partner with someone who’s going to give me all of these recipes or I’ll enter it into the system and I will not launch it until I have at least 1,000 recipes because my competitor has 20,000, and then I have to, once it’s launched, show the full product to a customer and say ‘what do you think of this?’ I’m sure that they’re going to sign up, because I’m going to give them discounts that are worth more than the price that I’m going to charge them. That’s the typical approach, they of course took a different approach by building the concierge’s MVP.
Eric: I love this. So, the other ones I was mentioning before were looking for early adopters in the supermarkets. And the nice thing about this is when they spend their time with customers in supermarkets, first of all, most customers are like ‘no, I’m not interested in that, not willing to buy some recipe thing,’ but eventually they find someone in the supermarket who’s like ‘oh, that sounds really exciting.’ So they’re doing the usual market research, they ask them ‘hey, do you like it, what do you think?’ And then at the end of the conversation they’re like ‘great, would you like to buy it right now?’ And eventually they will find a customer who is like ‘yeah, I’ll sign up for that,” and they say ‘great, we are going to give you the concierge’s treatment, which means we will literally go to your house once a week and we will sit down with you and get to know you really well, we will bring the recipes with us, we will figure out the best shopping list for you, we will have a circular, we will know what’s on sale, and then we will literally collect a check from you for $4.95 or whatever the price was in exchange, does that sound good to you?’
And you know, again, most customer’s are like ‘no, not interested,’ but someone who really has this problem is like ‘wait a minute, you’re going to personally come to my house and help me with my grocery shopping every week? Awesome.’ So that’s what they did, they only had one customer, not millions. And I see what having only one customer is, you don’t need to partner with any other grocery stores because you only have one store you have to support, you don’t have to worry about algorithmically figuring out what the person’s preference is, you just have one family you need to get to know well, and they serve that one customer. When that customer is satisfied, the product will work well.
Remember, there’s no technology now, it’s all done by hand, then they went to get their second customer. Again, you don’t have to support a second grocery store because guess which grocery store we’re going to go right back to? That same one that the first person stopped at. So, they’re adding customers by hand one at a time, and the only time they started to invest in any technology at all was when they had too many customers that they literally could not have served them any longer. The great thing about that is, when it’s time to go to technology, you’re only using the technology to automate something that’s already working. Any engineers who are watching, or if you have any engineers, they will tell you automating something that already works is way easier than building something from a hypothetical stance.
Andrew: What happens when you compare those customers who get to talk to a flesh and blood human being, who feel maybe uncomfortable canceling because they don’t want to make Manuel feel bad when they get everything that’s customized to them perfectly that has to go an automated system that doesn’t have any of those benefits.
Eric: Yeah, so it’s important to look at the data you collect during the concierge’s stage really critically because let’s say you figure out the model of how your business is going to grow and it requires a certain conversion rate of customers, you say ok, I have X percent of customers who we give a free trial to, this many will become paying customers. Well if the concierge’s version is only marginally profitable, barely can work according to your model, then you might be worried that the softer model’s not going to be as good, some of that’s going to be a little lower, maybe this actually isn’t working, we’re not on the right track. So it’s important to understand how to interpret that data so that you can imagine what it would be like in an automated form. But of course, the cusp may be right, because some customers actually prefer an automated form, it’s faster, easier, quicker, you can just get it on your iPhone today. Most people use it on their smart phone, they literally just do it in the parking lot of the store right before they go in. That actually works me for me than have to have a human being make an appointment for you. It’s not that you know everything you need to know doing the concierges approach, it just allows you to get started much sooner than if you take the complicated route.
Andrew: I see, so if I were going to use the concierge’s service to create the courses that we’ve got here on Mixergy. What I would probably need to do is find just one person to pay me, let’s say $25 to take a course, and if I find one person who’s going to teach that course, but customize it to this one individual, and then maybe do a second course where I look for two people who want it, and a third, like that, but look for those simple ways to do it. Now, I guess I would feel a little antsy at that point, and I would go, “I’m waiting for a hit here already,” or “I’m looking for something that’s going to show promise of going big!” How do you as an entrepreneur accept that this is going to work, it’s going to take a lot of time at first, but eventually it’s going to work? What do you say to entrepreneurs who feel as antsy as I do?
Eric: It’s super hard. We’re all antsy as entrepreneurs. We want to be big. When we imagine this thing we see it big on the cover of magazines. Just blown up and everyone’s using it.
Andrew: Yes. I don’t want to talk to just one customer at a supermarket.
Eric: How frustrating! I want millions of customers, not just one customer. But if you really accept that our job as entrepreneurs is to learn how to build a sustainable business, then learning is our unit of progress, which means that if we have a product that nobody likes, and 100 customers don’t like it, what’s the learning value of the 101st customer? Nothing. The 100,000th customer? Nothing. All those extra customers just don’t like our product. That’s not helpful. We don’t learn from having more customers. We have to actually be impatient to build a great product, not impatient to get big. But as a second thing, most entrepreneurs have no clue what causes growth. We just think, ‘Build a great product, put it out there and it’ll just grow on its own,’ through some magical force, maybe if we sell it through word of mouth it’ll grow super fast through word of mouth, but word-of-mouth is actually a very slow growth process. In the book I talk about these three things I call engines of growth, which are the true sources of sustainable growth.
The rule of sustainable growth is: every new customer comes from the actions of past customers. Either through viral growth, which means, that the past customers are an infection vector for the product. Something like Facebook or PayPal. Remember, in an epidemic, a virus is not optional. This is not word-of-mouth. This is not a voluntary thing. These are people involuntarily affecting our firm. Or, the past customers are coming back as repeat customers, as in a product we call the sticky engine of growth, where it’s addictive, there’s a network effect, for some reason the engagement is incredibly high. Something like a cable company or World of Warcraft. Or something called the paid engine of growth, where we take the revenue from past customers and we reinvest it into customer acquisitions through things like outsourcing.
Andrew: I’m testing to see, do I have one of those three? And if I don’t, then I’ve got to back away and it doesn’t matter how many new people I could get. If I don’t have it with one person, I’m not going to be twice as likely to have it with two people.
Eric: You’ve got it exactly right. And here’s the thing, we talk about this thing called product-market fit. This is one of those concepts that used to make me crazy as an entrepreneur. The thing about product-market fit is, if you’re ever asking, ‘Do I have product-market fit?’, then I know for sure that you don’t. It’s one of those things where you know it when you see it. When I was talking to Mitch Kapor, the founder of Lotus Development Corp. in the ’80s, remember? And Lotus wanted to create a commercially successful spreadsheet.
Andrew: They made computers meaningful.
Eric: They sold $54,000,000 worth of Lotus 1-2-3 in their first year of business. They went public the next year. These are 1983 dollars we’re talking about. So they never asked, ‘Do we have product-market fit?’. Their products were flying off the shelves as fast as they could manufacture them. When you’re in that situation, that’s great, but if it’s not happening to you, the concept of product-market fit does not help you know, ‘Am I getting closer to product-market fit or am I hopelessly far away?’ And I think that is the root of that frustration you talked about, which is, ‘Gosh, how does getting a handful of customers help me grow?’ It seems really slow but actually, those customers help you to assess whether you’re getting closer to that engine of growth. Remember, each engine of growth is a feedback loop, and like an engine, the faster it turns over, the faster you will go. So if you can get the feeling that the engine is revving and it’s almost turning, then you can start to realize, ‘Oh, I’m getting really close to product-market fit. Let me keep experimenting, let me get that fit going, let me get the engine turning, and then let me crank up the fuel.’ And you don’t have to the big launch and grow crazy fast.
Andrew: Let me ask you something. Do you have another half hour for us to spend together? What’s your schedule like right now?
Eric: I think we’re good.
Andrew: Another half hour is good?
Eric: Yes.
Andrew: Good. Here’s what I’m thinking we could do. You tell me how you feel about this. I’m thinking we’ll spend another ten minutes (and this is staying in the interview, I don’t edit out so the audience can see as I plan, here).
Eric: This is iteration, live iteration.
Andrew: Live. I figure we can go through two more tactics from the book, and then close this part of the interview, and open another part of the interview where we just talk about how you built up this huge movement. I hear you talk about this, and I say, ‘These ideas are great, but it’s more than the ideas.’ and I’m not picking up on that in this interview, and I’m holding myself back from asking questions like, ‘How did you get all these lean startup circles established across the country? How did you make your book a best-seller? Why did you package your books in bundles of dozens, hundreds, and so on?
And what did you do to insure, anyway, so we’ll get to that.
So I’m going to spend a few minutes on this and I’m going to close this out. We’ll going to do two posts out of this.
So here’s the next thing, you said you got to find that engine and grow, and you told this great story in the book, about Votism, That they identified four, is it four?
One, two, three, here I’ll read it, Registration, Activation, Retention, Referral, and Revenue. That they said those four things, one, two, three, four, five things were essential for them. And then they started tweaking each one.
Can they get enough registrations, can they get enough activations, can you tell that story?
I feel like if I can’t even count the number of key assumptions that they’ve made then I’m not the right person to tell their story. But their story I think is really interesting and useful for the audience to hear.
Eric: Well, I’m happy to tell it. I’m a real fan of this company. The acronym that you have trouble remember is the key to have a good acronym. And Dave McClure has this really famous presentation called, ‘Start up metrics for pirates’, that’s because the acronym falls. ‘AARRR’, For Acquisition, Activation, Retention, Revenue, and Referral. Anyway that’s just a specific way of modeling the metrics, for a start-up.
And just to get a sense of what Votism is, this is a product that’s designed to increase civic engagement, among the citizens in a democracy, who want to get involved in a democratic process. And their most famous product is one that allows you to basically tweet your support for a specific issue or candidate and they translate that into a physical paper letter that gets sent to a relevant politician. So we’re taking your digital involvement, and they can get translated into the currency that politicians can actually understand.
That’s not how it started at all. And in fact they spend a lot of time trying to build out first couple of versions of product, through all that political oriented social network. And the cool thing about Votism, is that they, of course, they use this ‘R’ model of metrics. They were able to kind of make predictions about what was supposed to happen. In a social network, you should be getting really high levels of referral, at people like virally spread their product to their friends, and they should have really high retention, if they get sucked in, to the network effects and all that stuff.
Again, when that wasn’t happening, then it became easy for them to say, ‘Oh, we need to pivot’. But here’s the thing, this is the really key point, and this is a little hard to get across the interview, but this is something you actually will have to read the book to get.
We call it, innovation counting, which is a specific way of tracking your progress, using one of these frame works. That helps you figure out if you’re hitting the point of diminishing returns or not. It’s really important because, here’s what happens, most of us have been trained in our job. That if we optimize, we make our product easier to use, better performance, higher quality, then the incremental investment will yield incremental results.
I found this out the hard way, and they found this out the hard way too at Votism. If you had the wrong product, people don’t want it, making it easier to use, just helps them realize they don’t want it, faster. It’s actually incremental investment for no gain. And that actually is the sign that it’s time to pivot. It’s that you’re hitting the point of diminishing returns, on the experiments, that you run.
And so we show in the book, the Votism story is one that’s doing several pivots. Each one they would start to optimize, now we got the model, and they start to optimize it, make it better easier to use, you see this, they just hit this plateau on those key metrics. And they’re working just as hard, they’re just not getting any kind of return, so then they pivot. The sign of the successful pivot is, now your experiment actually get more productive, not because you’re working harder, because now, you’re working smarter.
Andrew: OK. And, you’re right, to watch them actually adjust and pivot, and measure, is for me was the best part of the book. Because it’s not the kind of thing that translated well on stage or…actually I haven’t seen it on stage, but in an interview it’s hard to get it out there. But it’s key, if you guys are reading the book, you got to go the Votism story, just watch how they figure out where their model is.
Finally, tactics, now excuse me, metrics. You gave the story of Rocket, they were looking at their metrics and their numbers were going up, right? They were getting more and more users, but what was happening there?
Eric: I call these the vanity metrics. And these are the first of all start-ups. Vanity metrics are the numbers you put in your press release. So that when TechCrunch writes about you, it’ll make your competitors feel bad. That’s what they’re for.
And the reason companies like to tell TechCrunch their vanity metric, because vanity metrics make your company look good, without actually revealing, any actual information about what’s going on.
So I go on TechCrunch, and I like, ‘ bon, bon, bon, bon’, I got 10 million messages send to my platform. You like, ‘Wow, that’s huge’. Is it 10 million people, who all use my product once, and then turned out. Or is it, one guy who really, really likes my product, and scripted his web browser and he’s going crazy.
You can’t tell the difference. Well, if you can’t tell the difference, there is no real information and here’s the problem. With Manatee Matrix, if you use that to measure your progress, you’re totally screwed. There’s a bug in human psychology. It’s the second bug we talk about today. And that is, when the Manatee Matrix’s go up, why is that? Well it’s always because of what I was working on at the time. So when I meet with engineers and say, hey how do you know you’re making a progress, they always tell me the exact same thing. It’s always like, well last month we did these awesome features, and this month, we’re getting way more hits to our website, so obviously, QED, that thing caused this thing. And I was like, oh interesting, I have this alternate explanation. I heard that mercury was in retrograde last month, and I believe in astrology, and so if in astrology if mercury is in retrograde, then everybody’s numbers go up the next month.
And engineers get so offended. Like, what, how dare you, you’re crazy. And I’m like, wait a minute, I’m just visiting, but you live here, and neither of us knows what caused your numbers to go up. So who’s crazy exactly? Right? If you can’t tell the difference between astrology and science, you’re in trouble. And most of us are currently doing product development in astrology.
But here’s what’s even worse. When the numbers go down, now who’s fault is it? It’s never my fault. It’s always those idiots down the hall, who sabotaged our great products. So like engineers are like, oh but we made these great features, but those idiots in marketing put a bad campaign out and that’s why the numbers went down. But if I go to talk to marketing, they went, we did this great campaign, and the numbers would have been up but those idiots in engineering. It’s always like, [??] pointed across.
So in vanity Matrix land, everyone gets to live in their own private reality, or the stuff they do always makes the numbers go up, and the stuff those idiots down the halls do are constantly sabotaging them. And is it any wonder that those companies wind up in this horrible [??] warfare.
So in the Grocket story, Grocket’s an online education company that I think is going to be very disruptive. They started out in test prep, and in test prep they had this really innovative solution that allows students who are studying for something like the GMAT or SAT to go online and study in a group with other people together so they can basically work together to figure out the answers and teach each other. It’s a much more fun way of studying, much more effective way of studying.
When I met them, they were using the classic vanity Matrix for that kind of business to judge success. Something like, one of their matrix was, how many practice questions did students answer yesterday? And that’s a giant number, but each student was on there for a long time and they answered all the questions and when that number would go up normally everything’s good, but it doesn’t tell you if the changes you’re making for the product actually caused the numbers to go up, cause the numbers go up because you’re getting more press and excitement, and word of mouth. Like if you just took a month off, and nobody came into the office and didn’t do any progress development at all, your numbers would probably still be up into the [??] Word of mouth will keep going. Your engine of growth will keep turning.
So we really had to transition that company from these vanity matrix to what we call actionable matrix. Cohort-style matrix and split-testing are the specific techniques. And that’s a way of looking at your business. It helps you get a sense of whether your actually impacting customer behavior or not.
Andrew: Cohort meetings say all the people who signed up using this method at this time are going to call them one group of people and we’re going to watch them. Are these people using the system more? Are these people returning? Are these people sharing with their friends? And then you do something to the site. Build a new group of people who interacted with this something that you did and measure that. Now how do you know whether the matrix that you’re using for these cohorts are vanity matrix or not? Because the number that you gave me made sense. It’s not like they said, well do we have more Twitter followers in Grocket today than we did yesterday. Yay, we did, so we’re ahead. They actually looked at engagement on their site. And they use that as a measurement.
So it’s not the kind of matrix that I instantly would call a vanity matrix. How do you know what the right matrix is to follow? A general rule of thumb is gross numbers. Big totals are vanity matrix. Poor customer, unit cost, poor unit, conversion rate type numbers, generally are more actionable. And, of course, in the book I lay out a lot of detail with it, specifically how you do it. But that’s the rule of thumb. The reason we spent so much time on engines of growth, is that the engine of growth can help you identify the critical few numbers that are the most important for your business because you’re running a viral business and really what you want to know is everything related to the viral co-efficient. And how much money you make per customer kind of doesn’t matter. You can always optimize that later. [??], you’re doing paid advertising, you need to know what the marginal profit per customer is and know the viral co-efficient really is much less important.
Eric: I’m smiling because it’s makes sense, given the earlier parts of the book. Absolutely.
Eric: Exactly. It’s tricky. Like people want slogans and quick answers and the truth is, [??] are so frickin’ hard, slogans are not going to get it done. It requires a deep understanding of these E-principles and that’s why I wrote a whole freakin’ book about it. But before we get to that, let me just finish by saying, there’s one thing that happens with cohort matrix, which is you start to feel like there’s a conspiracy against you. I became really paranoid when I first started using Forward Metrics, because, I mentioned we used to do that five dollars a day thing at entry, so we would literally bring in a hundred customers every day, so each hundred customers is the cohort, and we would ask ourselves a simple question like “Out of the hundred, how many of them bought the product?” and it was really creepy when it was the same every day, like one out of a hundred would buy the product.
After a while, it felt like there was this conspiracy, literally today’s customers were all getting on a conference call with yesterday’s. “Okay guys, how many of you bought the product? One?” But no, that’s impossible. That couldn’t be happening, but how could it be so consistent, especially when we’re making the product better every day? Right, so when it’s a couple days in a row you’re like, “All right, whatever, maybe the things we did from the last few days don’t matter.” But when it’s a month and it’s the same every day, but we’re fixing hundreds of bugs, adding dozens of new features, making the product so much easier to use, how can the conversion rate be the same? Well, the only logical conclusion I think from that experience, is that what we think is better actually isn’t. We’re wrong about what product we’re building, we’re wrong about customers wanted. We are in so much trouble. We’ve got to pivot. That’s why we use these methods, it forces us to make that pivot realization faster, sooner, quicker, cheaper.
Andrew: All right, I’ve been recommending the book, but really, frankly, I get nothing out of it. Eric already is on top of the world. The real sign that this book is the book that you need to read is the guys who invest money in start-ups. The guys who give entrepreneurs cash and their trust are saying, “This is the book to read.” When you see Mark Souster[sp?] say, “I got the book for free but I’m buying it because it’s important to support this movement”, when you see him tell his entrepreneurs that this is what they should be reading, it says something. When you see Fred Wilson say, “Guys, this is the method that I am using, that I recommend,” to the people he’s handing millions of dollars and his trust to, that’s the statement you should listen to, not what I say, not what Eric says. So, I’m going to leave it with two things: congratulations, Eric, and the book is called “The Lean Start-Up.” Thank you all for watching.
