How to systemize idea validation

+ Add to

If you want to launch a new product but you want to figure out if it will even work before you launch it, today’s guest has found a way to help.

Thor Ernstsson is the founder of Alpha which helps teams make better decisions based on users, products, and new markets.

Here’s what that means, before you invest time in coding, before you invest time in marketing an idea, Alpha helps you understand what people will really think of your idea.

Thor Ernstsson

Thor Ernstsson


Thor Ernstsson is the founder of Alpha which helps teams make better decisions based on users, products, and new markets.


Full Interview Transcript

Andrew: Hey, everyone. My name is Andrew Warner. I’m the founder of Mixergy, where I interview entrepreneurs about how they built their businesses. If you are a business person—business person? If you’re an entrepreneur, if you’re a startup person and you want to launch a new product, you want to figure out before you launch it, before you invest time in coding it up, before you invest time in marketing it, whether it will even work out.

Well, today’s guest has found a way to help you decide that, to help you understand based on what people will really think of your idea. So what they’ll do is they’ll take a mockup of your idea. They’ll do some variations of it. They’ll put it in front of real users. They’ll tell you what those real users think and help you decide whether you should continue with it or not.

His name is Thor Ernstsson. He is the founder of Alpha. Alpha helps teams make better decisions based on users, products, and new markets.

This whole thing is sponsored by two companies. The first will help you schedule calls with your clients. It’s called Acuity Scheduling. The second I wasn’t going to include in this interview, but then Thor told me he’s been a client of theirs, paying them for eight years, so I’ve got to use them. It’s HostGator. It’s great for hosting websites.

I’ll tell you more about those later, but first, good to have you here.

Thor: Good to be here.

Andrew: Thor, you’ve got to understand. I’ve got notes on your right here, a lot of information about you. But at the very top in bold, big letters and highlighted in yellow is they don’t disclose actual revenue. Whatever is here in our notes, you better keep private. So to make sure I don’t actually reveal the numbers, I deleted it. But I will let you give me what range you feel comfortable with because I do want to give people a sense of how big the company is, but I don’t want to get you to a point where you feel threatened and the rest of the interview is shut down.

Thor: Oh, no. I’m not threatened at all. Where we are as a company is we’re reasonably early staged by some metrics and very late staged by others. So we work with almost a third of the Fortune 100. So we have our clients that are from the biggest, gnarliest, biggest companies in the world to work with. We helped them move a lot faster. So, when we do that, what we don’t want to do is give them ammo for people internally to get into political battles over one vendor or another.

So, if you compare Alpha with McKinsey, it should be apples to oranges, but in reality, that is often the comparison that we get because we enable the teams to move not just through our experiments, but get data behind management decisions that normally you would have to go to very expensive external agencies and consulting firms for. So we help you do that.

Andrew: Why would that mean that you can’t reveal your revenue?

Thor: It’s not about not revealing the revenue. It’s about if somebody within the organization says, “Well, McKinsey has been around for 100 years and they do this and this,” versus Alpha, that’s effectively a startup. So we don’t look like a startup to our clients.

Andrew: I see. Because you guys were founded just three years ago, 2014, I guess almost four years at this point, it seems like a startup. If we reveal the revenues, it might seem—it’s definitely smaller than McKinsey, but it makes you guys look small. Meanwhile, to me, I’m looking at the number here—can I say it’s in the millions?

Thor: Of course, yeah.

Andrew: It’s in the millions of revenue for a new company in the startup world. That is really impressive, but I can understand why you don’t want to say a number that would then show that you’re smaller than McKinsey and put it in black and white terms. Okay. I get that. The whole thing started when you were trying to do this yourself, right?

Thor: Yes. Before Alpha, I had a company in D.C. in the healthcare space where what we were doing was trying to figure out how to get people to care about their health.

Andrew: Was this company called Thor’s Hammer?

Thor: No, that was a consulting firm.

Andrew: Casual Corp?

Thor: So the history is Thor’s Hammer . . .

Andrew: We just lost you. Hang on, I’m going to call you right back. I just lost you. There we go. We’re back. The history was Thor’s Hammer was what? This was a company you started in 2001, ran it until 2010. What was that business?

Thor: We were helping entrepreneurs, basically, mostly around tech and product, but many other things too, business concerns, general business consulting and technical consulting, had about 12 people working on different kinds of problems and challenges, worked on three or four startups in that period that were all very interesting. But what we found was there’s always a common problem, which was that once we build something, how do you know if anybody wants to use it.

Andrew: Did you have a big example of something that you built that no one used and you felt like this is the end, I can’t allow this to go on?

Thor: No, but I have many examples of something where we built it and they were just shocked that nobody used it. It’s just like such a good idea.

Andrew: What’s one that stands out? Give me a picture of something that if I were sitting in the room with you guys, I would think it’s a great idea, but you have to run ideas, even the great ones by potential customers. Do you have one that’s an example?
Thor: Yeah. There’s a simple one around supporting people in need. If you have cancer, being able to receive gifts or support or money or anything like that and being able to easily facilitate that transaction, help a family member, a friend, whatever. So everybody we talked to loved the idea, everyone. But once we actually built it and launched it, we ran into unbelievable friction, which made sense in retrospect and we should have figured it out, but it was that people felt awkward looking like they were asking for something.

Andrew: Yes.

Thor: Setting up pages for other people where they can receive something, the human element of what they have to do, especially in a vulnerable time like that made it very difficult, even though it’s still a great idea. Obviously, you have the financial versions of it on GoFundMe and all that stuff. The more support use cases aren’t really there.

Andrew: Okay. That makes sense. That’s an example where you said there should be a way to do this that’s better. There should be a way to know ahead of time before we disappoint ourselves and look like fools in front of our team, our investors. It seemed like it did well for you. What ended up happening with Thor’s Hammer?

Thor: We ended up folding it into dev shops that were doing social gaming, led to Zynga, where we’re making virtual—

Andrew: Does that mean that Zynga bought you guys? You merged it into another company and then Zynga bought you? Is that how you became their lead architect?

Thor: Not quite. The relationship with Zynga started as an M&A conversation because at the time, they were growing very rapidly, started talking to them and there were about 40 people, and then by the time the conversation was real, it turned into 120 people. And then at that point, I moved to Baltimore to set up their first remote studio, and we were going to use potentially Thor’s Hammer as an external dev shop if we needed to, but long story short, we didn’t need to.

So we kept making games, kept doing things like that. That turned into the core team of Casual Corp, and we were also working on this healthcare company in the meantime called Rally Health, where we raised $55 million and sold to United for $2 billion.

Andrew: You created that company?

Thor: Yeah.

Andrew: Am I missing something on your LinkedIn profile because I didn’t see that on there?

Thor: Audax/Rally Health?

Andrew: Maybe I just overlooked it when I was looking. That’s impressive. So you had a really impressive career. I get how far you’ve come. Do you remember the day that you sold that business?

Thor: Yeah. It was 2012 was the first time, and then United came and bought it. It sold to Cigna and United came and bought it. And I’d say within three days from signing the paperwork, I moved to New York. I wanted to be in New York. I’d been on my way to New York for a while, originally from Iceland. It felt so great sort of wrapping something up and then coming here. Obviously, nothing is quite that simple.

Andrew: But you did it. And you personally did well financially. You suddenly became a millionaire.

Thor: Yeah.

Andrew: The only thing you did for yourself is go to New York. You didn’t buy a crazy car. You can’t because you’re in New York. You didn’t take yourself on a vacation or anything.

Thor: Spent a lot of money over a period of years. The most expensive is when you hire a dozen people and pay them out of your pocket. It’s expensive.

Andrew: This whole company, Alpha, is out of your pocket?

Thor: It was in the beginning and then Casual Corp was before that, last five years.

Andrew: As you’re telling me this story, all I think about is you wanted to come into the country and we said to you—we, the U.S. government—said, “You don’t have a citizenship. You shouldn’t be working here,” am I right?

Thor: That’s right.

Andrew: Where did you want to come from, and why did they say no to you?

Thor: I’m from Iceland. When I moved here, the first place I moved to, which is in the news a lot today, was Alabama. It was an interesting place, especially because where I was, was Huntsville, Alabama, which is one of the highest educated cities in the U.S., but it’s still in Alabama. The predominant industry is Department of Defense contracting and NASA. To do that kind of work, you have to have a security clearance. To get the security clearance, you have to be a U.S. citizen, which I was not.

Andrew: I guess I get that. Were you allowed to do other work in the U.S., or did you have to go back to Iceland?

Thor: In the beginning, no. I didn’t have a work permit, didn’t know the real process. I still got a job, basically just working for cash and super shitty jobs.

Andrew: I see. One of the things I’m uncovering in these interviews is there’s so many great people, entrepreneurs outside the U.S. I can’t have a political point of view on everything, but this is the one thing that I care about. If we have great, smart entrepreneurs, bring them in. I’m almost to the point where I think North Korea is right where they were kidnapping the kind of people they needed in order to grow their country. I’m not saying we should kidnap, but we should do everything we can to get a guy like you into the country. Gift basket maybe?

Thor: The place I moved to, that’s really what it’s all about. It’s where the space program was headquartered because of literally rocket scientists from Germany.

Andrew: That was the beginning, though, that’s not today.

Thor: That is not today.

Andrew: All right. So then you’ve got years of experience doing this. Why did you decide, “I have to start a company”? What was the opportunity you saw that made you think, “This is what I’m going to bank my life on the next few years at least”?

Thor: I’ve had to do this so many times manually that there’s a real frustration around not having the right toolkit, not having the right capability. So I’m not going to hire McKinsey to go tell me if I should build an app or not, but it’s literally what companies have to do.

Andrew: They literally hire McKinsey to tell them, “Should I build this app? Is there a market for it?”

Thor: Absolutely.

Andrew: I had no idea.

Thor: There’s a great little tidbit I saw yesterday from AT&T, where AT&T hired McKinsey in the early ’80s to tell them what the market for cellphones was going to be in the early ’80s. They told them by the year 2000, the total market size for mobile devices is going to be 900,000 units. So AT&T did not push into mobile because there wasn’t a market. Of course, the real number is like 738 million in 2000.

Andrew: You’re basically telling me more than everyone in the country, more than one phone per person.

Thor: Global.

Andrew: Oh, I see. I got it. Wouldn’t that make you think all this research, if the top people can’t get this right, it’s not doable. You have to experiment, get into a market and the answer isn’t to know how big the market can be, but how small can you get into it to test to see whether you should continue. You’re nodding. You’ve heard this line of reasoning before. What’s your response?

Thor: That’s what we do at Alpha. That is 100% what we do. We don’t do market research, we don’t do sizing, we don’t do consulting. We don’t do any of that. All we do is get you data as fast as possible to make decisions like that. Do people want A, B, or C? Should we build this product isn’t really a question that we can answer because the business gets behind it. But if we were to build something, should it be A, B or C? Or you have a product backlog of 30 features. What three are the most important and why? You have limited resources, limited time, whatever.

Andrew: I want to hear more specific examples of people who you’ve helped make decisions like that and some case studies here, but let me take a step back. When you say that you did this so many times that you realized someone has to do it for me, I get that kind of a problem. Can you tell me a little bit more—when you did it yourself, when you finally said, “No more of this idea where we’re going to help people with cancer get funding and realize only once we’re starting that no one with cancer wants to go out with their hand out in addition to saying, “I’m dying,” say, “Give me some cash.” Talk to me about the manual way you did it before Alpha existed?

Thor: So I hired 60 people and had three teams running independent experiments, product design, dev, where we would have fake sites, we’d have mockups, partially built products, then we would drive about $200,000 a month of Google traffic to it. Then we would just see what people clicked on, what resonates, what doesn’t resonate. So every two weeks, we’d run a different experiment across all three. We’d have basically six experiments a month we’d be able to do. It would cost us all in all about $1.5 million a month.

Andrew: And this is at Zynga.

Thor: This is at Audax. This is after Zynga, the healthcare company. At Zynga, it was totally different experience because we’re not dealing with anything sensitive. We’re literally selling virtual pig trackers. So we would just put something up, let it sit for a few days, for a few hours, and we’d see how people reacted to it. Coming up with the idea still took just as long. It might take six months, nine months to build a game.

The way we’d figure out, “Do you Farmville in outer space or Wild West or wherever,” is actually putting up ads, putting up products, putting the first few clicks together, the first few screens, the first-time user experience and then testing it against the target demographic, which was 43-year old women.

Andrew: Okay.

Thor: That’s the core user of most social games. Through actually seeing what people did, we learned we should build this and not that, then in healthcare, realized there’s a lot of red tape, a lot of hurdles, HIPAA, there’s regulatory issues. So you can’t just launch products that put your health information out there and hope for the best.

So what we did was we replicated the process and the capability we had before but had to do it before. Then afterwards, with Alpha, we don’t outsource it. We actually just give you a platform to drive it at scale so that you can get almost instantaneous feedback. We have 100 million users. So you can target just about everybody and test your idea.

Andrew: 100 million users now at Alpha that you can reach out to. They’re not in your database. You’re saying you can go out and reach them by buying ads on Google or wherever, right?

Thor: They’re in our database capability for 100 million users, then we actively test and pay about 50,000 or 60,000 people every month.

Andrew: When you say that many, 100 million, something seems off in my understanding. I imagine that you have 100 million people in a database and when you want to reach them, you can. It sounds like what you have is access to them through some kind of relationships, right?

Thor: We have publisher networks. It’s 100 million active users a month, just to be clear. So it’s not static people in a database.

Andrew: Through the publisher networks and then you reach out to them if you need to.

Thor: Let’s say I want to target entrepreneurs, I would get to, because the audience are entrepreneurs, to incentivize them somehow.

Andrew: How would I incentivize them to go check out?

Thor: Whatever might be appealing to them. So let’s say you’re a New York Times reader and you run out of articles for the month and we know something about you, you drive a Tesla or whatever, and the New York Times would say, “Get 30 free articles in exchange for giving feedback on your experience with your Tesla or new products that might help your ownership experience.

Andrew: I see.

Thor: So it’s a highly contextual, highly targeted. You’re going to be interested in doing it no matter what. And you get 30 free articles out of it and the New York Times gets $5 or whatever. So everyone gets what they want out of it. The overall experience is very organic, and it doesn’t feel like you’re being paid for your opinion, even though technically speaking, that’s what it is.

At Zynga, we would have offer walls where you get in-game currency. You get tokens. You get special limited edition items. It could be signing up for Netflix. It could be for answering a few questions. It could be at one point, we had a thing where we go to 7-Eleven and buy whatever they had, they literally had like Farmville-branded apples. You would get an in-game item.

Andrew: If you bought the apple, then you got an in-game item to incentivize people to buy the apples that you guys were partnering up with. I see. Got it. Let me take a moment to talk about my sponsor, then come back and say this is a pretty big project, pretty big business. How did you start it? What’s the first thing that you did and then take it from there?

But my sponsor is HostGator. I was going through a list of potential sponsors. You said, “I’ve actually been with HostGator,” for how long, right years?

Thor: 12 years.

Andrew: Doing what?

Thor: So part of the consulting business back in the day would be to bring offline businesses online. So there’s still probably, I don’t even know, 30 to 40 clients that use technically speaking, my hosting through HostGator, even though I haven’t touched it in at least a decade. Now my son uses it as this playground for different web projects.

Andrew: So whenever he wants to experiment with something, he just goes to HostGator fires up a new—yeah, that’s something that I think most people don’t realize. When you’re with HostGator, you can have unlimited domains, pretty inexpensively, which means that you could create anything. My wife and I started dating. I did something that was that was—I forget the word that she used—it was something like uncouth for what you did. I went back and copied a page from the encyclopedia for the word “uncouth” or whatever the word was. I put my photo next to it and I put it on and I sent it to her. It made her laugh. It costs nothing because you have unlimited domains. You could do something like that.

Thor: Exactly.

Andrew: Your son, what are some of the projects that he’s working on with HostGator?

Thor: Chatbots. We’re going to do some Alexa stuff. He’s got all kinds of different web hosting projects, games. He builds a lot of games, minorly interactive, but it’s all learning.

Andrew: So whenever he wants to try something like a chatbot, he might create a website to house this little project. He just goes to HostGator, fires up another domain. Guys, if you haven’t signed up for HostGator, I keep talking about the unlimited domain plan. Go check it out. Frankly, if you hate HostGator and you don’t care for me in the way that I’m reading these ads and you decide you want to go with someone else, go for it, but also look for unlimited domains.

The power of unlimited domains allows you to do what Thor’s son does, which is experiment, learn by playing, by doing. The reason I recommend HostGator is because people like Thor who I’ve interviewed have said over and over that they have had or still have an account with HostGator because it’s so popular, because it’s so good, and because it frankly does the job. At some point, you don’t need the best hammer on the planet, you just need a hammer that hangs nails in.

You can create a website with them and have it work. I urge you to not sign up for the cheap plan, even though it doesn’t affect me financially which plan you sign up for. Go up in level, go up to at least a baby plan, which gives you unlimited domains. Along with that and any other plan you pick, you’re going to get unmetered disk space, so no one’s watching you to make sure you’re take up too much space, unlimited bandwidth, no one’s watching you on that either, unlimited email addresses and a 45-day money-back guarantee in case you decide I’m full of it and you don’t want to sign up with them anymore or be with them. Get your money back.

Go to Think of that gator taking a bite out of our butt— I’m trying to make it visual for people, make sure they remember it.

All right. So the first version—you had this idea, you said, “I want to go out. . .” You told our producer, “I went out, the first thing I did was talk to Fortune 100 companies. Why did you talk to them, and what was it that you were doing for them?”

Thor: So what we optimized around was learning. So I’m trying to solve the hardest problems to go a lot deeper to getting some startup buddies to pay you $5 a month to use the enterprise SaaS platform. So we don’t want to be a startup that just services other startups. We actually want to solve a real problem in a real industry. So we started with not just big companies, but heavily regulated companies — Citibank, Pfizer, big companies that have a lot of issues around getting data, controlling data, understanding users, all these things. Yet, they have a CEO and a market that’s telling them to be more focused on their users, to be more customer-centric, to be more patient-centric. They don’t have the capability or the tool to do it. The first two years is really us learning how do we better do that and how do we better help them. Then the last two years has been scaling the business. So how do we get the rest of the Fortune 100 and then Fortune 1000 onto the platform and moving a lot faster.

Andrew: Thor, you said that you didn’t want to be a startup selling to other startups. One of the reasons why so many startups sell to other startups is they’re accessible. They’re your friends. They’re at the same events. They’ll take your phone call because they’re hoping somebody’s calling them or emailing them. When you’re talking about Fortune 100, there’s so many people at those companies, you don’t even know where the right person is, let alone how to reach out to them. So how did you talk to so many and actually sign a six-figure deal? How’d you know who to talk to and how’d you get them on the phone?

Thor: We didn’t.

Andrew: What did you do?

Thor: We bought a bunch of email lists. We emailed probably 10,000 people. We split tested everything — job title, job function, email subject, call to action, everything. It’s not what word resonates. We’re literally split testing the entire business value property before we write a line of code. So will people respond to getting on demand insights? Will they respond to prototyping? Will they respond to audience access? We didn’t know. So it turns out the way to figure it out is by running experiments, testing it, put himself out there, see what resonates and do it over and over and over and over. Then you learn.

Andrew: What did you learn from all those different email tests?

Thor: So you iteratively move towards a product. So you start with testing the concept or the idea or the language around it, but then you work your way towards a real product as soon as possible. So what we learned is that even though every big company thinks they’re the worst, they think they have the most awful procurement, the most awful politics and all these barriers and red tape, in reality, they’re all very similar. They’re not that different than a startup. The hurdles are different. You don’t necessarily have a board and limited budget and one engineer. You have 500 engineers and unlimited budget. But in reality, you still have to make a decision on which 3 of those 30 features do I build right now. It’s the exact same decision point you’re going to come to over and over and over.

So the problem we solve applies to everything from startups, especially like tech startups that are trying to innovate, that are trying to [inaudible 00:25:00], that are trying to out-compete with limited resources all the way to the biggest companies in the world.

Andrew: So that now seems obvious, maybe because you and I have been talking for 25 minutes and I get now the value prop. What wasn’t as obvious before? What would you have thought of? If you hadn’t sent out all those emails, where would you have thought to take this business if you were just saying, “I have a lot of experience, I know what they need. I’m going to present it this way. Here’s the solution.” You made it better by getting their feedback. I think I just lost you there. Can you hear me now?

Thor: I can hear you.

Andrew: If you hadn’t done all that or before you did that, what would you have thought you needed to build?

Thor: I would have thought and still do think that getting access to data in effectively real time is the most important thing, being able to do this on demand so no matter what your question is, within a few minutes or few hours, you have data. What we learned is that anything less than a week is effectively real time.

Andrew: I see. Okay. So you thought they need this and they want this immediately while they’re in a meeting almost, but in reality, as long as it’s within a week, it’s fine. But the overall product, you thought that you were right in thinking they’re going to want to know A, B or C, they’re not going to want to know what should we do.

Thor: Yeah. Those are huge decisions that the smaller the company, the bigger the risk because you may go out of business. The larger the company—Amazon has a good quote, Jeff Bezos has a good quote around the larger you get, the bigger your bet should be. So it should scale with the scale of your business. I think that’s spit on because if you’re not doing that, you’re not pushing yourself, you’re not learning and you will have other companies large or small that are going to eat your lunch.

Andrew: You signed a six-figure deal, the company says, “Here’s the money?” Can you say who the company was?

Thor: The first big company we worked with was DirecTV, now AT&T.

Andrew: You said to yourself, “If I can’t deliver what DirecTV wants at the price that they’re paying me, then I’m not going to launch Alpha. I’m actually using this as a test.” What did they want?

Thor: So they were working through a series of products that some were live, some were not live, and they had a bunch of questions. How do they better serve this very specific market? They knew that they did not know everything about the market. So, with us, we told them there’s a platform, you can log in and get your data. What we did not tell them is that everything on the other side of that platform is manual. Then we would use whatever tools are available to generate the data.

We did that for a year. Then we built out all those tools. So now we have an entire, fully integrated platform, everything from the ideation, brainstorming, sprint planning, all that stuff through generating data, running the experiments and then eventually reporting and sharing insights and all that stuff. So we can manage it across a two-person startup all the way to a 400,000-person multinational company and drive the same process. That’s because of what the problem was that we were solving in the beginning, even though that problem might seem like only exists in certain spaces or certain industries, in reality, it comes down to decision making.

Andrew: So you have one client, you’re able to produce. You see this is a possibility. This is working out well. Still it wasn’t a cakewalk, because soon after that, you had a global security company as a client and what happened with them?

Thor: So we had a lot of different clients that were challenging. So we had to do things like send people down to Colombia to interview small business owners in Bogota and Medellin, and obviously that’s not a scalable business. We don’t want to be in that space. However, we had to figure out what the local market needed for, in that case it was accepting credit cards. A lot of the stuff we had to do in the beginning was really, again, around learning.

How do we better help these companies, how do we get them data faster and then we built a platform on the back end as well to be able to access all these people to be able to test to stuff in real time and the most important learning and the thing that we got from the beginning and it keeps getting reinforced even to this day is that these large companies are very complicated to work with. It’s not really about what you’re doing with them as much as it’s about helping an individual or a team be successful in their role.

So we don’t look at a complex organization of hundreds of thousands of people as one company, because there might as well be 80 different companies. So we look at as basically a large scale team of teams. So our unit that we operate on as a team will help the team be successful. Within that, there might be product managers. We make them heroes, decision makers and budget holders.

We help them get the right insights that they need at the right time, and then the whole team uses our product to move a lot faster. But you can’t say that we impact every single employee at the company. That’s not what we do. We solve a specific pain point for anyone looking at digital product, digital channels, content apps, stuff like that.

Andrew: So you started out by doing one, doing it manually and then you kept thinking, “How do we avoid this whole manual process?” the manual process that requires that we get on a flight. That’s when you started to think about things like partnering up with people who have the audiences you need, creating software to make it easier to tap into them. At this point, how much of it is manual and how much of it is still a service business? If somebody wants to test something, is it all happening automatically?

Thor: It is all happening automatically. There’s no [inaudible 00:31:15]. We do have a customer success component where our clients work with a team to help with training and onboarding and stuff like that, but there’s no manual service at all.

Andrew: We asked you in the pre-interview about big milestones in the business. I was surprised to see that one of the big milestones that you considered big enough to make the first one was closing a deal without needing to see a customer face to face. Why was that such a big deal, and how did you do it?

Thor: So that was in the very beginning. It was being able to have a pain point clearly enough articulated and a solution to that pain point. That’s what they buy. They’re not buying things we’re selling it to them.

Andrew: They’re not buying your relationship. They’re not buying Thor who has all this experience in his background and want to be associated with him because he made magic happen for other companies like Zynga. He’ll do it for us. It’s he understands our problem and we believe he can solve it.

Thor: I wasn’t even in the room.

Andrew: It’s this company understands our problem, and we believe they could solve it. That’s what you mean.

Thor: It’s like here is a solution to a problem I have. So the test was is it an important enough problem. Is it an important enough solution so that would stand on its own. If it’s me selling it to them, I don’t know why they’re buying. I don’t know if they’re buying because I sold it to them because of past success, or because it’s actually something they care about. So, your example of startups selling to other startups, two startup founders and Y Combinator, one buys from the other, why do they buy?

Andrew: How much of it is because they’re in the network, because they care about each other. Right. Frankly, that’s a big part of Y Combinator and other accelerators is that you get all these people using your software as opposed to a competitor, which might be a little bit less powerful or as opposed to doing nothing and still being okay. How long after you started the company? How long after that first batch of emails you sent out did you get the first customer who bought without a face to face interaction?

Thor: That’s a good question I don’t have an answer to.

Andrew: Is it weeks, days, months?

Thor: It’s three or four months.

Andrew: Three or four months. That’s pretty fast. How did they come across your website?

Thor: There wasn’t even a website.

Andrew: It was just the emails?

Thor: It was an email and a one-page PDF.

Andrew: That was it? That’s what they sold and they bought how much worth?

Thor: $150,000 a year.

Andrew: Why didn’t you create a website at the time? You know how to create websites?

Thor: Obviously, we did afterwards.

Andrew: But I feel like there’s a sense of at this point, sometimes people rush to create a website and build it and make it perfect of just saying, “Let me see if I can get a first customer.”

Thor: Got it.

Andrew: So that’s what it was for you.

Thor: Yeah. Our first three customers were particularly AT&T, Pfizer, and Citibank.

Andrew: AT&T for DirecTV.

Thor: We targeted those three verticals. We targeted media and tech, healthcare and finance and still do. So we knew if we could sell to those companies and go through their procurement process, their diligence, all the stuff they have to go through. We can work with just about anybody in those spaces. So, now we’re doing the same thing and we’re opening up new verticals around retail, around CPG and a few others and we’re doing the exact same playbook. It’s really just around scaling the team, scaling Salesforce and putting people in the field to help clients.

So the thing we did on day one is effectively the same thing we do today, both in terms of methodology process but also in terms of the product we provide if you look at it as the value prop and how we deliver it. Obviously, the product itself has evolved hugely in crazy ways that it doesn’t even resemble what we started with. But the value prop is still exactly the same.

Andrew: Okay. Let me talk about a sponsor then come back and say you knew what you were going to do. The next step was scaling it up so that it’s not just a couple of projects that you can take at a time. The sponsor is a company called Acuity Scheduling. I noticed that you do something with your website.

Here’s what happened with us. We have this chatbot business where as a service, we’ll build chatbots for clients. We built a website and a sales page and a set of emails. We had thousands of people on our mailing list and you know how much we sold? Not one. I don’t know why. We’re really good at selling via email and landing pages and all that. Then it occurred to me when it comes to a product like a chatbot, kind of like websites in the old days, people want to talk to a human being. They want to at least understand that there’s someone there building their vision.

We said let’s experiment. Let’s send out an email to people saying here’s what we can do. Schedule a demo. Anyone who clicks schedule a demo could go to the calendar of the person who we hired to do these demos, picks the dates and times from her calendar that they want to get on a call and immediately have the whole thing on their calendar and a reminder and the whole thing taken care of.

To do that, we used Acuity Scheduling. So, here’s the thing. We went from closing zero sales, zero percent, to closing about 50% to 60% of people who are interested in chat bots suddenly got on a demo and bot because they got to talk to a person for just 15 minutes. That’s all it took. I say that this is, Thor, something that you do because I noticed on your website too the option isn’t press this button and go by, it’s request a demo, which then leads to a scheduler.

If you’re out there and you haven’t tried this, you owe it to yourself to do it. The reason I advocate Acuity Scheduling and we use them for years long before they were ever a sponsor is because it scales with you. You discover something like everyone needs to get a link to our Zoom meeting so that they can do screen share. You just add it to your Acuity Scheduling and it gets taken care of. You realize some people don’t show up because they forget.

You go back to Acuity Scheduling and you say, “Ten minutes before the call, I want to send a reminder.” You realize some people have feedback to give you, go back to Acuity and you send an email for after the call so they could go and do feedback. All these are programmed in and as your team grows, you could add multiple people’s calendars, everyone who comes will see a single calendar with options, but based on when they pick, they go to the right salesperson.

So it scales with you. It connects using Zapier and other tools to every other tool that you guys have. If you want to try it, don’t go to, know that’s for everyone else, go to When you go to that link, you’re going to get extra time to use this. They’re giving you 45 days trial and they’re going to throw in an exclusive, private, one on one consulting session with a pro who will help you set this up right and know how to use it to grow your business.

Try it. Even if you planned to only sell on the website and never have your people talk to a client again, try it for a couple of weeks and you’re going to see how much you learn that will influence the way your sales page is and the way your copy is written in the future.

There’s whisky over your shoulder. Why are there two posters of whiskey over your shoulder?

Thor: There’s a third that’s out of frame. Each conference room that has a theme that isn’t really tied to anything other than random words or things that we came up with. So it’s whisky, bacon, coffee and waffles.

Andrew: Is there any connection with those foods? Are those your favorite foods or the most popular in the company?

Thor: Waffles, we had Waffle Wednesdays in the beginning, where we would experiment and split test different waffles every Wednesday.

Andrew: You literally would split test waffles?

Thor: Oh yeah. We had some disgusting ones and we had some great ones.

Andrew: Are you someone who split tests everything in life? Do you bring this methodology to your personal life, to any parts of it that are interesting?

Thor: Sure. But not necessarily the split testing part, but the learning part. Absolutely.

Andrew: Give me an example of how you do it. I want to get a sense of you as a person.

Thor: Sure. Let’s say you’re trying to figure out a new skill. You can strategize all you want. You can have a business plan for learning a new language or whatever. But really, the way to do it is by iteratively experimenting. Just take one step at a time. Put yourself in a situation you haven’t been in and learn from them. So what I’ll do a lot is if I want to see what something is like, I’ll literally go there. I’ll book a ticket for two days from now and go check it out anywhere in the world. That kind of experimentation, that kind of insight you get from that is unlike anything you’ll ever learn through planning.

Andrew: What’s an example of a trip you took just to go and learn something instead of planning it or researching it on the web?

Thor: It’s almost every—not quite every month, but every couple of months.

Andrew: What’s one that you took recently?

Thor: It’s a late night decision with a few friends at a bar, but recently, there was a Colombia trip, and India last week.

Andrew: What did you want to learn from India?

Thor: Nothing specific, just hadn’t been. I’ve been to Bangladesh and it’s interesting. I wanted to [inaudible 00:40:43] like food and culture and stuff like that.

Andrew: I picture you watching “Jiro Dreams of Sushi” and then saying, “I want to go see what the subway food is like in Japan,” and then you go and do that.

Thor: That is exactly right.

Andrew: Wow. So now you have this idea. It makes sense. You’ve got clients but it’s not scaling because you have to fly people out and have people talk to you. What did you do first? Did you start to staff up or automate?

Thor: Both. So we staffed by hiring product managers that are ops people that automate themselves out of a job. So everybody here in the product role starts as doing the job of the thing they’re building. Let’s say you’re surveying, for example. You’re recruiting an audience or doing interviews or whatever. You start by doing that. Whenever a person needs to recruit, they click a button. What happens on the other side of that button is you. Then eventually, you get so tired of whatever it is that you’re doing manually because it’s a pain in the ass, you have the engineers build that function for you to make your life easier and easier until you’re not even involved anymore, and then you’re managing the product. You’re not doing the operational part. You click a button and magically 100 million people show up.

Andrew: And you have to think about where are those people coming up. The reason you do a good job doing that is because you’ve already been looking at those places as sources and you’ve been doing it manually and you not just want it all automated.

Thor: Exactly. The cost is much higher when you do it manually. Over time, because we’re building the infrastructure around it, the cost goes down very, very dramatically, so the stress to run an experiment compared to anything else is going to be an order of magnitude, both faster and cheaper.

Andrew: It does seem like two different skill sets, though, doing something manual, figuring it out and then being a product manager. Are they?

Thor: No, it’s problem solving.

Andrew: If you can get someone to problem solve by doing the work themselves, they could also problem solve by thinking about all the different things that could be automated to have software do it.

Thor: Yeah. The product part of it is an easily trainable skill set, problem solving not so much.

Andrew: You told our producer that one of the challenges you have is your customers need a lot more help, a lot more understanding than you would expect in the beginning. Can you tell me about that?

Thor: Yeah. That was what I mentioned earlier, where a lot of the stuff that we have to do is helping them better communicate success within an organization, helping them understand different use cases for technology and a platform like Alpha and then helping them really understand how to think through a problem or a product in terms of hypotheses and assumptions. Like being able to say clearly and articulate what are the assumptions that I’ve baked into this plan?

What are the things that must be true and what are the things I’ve assumed are true and might not be? That’s a fairly difficult thing to do in a large organization because there’s so much inertia and there’s so much momentum that could be in the wrong direction. Whenever you start launching a product and kick off your marketing plan 18 months in advance and then you learn, “Oh shit, nobody wants an app to meet your long lost relatives wherever,” that kind of thing might be too late because you’ve already committed a seven or eight-figure marketing budget to it.

So you learning that might not actually matter to launching the product, but what you can do it optimize it. What you can do is make it better, iterate on the whatever the thing you’re doing to actually come up with something that would work.

Andrew: That’s where the success people on your team come in. They’re the ones coaching your clients in this process.

Thor: They help them. We’re not training or coaching like that, but what we are doing is helping identify these use cases and these opportunities, helping structure it properly. Helping contextualize why it’s important to listen to your customer or users or why it’s important to listen to your competitors’ users, the ones that didn’t choose you. Why did they go with competitors? Things like that.

Andrew: Why do you guys charge on a recurring basis? Wouldn’t someone just want a quick answer and move on?

Thor: They all do. That is exactly what everybody does want. But that’s not our business. Our business is adding this as a capability to change how the team works. So the team is faster on a continual basis as opposed to a transactional basis and then going back to business as usual. So we don’t want you to do to something once and then go back to the shitty way of doing things after you’ve seen something once. We wanted to be really transformative to the business and what we found is across every company of every size, once they run, really it’s about two iterations.

So they run one test, maybe two tests and then they get a sense for it. When they iterate on their findings and go a lot deeper, that’s when a light bulb goes off and they see how it can actually change how they do their job, how they make discussions, how they think of digital products. By the way, if you want, you can give the community, you don’t even have to have a special link. So you can give the listeners the ability to run a free experiment on Alpha. Just go to and mention Mixergy somewhere in the—there’s a form on there. Mention Mixergy somewhere and we’ll run it.

Andrew: What’s an example of an experiment that people can run if they say they’re connected to Mixergy? There, it’s at the bottom of the site.

Thor: They can run that test like literally what do people think of my product or idea, concept tests, very straightforward. More interesting ones would be like competitive benchmarking. How do people solve this problem today and how does my idea stack up against that? So you might be thinking of how to build a subscription content business where you sell expert interviews with startup founders into a community of small business people that are trying to figure something out.

Like that’s the kind of stuff we’d be able to test. What we would test would be how do people find information today? Where do they go? What do they read? What do they watch? Then out of all that? How much would they pay for these different things? Why would they pay? Why would they not pay? When you get back, there’s both quantitative and qualitative insights, interviews, all that kind of stuff of people reacting to your product or your concept and your competitors’ ideas as well.

Andrew: You get a full report back. I’m looking at one of the sample reports. This is really in depth.

Thor: Yeah. You get a full shareable report almost immediately.

Andrew: I don’t want to overlook what you said earlier about why you guys charge on a recurring base because what you said there needs to be clipped, copied to every business, especially consulting companies because you’re right, people will hire a consulting company to build something for them, then they’ll say, “Great, I got my app. I’m ready to move on,” and they’ll go back to life the way it was,” or I signed up to have somebody do my email funnel and then go back and try to manage it yourself, but you really don’t understand and you go back to doing things the way it was before. I like the way that you phrased it a lot. I’m going to go back and look at the transcript for that.

Let’s close it out with this—the funding, you said that you funded it yourself and then you eventually decided to raise money. Why did you do that?

Thor: We raised three rounds so far? The main reason was that there’s a different level of investment required to take a bigger bet. So, obviously, building an engineering team is expensive. But we had these clients and revenue, like real revenue.

Andrew: The connection is just starting to get a little wonky. There we go. It’s back. I think what’s happening is we’re using the new Skype. I think it disconnects and then reconnects so fast that I don’t have a chance to call back. You’re back with me.

Thor: Okay. So I was saying the reason we raised three rounds is to make bigger bets is the short and simple version of it.

Andrew: You talked about how you want to go after new markets now. Is that the big bet? What’s the biggest new bet?

Thor: The biggest new bet is regional sales teams, putting people in the field, hiring 50 people across, starting whether the U.S. but then globally.

Andrew: So going from selling without a face to face interaction to face to face interactions as a way of selling more.

Thor: Exactly. We validated that we can actually build it in a very hands-off way and scale it, but what we also learned is the more time we spend with the person face to face, things just move a lot faster and better and everything is easier. So we’re hiring actively in Chicago, LA and SF and we have an office in Utah and we’re going to be hiring in pretty much every major market.

Andrew: I see that, actually. I’m on your AngelList profile. You’ve got a great AngelList name. It’s just Frankly, you’ve got just a great name in general. Thor is a kickass name. But I can see all the different job openings that you have right now—software developer, programmer, business director in Chicago, customer success analyst in New York. You’re looking for an executive assistant for yourself, it looks like?

Thor: I’m not, but the team is.

Andrew: All right. The website, for anyone who wants to go check out and more importantly, maybe run a free test is Before you run a free test, I urge you to take a look at right under the start button for the free test is a show me sample results. I almost think that should be linked up above so that people can see it. I think if you see a sample result, you get a sense of how powerful this is and you’re going to want to take that free testing very seriously. Either way, go check it out AlphaHQ—what do you think of me giving feedback right here in real time on this form that you probably have tested so many times.

Thor: We’ve gone through so many iterations of that. It’s shockingly difficult to get somebody to think critically and ask a question on the spot.

Andrew: Yeah, it is. Think critically and then answer the questions you guys have on the spot?

Thor: No, if I ask you what’s the most critical assumption you have with your business, coming up with a clear answer is actually very different.

Andrew: Yes, that is very hard. You make it so simple to start one of these tests and even these two questions, I’m going to have to sit and spend some time on. If I wanted to test my competitors, the first question is what is the problem you’re solving and who else is solving it and there’s a little bit of detail underneath it and then the second on is what do you want to learn, I have to pause and not just think of what comes to mind, but think about what am I trying to learn so I can do something with what I learn. Anyway, really well done form, easy to understand.

Anyone who wants to check it out can go to I have no relationship with Alpha. So I’m not here to promote it. I’m just telling you this is interesting and looks good. I do have a financial relationship with my two sponsors. The first will help you get calls for your sales people calls with clients. It’s so easy to schedule with this that your people are going to schedule with you fast. Go to The second is a hosting company that Thor and I and so many other people use. It’s called HostGator. Check them out at

And finally, give us feedback on the audio and the video of these interviews. My team and I keep wanting to improve, especially as we continue through 2018. Just email us, the team I are looking through to see everyone’s feedback. Thank you, Thor.

Who should we feature on Mixergy? Let us know who you think would make a great interviewee.