How Doron Reuveni turned app testing into a $100M dollar business

As a user, do you ever try a new app and come across a problem? What do you do? You probably give up on the app and go with one the competitors instead.

Well, today’s guest founded a company that helps app makers identify those problems before that can happen.

Doron Reuveni is the founder of Applause, which offers software testing, usability, and market research.

Doron Reuveni

Doron Reuveni


Doron Reuveni is the founder of Applause, which offers software testing, usability, and market research.


Full Interview Transcript

Andrew: Hey, everyone. My name is Andrew Warner. I’m the founder of Mixergy, where I interview entrepreneurs about how they built their businesses. We get the stories behind the companies. We get to figure out what they did that worked that allowed them to grow, often really, really, really big.

And as a user, do you ever try a new app and come across a small problem? What do you do at that point? Usually what you do is you just give up on the app, right? And you move on to their competitors and their competitors get to benefit. Well, today’s guest founded a company that helps app makers and other software creators identify those problems before they hit your hands so that they can do well and beat their competitors.

His name is Doron Reuveni. His company is called–and I freaking love this name–it’s called Applause. is the domain name, impressive that they have it. They offer software testing, usability and market research, and they do it in an interesting way which we will talk about here in the interview.

This interview is sponsored by two great companies. The first will help you hire your next great developer. It’s called Toptal. The second will help you do marketing automation right. It’s called ActiveCampaign. I’ll tell you more about them later. First, Doron, welcome, man.

Doron: Thank you. Great being here.

Andrew: I love it. How much revenue are you guys doing right now?

Doron: So we’re just hitting the $100 million run rate from a revenue perspective.

Andrew: $100 million?

Doron: Yes.

Andrew: With a business that started out with so little money that you guys were basically scraping cash to try to figure it out. You couldn’t even pay your developers in the beginning from what I understood.

Doron: That’s correct. When we started the company, that was back in 2007, me and a cofounder, we actually had a pretty hard time getting funding. I think part of it is also related to the fact that when you’re a first time founder/CEO, it’s much easier to get it when you’ve done it before. So we couldn’t get any funding.

I think one of the things that we did is we actually really believed in the vision of ensuring quality in the wild across all devices and configurations. We actually gave a small percentage of our company to a small dev shop in Argentina, which was kind of interesting, and they developed the product until the first beta launch. So I was going back and forth between Boston and Buenos Aires at the time.

Andrew: Which is not a bad way to live. But you said you had this need for–I forget the phrase. But the thing that strikes me is nobody wakes up in the morning going, “I can’t wait to do user testing. I can’t wait to start a user testing business.” Where did you get the idea to do this?

Doron: I’ll be honest with you, the idea wasn’t mine. It was actually my cofounder, Roy, who came from a product and quality type of background. My background is actually engineering. The thing that is so in the market at the time was no matter how much I invest in quality and testing, when software gets in the hands of real people, there are still multiple issues that get found.

His concept was relatively simple–what if we build a global community of experts and test engineers and research engineers and actually test on real devices and real configurations? So it was his idea. I was a business partner. I would say that in 2008, that was kind of when iPhone first launched and apps were [inaudible 00:03:29] into the market. IoT didn’t exist. Communities were kind of early stage. LinkedIn was really–it was a pretty innovative idea. Now everybody understands the gig economy, communities and digital workforce and all that kind of stuff.

Andrew: And I can see why he would be thinking about this. I’m looking at his background on LinkedIn. He did QA. He was a QA manager at IncrediMail. He was developer and QA manager at Blue Security. I don’t know those companies super well, but I do see that QA, quality assurance, that’s his focus. That’s what made him think, “You know what? There needs to be a better way to do quality assurance. There needs to be a better way to test,” right?

Doron: Right, especially at scale. I think many things–many things happened in the market since then. Today’s market is much more focused around digital experience and app experience. I’ll give you an example. I’ve had multiple customers that started with us because they had a pain point or an event, meaning they launched at a country and they wanted to cover specific demographics or specific testing. They had their ratings for their apps drop, and they had to figure out, “Hey, what’s going on?”

And this was the way we helped them. So, many times, the customers are driven by an event or a need, but then once they start working with us, they love the speed and the scale and how much velocity they can drive into the development process that they become customers for life.

Andrew: So is there an issue sometimes where a company launches their product in the U.S. and everything goes well, then they launch the exact same app overseas and they have an issue and that’s when they come to you and say, “Can you help us figure out what’s going on overseas?”

Doron: Absolutely.

Andrew: What would cause a problem? What would cause inconsistency in the app performance between two countries?

Doron: Yeah. It’s not just performance. Look at even retail applications, for example. When Puma, for example, launched in 16 countries worldwide, we helped them test all their various stores in different countries. You run into issues like payment validation, different types of credit cards, with pin, without pin, ZIP codes that are being used, not being used, simple stuff like that.

Sometimes it’s even the context of how the app or the website is performing, meaning it doesn’t resonate within the current market environment. What we bring to the table is actually the capability to test with real professionals, which are feet on the ground in specific countries, which provide something that no one else can do if you have something which is really global and at scale, right?

Andrew: I see. And before that–tell me if I’m wrong, but it seems like the way that companies would test was they would have multiple computers and they would test on all those computers and see does it work on a Mac, does it work on a PC, does it work in Safari and Internet Explorer, etc. And then if it tested in their office, they felt like it was good. Is that largely the way things were done?

Doron: Yeah, that’s how things were done, meaning testing in a lab.

Andrew: And you said, “I want to get people all over the world testing in the real environment. So, if you’re going to launch in Canada, I want to find testers in Canada who are going to do the test.”

Doron: Correct. Thank about it. Today when you talk about device and form factors, how many versions of Android do you have? They’re different in every country. You’re talking about media, streaming media. We test for Fox worldwide, all the way from Fox Sports for all of their media assets. Think about interactive TVs and gaming consoles and streaming capabilities. The complexity of what you’re trying to achieve now as a media brand, as a retail brand, as a financial brand, it’s much more complex than it ever was in the past.

Andrew: I see. Even with this big vision and this big need, you weren’t able to raise money because you were a first-time entrepreneur. Your background was you worked for a company called Enigma. They’re a data management company, right?

Doron: Correct. So I was actually an engineering–I was VP of Engineering. At some point in time, I think I figured out that I’m not a great engineer. I figured that out when other engineers were complaining that they need to maintain my code. I said, “I’ve got to switch to more the business background.” So I did that in Enigma. But then I met Roy, my cofounder, and I believed in the idea. We spent some time building it out. I think at a certain point in time, it started taking off.

Andrew: Our producer asked you what you learned when you worked at Enigma and you said, “One of the biggest lessons was to not rely on a single vertical.” How did you get that lesson? Tell me about what happened there.

Doron: Yeah. So Enigma was an interesting story. They finally got acquired PTC, but Enigma was focused primarily on the aerospace industry and the airline industry. So mainly customers were airlines and big manufacturers of OEM equipment.

Andrew: Who wanted to buy what kind of data?

Doron: Well, it was basically managing parts catalogues for those big airlines and those big jet engine manufacturers. What happened in 2001 is 9/11 happened. This industry stopped buying anything. They basically came to a standstill. This is where I came to the realization, I guess lesson learned, that I want to build something which is applicable to multiple verticals, meaning more resilient, and it’s also much more international in aspect. And obviously a community that has 200,000 people in 200 countries, that’s very international, right?

Andrew: That’s your community of testers right now?

Doron: Correct. Also if you look at Applause as a company, we sell to software industries. We sell to media companies. We work with eight out of the ten biggest retailers in the world. We work with financial institutions. We work with automotive companies on smart cars initiatives, and we work with companies all the way from startup to significantly large enterprises. So the company is so solid from that perspective that even if an industry has a hard time, we stand that. So we’re good.

Andrew: That’s a really important lesson. I remember actually in my first company being really dependent on email marketers, and as soon as the dotcom bubble burst and email marketers especially suffered, suddenly 90% of my revenue dried up and I had to rebuild my business from scratch, which is really painful when you’re not starting with zero expenses, you’re starting with all the expenses that you had when your business was big.

Doron: Yeah. It’s a challenge building a company that is applicable to multiple industries and multiple verticals. I’ll give you an example. One of the challenges that you need to kind of brand and market to your market that it’s applicable, so it’s more challenging. The good thing is it’s a broad market, it’s very large in size. They sky is this limit. But then you need to really focus on branding correctly in order to appeal to that type of mass market.

Andrew: For each different segment you have to brand yourself differently?

Doron: No. You need to make sure that your brand as a brand can be appealing to all various segments, but then you need to create content that is relevant for those types of different verticals. That’s not an easy thing to do, right? This is something that we are still working–it’s kind of a work in progress.

Andrew: Can you give me an example of how you do it now, how you create content that appeals to two different groups of people without alienating either one?

Doron: Yeah. So, today, we create content for financial institutions, and we focus on the value driven add for financial institutions. We create content for retailers and we create content for media. It doesn’t really contradict with one another, but you do need to speak to them in their own language.

I think the interesting thing today is when you look at all those different types of enterprises and all those startups and SMEs, they’re all going through digital transformation, meaning the form factor by which they interact with customers and users is digital, right? That’s what Applause is all about. That’s why we’re relevant in helping them grow, right?

Andrew: Okay. When you started, even though Roy had an understanding of this business, you said, “I need to validate this. We need to make sure we’re really creating something that’s useful.” Was it you who went out and talked to the VPs of engineering at different companies to see do they really have this problem? Would they buy this solution?

Doron: Yeah. So when Roy kind of talked to me about the idea–again, we need to remember that Roy is a really smart guy, probably smarter than me by a lot, very innovative. When he told me the idea–I came from an engineering background. My focus was enterprise software. He came from a consumer software background. I basically said, “I don’t really know if that’s such a good idea.”

One of the things I did do is go to a few VP of Engineering friends that I had which were more on the consumer side of the business. I actually basically floated the idea by them and basically asked them a very simple question, “If we build this platform that facilitates interaction and this community that works for you, will you come? Are you willing to pay for that?” And 10 out of 10, they said yes.

Andrew: Why do you think they said yes? Was it that the price–was it about the price? What was it?

Doron: It wasn’t about the price. It was they–no one was really 100% satisfied with the quality of product that they’re putting out there. I think they looked at it as a way of getting closer to the users and customers and getting real feedback that will help them become better, beat the competition, iterate faster from an innovation perspective while keeping tabs on what the market is like, right? And this is something that at the time didn’t exist and they felt in their gut, “Yeah, that could help me be better. That can help my company grow and beat any type of competition that I have.”

Andrew: Okay. So they all said yes. Did you do anything to–did you ask them to pay, or did you do anything to make sure you were validating?

Doron: I actually asked them to–we created a side letter by which they would be tied to the beta program. They all signed it. Once we built the platform, it took us about six months to get to kind of an alpha level state by which we can open it up. They all came and became beta customers.

Andrew: I see. And the side letter said, “If we build this, you agree to test it?”

Doron: “I agree to be a beta customer.”

Andrew: A customer, I see. So they were agreeing to pay for it and to use it?

Doron: They were agreeing to pay for it, get it at a discount, be a beta customer and if everything goes well, become an ongoing customer. And you know, the shocking thing is they all converted, meaning after the beta period, which was about a month, they all become customers, different size customers, but they all became customers.

Andrew: I’m actually writing–after doing these interviews for years, I’m actually writing a book. The book is about this whole process of understanding a pain a customer has and validating it to make sure you’re really on the right track. You did it. What did you learn about the product that you didn’t know through this validation process?

Doron: Well, I learned–so the concept and the idea was correct. I think what we helped refine during the evolution of the company is many things around the business model and then the interaction. We learned a lot about community building, community management, community incentives. We learned a lot about what’s the right price to pay on an ongoing basis.

Andrew: To pay the tester.

Doron: To pay the tester and also to charge the customer.

Andrew: So how did you know what to charge the customer?

Doron: I didn’t. I just put a price out there and asked if they were willing to pay for it. They were. Ever since then, in the first three years, we started playing with the business model. I’ll give you an example. When we started, the concept was on demand testing. Come to us, pay $3,000 and I’ll test your product for you.

One day, I think I came to the office probably two years after we started and I said, “You know what? Let’s see how much we can push the business and make it more aggressively. Let’s just tell them $3,000 a month, test as much as you can, no limits.” That’s how we moved to subscription.

Andrew: I see.

Doron: That was back, I think, in 2011. We basically saw that they loved the concept. They had no limitation on how much they could test at the time. They got a huge amount of value and we actually transitioned our business to subscription business in 24 hours.

Andrew: And the developers that you put together, how did you find these developers and how did you get them to accept a piece of this business when venture capitalists who are paid to figure out who’s got the right idea didn’t believe enough?

Doron: Yeah. That was kind of difficult at the beginning. One of the things that we did is when we had this firm in Argentina, which did our first, I would say, duration of development, we actually opened it up and thanks for social network and, you know, the capability to appeal to multiple people at the time via social network. We actually invited testers on various forums to basically come and test the first version of our software. So we actually used dogfooding, meaning testers that we invited to test our technology platform first.

Andrew: To test this community you were building for testers who will then test other people’s product.

Doron: Correct, test our product which facilitates community management and project management for customers.

Andrew: So what did you learn by having these guys test it?

Doron: We actually learned that the idea that we had actually provided a lot of value because we were our first customer.

Andrew: Did they identify any problems with the software that you didn’t know before? Do you remember any of them? Was there anything that was kind of a head slap?

Doron: I don’t remember anything specific. I remember we had to fix it, and we were actually shocked and amazed by how quickly and how insightful the feedback was. I think that just validated the idea that user customer is king. It doesn’t matter what you think internally, meaning what your product team says, what your testing team says, what your engineering said. The only thing that matters is what your customers and users are saying. That’s kind of at the core of what Applause is today. I think we saw that on day one when we opened our platform.

Andrew: So then how did you get this developers to work for you for equity?

Doron: So, at the beginning, we actually started paying them.

Andrew: I see.

Doron: We had a very interesting model, which is a little bit different today, where it’s actually pay for value. So every time they execute a usability feedback or test something on a device or have a bug, everything gets rated and they actually get paid based on the value that they provide, meaning if it’s an exceptional value, they get paid at the highest rate.

Andrew: Wait, the developers did?

Doron: Yes, the developers and testers actually get paid at the high rate.

Andrew: No, the developers that built out your software got paid based on the results and how solid the software was?

Doron: Correct.

Andrew: Oh wow. Okay.

Doron: And the testers from our communities that tested it got paid based on the feedback that they delivered to the developers. So we built a model that completely aligned incentives between the testers and the developers and then eventually between the community and our customers.

Andrew: Okay. All right. I want to find out more about how you got this community, how you grew sales because so far we’re just talking to your friends. I want to know how you moved beyond your friends to bigger companies and god knows you’ve got huge companies working with you now.

But first I’ve got to tell everyone about my sponsor. It’s called ActiveCampaign. Have you heard of these guys, ActiveCampaign, Doron?

Doron: I have no heard of them, no.

Andrew: You’re about to find out about them. Here’s the thing–everyone knows at this point that email marketing is really powerful. You ask someone who comes to your site for their email address so you can follow up with them and stay in touch with them and over time educate them enough so maybe they become your customer and you build trust so they become your customer.

The problem with email marketing is that it’s stupid. Literally, it’s stupid. You get their email address and you send everyone the exact same thing. Over the past few years, we’ve all discovered something called marketing automation, which means that if somebody clicks on an article for your site, for example, Doron, I’m sure you guys do some marketing automation, if someone clicks on an article that’s aimed at media companies, you want to start understanding they care more about media company information than financial services information or the general content that you have.

Based on what they click on, you start changing the messaging to them so it’s customized to them. That’s marketing automation in a very basic sense. You can do so much more. You can, based on what people have selected when they sign up do some automation, based on what they’ve asked for, based on whether they bought or not, based on how many times they’ve clicked. So, if they click on every single thing, maybe it’s time to set them up with a call with one of your sales people.

That’s marketing automation. It has been around for a few years, but it has been insanely difficult. We at Mixergy had to pay $12,000 just to have a team come and help us with our marketing automation software because it’s such a drag. That’s when these guys at ActiveCampaign said, “This is an opportunity for us. If it’s really powerful and it’s really tough, we’re going to find a way to make it easy enough for companies to fully use all the features of marketing automation.

So that’s what they’ve done. It works beautifully well. People at Mixergy audience have used it so much they’ve come back and bought more ads from us. I’ve got a special offer here. For anyone who wants to try marketing automation for their business, marketing automation that’s so good, you’ll actually use it, you’re going to get with ActiveCampaign a free month, your second month is going to be free.

You’re going to get two free one on one sessions. You wouldn’t have to pay the $12,000 that we did to get real support. You’re going to actually have a strategist, a consultant who knows marketing automation help you set it up, two free one on ones, private calls with them. And if you’re already with some other software provider for your email, they will migrate you for free.

All that’s available on this special URL, Even if you don’t sign up, I urge you to go check out all the features to understand how powerful marketing automation is today–

Doron, on the community side, how did you get all these people to come in and test?

Doron: Word of mouth, basically. We built social technology within our community and just friends bring the friends.

Andrew: Did you incentive them to bring their friends?

Doron: No.

Andrew: But they got paid for doing testing, so they wanted to share with their friends how they were making some money, am I right?

Doron: Correct. By the way, making money is not the only thing, right? When you look at community incentives, money is one of it, but also being the best tester on testing this new Google-type of app that was just launched.

Andrew: You were doing leaderboards and things like that?

Doron: Yeah, we are doing today like leaderboards. A lot of credentials are not just monetized types of credentials, right? That’s how you kind of build community over time. I would also say that one of the things we do is we allow people in different types of areas of the world or United States to actually test software for companies that don’t have an office there.

So if I want to test for Amazon and I live in Minneapolis, I can. If I want to test for Fox and I live in Seattle, I can. So this capability of working from home and actually testing on digital devices and being the best tester for a specific team is something that people love.

Andrew: By the way, I’m smiling because as you’re talking about it, I’m going into the Wayback Machine to see what the site looked like in 2007 to understand what you did. You know what I see when I go back to 2007 is it’s kind of interesting–, that’s the old name of the company, uTest, is for sale. The natural traffic to this domain is great. Purchase now for $2,700 firm.

Doron: That’s how much I paid, by the way.

Andrew: It’s interesting. That’s January of 2007. I then moved over to July, 2007 and it says the exact same thing, “Purchase now for $4,800 firm.” It looks like you still got the earlier price.

Doron: Yeah, I got the earlier price.

Andrew: I see. What did you guys pay for

Doron: We launched in 2008. We paid for Applause just around six figures. I think it’s a hell of a deal.

Andrew: Yeah, it is. Why transition, though? uTest is a good name?

Doron: Yeah, uTest is a good name. I think what happened in 2014 is we actually figured out–again, this is by listening to our customers–that the value that we provide to them is far more than just testing. We actually do help them perfect the digital experience to the customers and users. We felt at a certain point in time two things. One, we’re not elevating the importance of the community enough and two, that the name uTest as a corporate name of the company doesn’t do us justice enough.

That’s where we basically said, “You know what? We want to elevate the community. Let’s have uTest be the community brand and let’s find a better name for the corporate brand which better mimics what we’re doing around digital experience for enterprises and startups and SMBs. That’s when we found the name applause, which we loved.

Andrew: Yeah. So I’ve continued hunting, by the way, through the old uTest site. I am now on the tester community. You say, “Here are the advantages of joining–flexible hours, work whenever you want on any product you wish, earn significant income.” Then there are pictures of flip-fops. “Work at your leisure. Then improve your testing skills.” Why would someone want to improve their testing skills?

Doron: Let me give you an example. Let’s say you’re testing a brokerage application in New York City and that’s what you do in your day job, but you want to expand your testing capability because you want to test mobile applications and you want to see how bots are connecting and how’s the interface of human voice like Alexa. How do you test that? You want to understand automation and you want to become a better expert around usability testing. How can you do that? uTest gives you the capability to do that.

Andrew: I see.

Doron: So belonging to that social network, taking courses on uTest University–by the way, it’s all free. That’s what makes you expand your horizons.

Andrew: I didn’t realize that testing was an expertise. I just thought testing means go try this product and come back and tell me if any part of it doesn’t work, but you’re saying it’s an expertise. What makes someone an expert? What makes someone really good as a tester? You’re smiling as I say this because you can see some naiveté in the question, but what does make someone an expert? I had no idea.

Doron: So there are various aspects of testing. There’s functional testing, meaning testing for functionality. There is security and vulnerability testing. I’m a security hacker and I’m looking for things around security of specific applications. I’m a usability expert and I’m helping companies make the application, the website, the app, the device more usable. I’m a load tester, meaning I’m the type of person that helps put load on a system and kind of perfect the performance.

So we as customers and users and consumers, we expect software to work like a light switch, right? You open it, it works. The problem is that software is more complicated than that and it still doesn’t work like a light switch and in order for it to work like a light switch, you actually need to make sure you test all those various aspects and perfect them what the user and the customer expects.

Andrew: So how does an individual test the load of an app?

Doron: So load, there are open source tools that allow you to put simulation of load on a specific app. You monitor the application under load, and then you look for bottlenecks, meaning what are the things that make the application go slow and then you fix them. So it’s kind of an iterative process. You do that all before you launch an application. But I think that point that I’m trying to make is that you need everything to work really well for the app to be successful. Now, having said that, you also need the business side to be successful as well.

But today, I’ll give you an example. Take any financial institution. The difference between a Wells Fargo and a Bank of America is no longer the branch experience. It’s the mobile experience. How frustrating is it if you take a snapshot of your checkout of your check and it doesn’t deposit, right? You need to go into the branch. No one loves that, right? It’s this type of digital experience that is so critical in today’s world.

Andrew: You know what? Even after testing–not testing, even after researching you, I still didn’t understand it until now. I always thought that user testing meant give it to as many users as possible and have them do what they would do with it naturally. You’re saying no, it’s not enough. It’s way more intense than that and way more methodical than that and I had no idea.

Doron: Yeah. I’ll tell you the difference. When you give it to users, you actually get feedback. That’s very important. Don’t misunderstand me. We do usability testing. We do feedback testing. But when you give it to testers, you’re actually trying to find the core issue of the problem so you can fix it. Testers have the capability to dig deeper to specific areas and float the things that–and also, find potential solutions on how you need to do it order to improve it moving forward.

Andrew: I had no idea. I see. For example, before you and I started, our Skype connection was a little bit funky. Our user might tell us it’s a little bit funky. Maybe if they’re a little more advanced, they’d say that it’s slow, but they wouldn’t tell us why and they wouldn’t know it’s because of this issue that was fired off by having Chrome in the background.

Doron: For example.

Andrew: That’s what you want. Okay. I get it. I see the business model. I see where you got your first customers. I’m wondering was the next step then to go and try to raise money and actually get it, or did you continue to grow your customer base through PR?

Doron: No. We actually went and raised money.

Andrew: Once you had your product and you had 10 people who said, “I’m signing a letter. I will be your tester and pay for it,” then you took it and you raised money. Where did you get your first investors?

Doron: So the first investors were actually angel investors. So I basically moved around between people that invested in young entrepreneurs and I found this nice guy in New York. His name was Joe. He said, “Yeah, why not?”

Andrew: It was just you making phone calls and networking and getting through to people.

Doron: More or less. I still remember my cofounder after we got the first term sheets, he basically said, “You know . . .” He said something very smart, Roy, he basically said, “You know, that’s a big milestone because now we’re really entrepreneurs. We’re really founders of a company because there’s really significant essence behind it.”

Andrew: I’m looking at Crunchbase. Since then, you guys have raised a massive amount of money. You’ve raised–is it $115 million, $115.8 million?

Doron: It’s correct. Yes.

Andrew: From Goldman Sachs now, Credit Suisse most recently. Wow. You guys have really grown since then. All right. So now you have clients. You have testers and you have funding. Is PR the next big thing?

Doron: Yeah. I think one other thing about the company which we haven’t done great up until now is create awareness. People understand the pain around making sure that the digital experience that they put out there is great. They still–a lot of people that I talk to still don’t know Applause, right? I think investing in PR and scaling our presence across the U.S. in the Valley, New York, in Boston where content and technology and digital assets are being developed is super critical for us.

Andrew: Why? I’m guessing that’s why you’re doing this interview. But I’m wondering why does that matter to you? You already have big clients. They don’t need more PR to trust you guys, do they?

Doron: I don’t think they need more PR, but you know how it is. Companies are always focused on topline growth. Our mission in life is to own digital experience worldwide. If you want to be the company that owns digital experience worldwide–I’m not talking about hundreds of million dollars in revenue, I’m talking about billion. I’m talking more than that. If you really own a market, people need to know about you. I think this is not something that you can do just at once. This is something that you keep on working on all the time. I think for us, the journey starts now.

Andrew: I guess I always thought that you guys had more brand awareness than maybe you did because I’m in the tech space. TechCrunch, I’m looking it up. TechCrunch covered you guys pretty well over the years, right? The tech software community seems to have covered you guys well.

Doron: TechCrunch actually appeals significantly to technology audience. I’ll tell you something. Today, it’s not just a technology audience that we sell to. We sell to Chief Digital Officer and Chief Innovation Officer and VPs of Digital because a lot of the technologies that are built today that interact with users and customers today is not just owned by the IT manager. It’s owned by the Chief Marketing Officer. It’s especially true to non-software companies. I think that the crazy thing that the world that we live today is that technology software powers everything, right?

Andrew: Yeah.

Doron: From your smart home to your smart car to basically everything that you touch. This is being purchased by CMOs and Chief Digital Officers and that’s the kind of audience that buys from us today, right?

Andrew: Yeah. The example that you gave with the banks is a great one. I don’t pick banks on interest rates because frankly the interest rates aren’t that different. I don’t pick it based on services because the services are the same. It’s not based on people because they don’t want to see their people. It’s based on how well the app works and how well the website works and how fast I can do what I want to do.

All right. Let’s get into 2009. Suddenly you get a phone call from Google. Why does Google call you guys?

Doron: Good question. I think they kind of heard about us, probably TechCrunch, maybe some kind of media. And you know, Google is a very innovative company and I think they like to try new things. They called actually when I was in Argentina at the time sitting with the developers. I did the conversation from Buenos Aires.

They basically said, “We love the idea. We love the concept. We want to try it out.” I said, “Sure. What did you have in mind?” They basically–I think they had a small site around books, Google Books, and they basically said, “Why don’t you try and test that. Tell us what you think. We need to cover it in multiple countries.” So we did. I think the results were so phenomenal that we immediately started expanding the relationship with Google at the time.

Andrew: And are they still an ongoing customer of yours?

Doron: Google is still an ongoing customer of ours. It’s one of our largest customers. We test for Google today 14 different product lines on an ongoing basis.

Andrew: Wow. Your focus at first was just startups. You and Roy said, “You know what? We can’t cover all these different . . .” Google is an enterprise customer at this point, but you can’t get everybody. Why did you decide to focus on startups? I feel like startups don’t have that much money.

Doron: So, when you focus on startups, it’s not about the money. It’s about refining your solution and it’s about understanding your ecosystem. What startups are doing today around technology and innovation, enterprises are going to do tomorrow. Startups are the ones that started with IoT and with mobile and with everything around it. I think the great thing about Google–some of those other big customers, take almost anybody, Facebook, Amazon, whatever, they almost behave like a startup but at a much higher scale.

So I think the evolution from startup to Google was relatively easy because it’s all about making sure the development innovation is there and making sure that the quality is there. The next transition after that actually happened to non-software companies, which is a different type of transition.

Andrew: All right. Let’s get into how you found them, but first I’ve got to tell everyone about a company called Toptal, top as in top of the mountain, tal as in talent. They’re my sponsor. Doron, the problem that these guys solve is that it is really hard to find great developers. I think you guys started back in Boston, am I right?

Doron: Yes.

Andrew: So, now you’re here in Silicon Valley and it feels like Silicon Valley is this small little world, Silicon Valley, a little valley, everyone lives together. But the truth is that’s not how it works. Most people live in San Francisco because they want to experience the city–maybe not most, but a lot of people do–and then they have to commute all the way down to Google, all the way down to Facebook, all the way down about an hour and a half to work at these great companies.

And they’re super smart people, which is why these great companies recruit them, but they don’t want to be stuck in traffic for an hour and a half and they don’t want to live in the suburbs around Google in their Mountain View office. So, a lot of great developers said, “I want to work around the world wherever I came from, the original country I’m from, the original city I grew up in. Forget this whole living in Silicon Valley and trying to have this lifestyle that we’re supposed to enjoy.”

So Toptal recognized that there are these brilliant people all over the world who are untapped and they put together a network of these great developers who want to work where they want to work and live the lives they enjoy. They said, “You know what? We have this great developer. Anyone who wants to hire them who doesn’t need these developers sitting in their office can come to Toptal. We will understand the company’s problems and then we’ll match them with the perfect developer for them.”

We at Mixergy did it. The first developer they introduced us to is okay. We didn’t end up going with them. The second developer was fan-freaking-tastic. We hired them and got started within a couple of days. That’s the way, if you’re listening to me, and you want to work with Toptal, that’s the way it will work for you. First step, you’re going to get on a call with them, second step, they’re going to find you a great developer that you can hire. Many people hire part-time, some do full-time. Some do even full on teams from Toptal.

If you want to work with Toptal, you should know they’re created by two Mixergy fans. So, they’re offering Mixergy listeners something they’re not giving anyone else, 80 hours of Toptal developer credit when you pay for your first 80 hours and that’s in addition to a no risk trial period of up to two weeks. Even Google doesn’t get a no risk trial period of up to two weeks when they hire people. They commit. Here, you get to see you get the best of the best and see for yourself at your company. Go to the special URL to get that offer. It’s

Doron, I’m kind of whipping through these ads now because I’ve gotten a lot of feedback from people saying I was going too slow. Someone said, “Andrew, I had to hit the fast forward button five times on you.” So now I’m whipping through the ads. I take that user feedback very seriously. You guys can’t take my podcast and play it for people all over the world and get feedback from where I’m going with this–no, I think I should go to my audience, shouldn’t I?

Doron: I think we can. Not only that, I think you can also pick and select various types of demographics and get feedback based on different demographics.

Andrew: Really? I can say, “You know what? We’re not appealing enough to women who are 20 to 25 years old. Can you get them to listen and give us feedback on the podcast?” You can do that?

Doron: I can do that.

Andrew: How much would that cost?

Doron: I can actually do that in a very quick turnaround. So if you tell me go now, I can probably have something started tomorrow.

Andrew: What does that cost?

Doron: So doing a usability kind of survey like that, I’m assuming you’re having a usability expert going through the main things you want feedback on and then having a panel of 100 people which are digital savvy with different demographics in different parts of the world is something like $10,000.

Andrew: $10,000. And it’s a one-time thing that’s not a subscription?

Doron: One-time thing. It’s fully managed. All you get is the value, very concise plus access to the raw data.

Andrew: I see. I’m guessing that what would happen is companies who might have this exact problem I just mentioned, “We’re not appealing enough to this demographic. Can you have them check out our product and tell us why or tell us what they think?” Is that right? Is that a common request?

Doron: It’s a very common request. There’s another common request. “Why don’t you compare it to this type of application and tell me where I’m better and where I’m not good.” You know how it is when I’m a company and I’m developing this kind of software and I think, “I’m better at that and I’m better at that.” But my competition is kind of kicking my ass.

So what do I do? Why? What am I missing here? This is my demographic. This is my target market. Can you help a specific panel evaluate me against my competition? Tell me where I’m better, tell me where I’m not so good. The whole idea is listen to your market, listen to your users, listen to your customers. That’s what makes you better.

Andrew: All right. I see how you got your first customers. How did you get the next batch of customer?

Doron: So one of the things which was pretty interesting is I actually hired a sales guy.

Andrew: Your first hire after you and Roy was a sales guy?

Doron: I had the development team. But beyond the development team, yes, it was a sales guy. So it’s a sales guy that I knew from prior life. The reason I hired a sales guy is not so much in order to sell, it’s more give me feedback from my customers, right? Tell me what my customers are saying and what else do they need. He actually started selling. It wasn’t that hard. Like me and you talked, pennies everywhere.

I’m going to launch in a specific country, I want to make sure that I do it right, right? I’m launching a new website. I want to make sure that usability works as intended, right? My ratings in the app store just stand with brand new release. Oh my god, what happened? Can you help me figure it out?

Andrew: So you asked him to first go to existing clients and understand how you guys were doing with them and then he went out and got new clients. How did he get the first new batch of clients? What did he do?

Doron: Just awareness, meaning we had a lot of very interesting content around crowdsource testing, which was what we called it initially. That was a brand new concept at the time.

Andrew: I saw that phrase used a lot. This was basically when the term crowdsourcing was just starting to make sense to people.

Doron: Yeah. That’s correct.

Andrew: So, content, was he writing the content at the beginning?

Doron: No. It was basically me and Roy and then we started hiring a marketing person and he started writing content and then we started having inbound leads based on the good content that we wrote around crowdsourcing. And crowdsourcing was a fascinating idea at the time. Now everybody knows what is communities and crowdsourcing and liquid workforce, everybody uses Airbnb, the digital economy. Everybody uses Uber, Lyft, whatever. At the time, it was a brand new concept. People were fascinated by it.

Andrew: I remember Paige Craig, the angel investor who famously got turned down by Airbnb when he wanted to invest in them because they didn’t know who he was and he’s also raised his profile really well for good reason, I told him I was getting married and he said, “Andrew, you should crowdsource your wedding.” That’s how excited he was about crowdsourcing. I said, “Paige, I’m not going to crowdsource my wedding. This is like you telling me I should drink the fifth beer at your party.” He goes, “No, crowdsource it. Here’s what it is.” I get it. There was a huge enthusiasm and you guys definitely leapt on it.

Here’s the other thing you did really well. I’m just going back and trying to understand it. Tell me if I’m right. You guys used to uncover bugs in apps that you had no business looking for bugs in, then you wrote reports about them. So you would do like, “TweetDeck has 77 bugs. 13% classified as showstoppers, 30% were technical.” So now you’re getting all these apps that people know about and you’re uncovering problems with them.

Doron: Correct. We’re talking early days here. We want to create some awareness to crowdsourcing and the scale and the value that it could provide. We ran bag bashes. So every quarter we’d use a subject–quarter one, it’s search, let’s do Yahoo search versus Bing versus Google search. Quarter number two, let’s take the three biggest retailers in the world. Let’s see how Target maps against Walmart against Amazon, right?

So we ran that, I think, for about four or five quarters in a row. It got quite a lot of press. One other thing that we did is the issues that we found, we actually gave to the companies.

Andrew: So TweetDeck would have gotten all their bugs even though they didn’t pay for it.

Doron: Yeah. It was all free.

Andrew: And everyone else got to see it. You published the results publicly. Then I can see here–was Matt Johnson that VP of Marketing?

Doron: Yeah. Matt Johnson was VP of Marketing. We published the high level results, but the actual bugs, meaning the issues themselves, we did not publicly put them. We actually gave them to the company.

Andrew: So one of the ways that you got new customers was publishing these kinds of posts, then people would see it and say, “I’d like to find out about my app and then they’d contact you guys and start working with you.”

Doron: That’s correct.

Andrew: Did you do any outbound work?

Doron: Very little outbound work at that time. Everything was primarily inbound and some PR awareness. We were one of the leaders in crowdsourcing at the time as well.

Andrew: What else? So these posts worked. Did you do anything else to bring in new customers? Did you go to events and speak? Did you do anything like that?

Doron: I did go to events. Primarily at the beginning it was testing events and speaking. We created more awareness. I think one other thing we also needed to remember was we had this uTest community. Those were testers that were actually working as part of uTest. Some of them were freelancers in the space, but the large part of them also had a day job, exactly like I’m an Uber driver today, but I may have a job somewhere else.

I think the critical point is that we actually build an ecosystem because those people which are part of our community which tested software for our customers also had a day job at some kind of company testing software. We actually went to their boss, their VP of Engineering, their Director of QA and told them, “Hey, we can scale our quality by using a model like uTest or a model like Applause.”

Andrew: I see.

Doron: So this ecosystem of testers, community members and then customers is something that drove a lot of our growth at the time.

Andrew: I’m seeing posts you guys did. Whenever you’d get new clients, you’d do a post about that too. “uTest adds four customers to its rapidly growing roster,” and the four customers were SweetIM, ON24, Synthesite and RepairPal.

Doron: Yeah. Those were early days.

Andrew: Yeah. They were. How do you know what else to expand beyond? You’ve talked about how the name uTest had to be transitioned to the community name and you created Applause as the company name and you said it was based on knowing you needed to expand your offerings and the name uTest was not descriptive enough of your new offerings. How do you talk to your customers to understand what else you should be creating, what else you should be doing for them?

Doron: So two things–we run basically an NPS survey with both of our stakeholders, by the way. We run an NPS and customer satisfaction survey every quarter where we get a bunch of feedback from our customers and then we do the same with our community. Once you have a community of a quarter of a million people or more and they’re engaged with you on an ongoing basis, they have good ideas.

What you need to do is be able to captivate those ideas and decide which one you want to go for and which one you don’t. Again, it all goes back to listen to the various stakeholders, whether it is community members or customers and kind of figure out where you want to go.

Andrew: How do you know what to listen to and what not to do?

Doron: I don’t. This is where kind of I would say CEO gut feel.

Andrew: But there’s no framework internally for figuring out what you should be doing and what you should pause on.

Doron: Exactly.

Andrew: I see.

Doron: And it’s a leap of faith, right? You invested six years in building a brand like uTest, which is actually going well, and yet you actually decide to rebrand the company and go to that bigger vision. That’s what you do, I think, as a CEO and entrepreneur. You always need to try and look forward to the expansion and how you expand your company.

Andrew: Was there a feature that people ask for that you created that you then realized, “We made a mistake by adding it? We shouldn’t have listened?”

Doron: There was.

Andrew: What was it?

Doron: Yeah. I’ll give you two examples of where we kind of missed. When we launched for the first time, we actually didn’t have PayPal as a payment mechanism for testers, meaning to pay them for bugs. We had other types of payment mechanisms. We had to develop that feature within a day. You know why? Because they were all screaming we need to get PayPal.

Andrew: I see.

Doron: That’s kind of one example.

Andrew: That’s a good one. They’re screaming. They need something and you understand why they need it and you go and create it.

Doron: Yeah.

Andrew: Was there something they were screaming for that you realized, “Hey, we shouldn’t have listened?” They actually meant something else?

Doron: No. That was really screaming because they wanted to get paid.

Andrew: I’ll tell you the example. Paul Buchheit, the creator of Gmail, told me once people wanted a way when they see their list of emails to just like mouse over it and see the message. So he said, “Why would I go and build that? Let me understand why they’re asking for it.” By asking them why they asked for it, he realized whenever they click one of the messages, it takes too long to load, so they want a faster way to see whether they should click into the messages and what he should really be working on is a faster way to load messages when people click on them.

That’s what he built instead. It was that understanding why people asked for what they asked for instead of building what they asked for. Do you have any experience like that, where someone asks for something and you shouldn’t have built it?

Doron: Let me give you a really good example. So, one of the things that we built in the early days was I think we got a lot of feedback, “We want a self-serve experience,” meaning all the way from putting your credit card, building my testing team and running my test. We actually built this whole product. I think it was called uTest Express or something like that, probably in 2010. If you do searches, you’ll find it. It was kind of interesting.

The media loved it. It was the Holy Grail, right? I can put $500 on my credit card, run the test, get immediate feedback and within 24 hours I’m good. You know what the problem was? The market wasn’t ready for it. The market wanted to–market meaning the customers. They wanted to talk someone. They wanted to understand how to build their team. They wanted to make sure they don’t spend a whole bunch of time getting the feedback. They wanted the data to come back to them much more matured.

The product really failed. Even though it was a great product, the customer didn’t come. They went into the website. They put their credit card in, but then they called for the emails.

Andrew: Interesting.

Doron: So the whole self-serve experience failed.

Andrew: I’m seeing it, actually. I see a bunch of articles about it. Here’s one–$500 per test cycle. That’s not much. And they still wanted to talk to a human being. I get that.

Doron: I think some of it was timing, right? The market wasn’t ready for it. The market wanted it, but when it came to crunch time, it wasn’t ready for it.

Andrew: How did you make the leap from being someone who hits $1 million in revenue to then becoming a $10 million company? What did you have to do organizationally to allow the business to grow to that level?

Doron: I think one of the things that I’ve learned in my old age as an entrepreneur is the way you need to think about your business is it’s primarily a relay race. It’s not the same people you started the business with that are the same people you’re going to have when you’re $100 million in revenue.

I think one of the things I always did was focus on the people and the employees because those are the people that actually bring a lot of the ideas and build the company and the second thing is focus on the management team, right? Bring in the right people for the right stage of the company, right? I think that’s one of the things that I did right, building the management team, focusing on the people. It’s the people that really build the value for the business.

Andrew: That means at some point you have to say to someone you took the baton as far as you can go. I need to pass it to someone else. And you really will have a sit down with someone and say, “You’re good, but you’re not ready for the next stage. We have to let you go.”

Doron: Remember my first sales guy. He made it to $1 million, but I needed someone else to take me to the $10 million.

Andrew: I see. As a first-time entrepreneur?

Doron: That’s kind of sad, right? He did everything right, but he’s an evangelistic sales guy. He can sell something evangelistically. He can’t build a team, hire people, build a process. You kind of need to, even though you love the guy, like he’s great. He’s not the right guy to go with you to the next $10 million, right?

Andrew: Was being in the army what helped you do that, having that background?

Doron: I think being in the army helps you kind of make tough decisions from time to time. I think the Army helped mature me to making decisions. The types of decisions you make in the army are much more important than, “I need to replace my VP of Sales.” I think you take the type of decision much deeper.

So I think the decision making process, I think the army helps that. By the way, not every decision that you make is the right decision. I think I once read an article that if 20% of the decisions are wrong and 80% are right, you’re good as a CEO/entrepreneur. So I’m trying to live by the 80-20 rule.

Andrew: I think it was my friend Jason Calacanis who said you could be wrong every time, but if you make one right decision, then you’re golden. You really do make a whole lot of bad decisions, which is hard to look at somebody in the face and say, “It’s not you.” I remember specifically saying to someone, “I wish you would have robbed us. Then it would have been easier for me to do this. It’s not you at all. You didn’t do anything wrong. This is where the company is right now. We have to transition away.”
Let me ask you this to close it out. When I looked you up, first of all, I saw that you were on the board of Runkeeper, I freaking love that app. I don’t use it that much anymore because, for some reason, it logged me out of–two weeks ago, I was on a long run with it. I think I hit 21 miles and it logged me out. It lost the whole thing. It was so frustrating. So, now for the last couple of weeks, I’ve just been upset at that app specifically. I’ve been using it for years. I looked you up. You’re a long distance runner. You’re a guy who’s run Boston nine times now?

Doron: I actually did it 10 times, yeah.

Andrew: 10 times, Boston Marathon. You’re running a company, a $100 million business. How do you have time to run, and how do you justify it when you’ve got to have things that call you at work?

Doron: Yeah. I think having the right work life balance–for me, work life balance means family, work, business and then making sure that I have some off time and for me, off time is actually exercising, whether it is biking or triathlon or running.

Andrew: But when someone needs you and you have to get your run in for the day, will you say, “I know you need me. This is very important. I have to go run.” When you say that–I know when I say that to people, they think I don’t take the business seriously enough.

Doron: No. I think when you build a company as a CEO and a leader, you kind of need to appreciate the work-life balance.

Andrew: And you’ve got to just keep telling people, “I love this company. I’m here every day. But I need to go for a run.”

Doron: Yeah. “I’ll call you in an hour.” When I come back after my hour run, I’m a better focused human being and I can go for the long run for the day around work. I think what people need to remember is that building a company is not a sprint. It’s a marathon, right? The only way to look at a marathon, you can’t look at mile 26. You kind of need to take it a mile at a time. That’s how you build a company. That’s how you build a long lasting business for the generation. So, if you’re in it for the long run, think about it like a marathon.

Andrew: So then where are you going to be in five years with the business, do you know?

Doron: It’s going to be bigger than it is today? There are various rumors, IPO, no IPO. Obviously we’re reaching the right critical milestone from everything we need to, meaning customers, revenue, everything associated with that. I think the market condition is good. So maybe we’re going to be a public company. Maybe we’re not. I think there’s a good probability that we are, five years from now.

But I think what’s really important for me is to be that dominant company in the digital experience economy. That’s where I want to be. I want to be that type of company when someone says, “I want to create this killer digital experience.” It doesn’t matter what it is. It’s smart home or smart car or mobile application or whatever form factors they have five years from now, artificial intelligence or whatever, I’m going to think, “I want to kill it. I’m going to use Applause for it.”

Andrew: I see. I need a design team. I need engineers and I need Applause.

Doron: Correct.

Andrew: I see.

Doron: I need to make sure the experience I create is the best out there. I’m going to use Applause to make sure it is.

Andrew: All right. What’s your fastest marathon?

Doron: I ran when I was young less than three hours. My fastest time in Boston is 03:02:00, never went below three hours in Boston.

Andrew: Boston, for people who don’t know, is the one that you actually have to qualify for by running super fast. I am not anywhere near that. I love long distance running. I’m fairly slow. My fastest is over four hours. But I love it. All right. Thank you for being on here. The website for everyone is What an easy domain.

The two sponsors for this interview are the company that will help you hire your next great developers. It’s called Toptal. And the company that will do your marketing automation so easily you’ll actually get to use all the features. Go check out the features just to understand what marketing automation could be for your business at Doron, thank you so much.

Doron: Thank you, Andrew. I had a great time.

Andrew: You bet. Me too. Bye, everyone.

Who should we feature on Mixergy? Let us know who you think would make a great interviewee.