Founder of Google Docs takes on a big problem

You may not know Steve Newman, but you probably regularly use the software he created. In 2005 he launched Writely, a pioneer online word processor which sold to Google and became the basis for Google Docs.

At Google he noticed a massive problem. Catch this interview to see what it was and how it led him to found Scalyr, which does log analysis to help DevOps teams solve problems faster.

Steve Newman

Steve Newman

Scalyr

Steve Newman is the founder of Scalyr which is a blazing fast log management platform for the DevOps front line.

roll-angle

Full Interview Transcript

Andrew: Hey there, freedom fighters. My name is Andrew Warner. I’m the founder of Mixergy, where I interview entrepreneurs about how they built their businesses. I am so excited to have today’s guest on, for personal reasons. We’re here to talk about his new company Scalyr. I, in preparing for this interview, saw that he is the founder of a company called Writely and very few people will remember, but every one of us is using right now.

I remember when Writely came out, they said, hey, this whole thing of like Wikis and word processing, we can do it online in a browser. And I remember when he first created it, it not only worked, it was beautiful, a thing of art. I remember . . . I’m getting worked up here. I remember at the time talking to Jimmy Wales, and the thing that annoyed him, I think, was when I said to him, did you intentionally make the editor of the Wiki ugly to prevent stupid people from editing?

And you can see why he’d be upset at that because he didn’t mean for that. It was just hard to do it. And these guys did it and then they were bought out by Google, and then it became Google Docs and I am looking at a Google Doc right now. All of us are using Google Docs and they are pioneers in what they did.

Now as he, Steve Newman was working at Google, he started to see a problem and he said, you know, well he actually didn’t say, you know, I’m going to solve it right away. It took them a little while but eventually said, you know, I think I could do something about it. And then he got together with his previous team and they did something about it. The solution is, it’s called Scalyr. That’s what we’re here to talk about, though I’ve got to ask him about his previous company, revolutionized software online.

Anyway, we’re here to talk about Scalyr, which is a blazing fast log management platform for dev ops frontline. And this interview is sponsored by two phenomenal companies. The first will help you get great design. It’s called DesignCrowd. I’ll tell you how, why and maybe even how one of our listeners use them. And the second is a company of both Steven and I have used, it’s called Toptal. If you’re hiring developers. I’ll tell you why you should use them. But first, Steve, welcome.

Steve: Thank you. Glad to be here.

Andrew: Are you a little embarrassed by my enthusiasm? Because I was watching you as I told you and I sensed maybe a little awkwardness. No?

Steve: I don’t know. I think I have a strong sort of anti-presumptuous reflex and sometimes it kicks in.

Andrew: I get it. Okay. I said, maybe something bad went down. Why is he not excited that I’m doing . . . All right. Good. I’m glad because it is being used by a lot of people, including a couple of people close to you.

Steve: Yeah. I was mentioning earlier. Both of my kids who are both in high school use it literally every day and have been for years now and not because their dad built it. We all know that you know teenagers run away from things that involve their parents.

Andrew: Right. They can’t run away from it. What’d you sell it for? How much?

Steve: So that we’ve never disclosed.

Andrew: Can you give me a ballpark? Was it life changing?

Steve: I mean, I’d rather not get too specific, but it was. Yeah, I mean it was definitely, it bought us a house and, you know, that’s no small thing in this region.

Andrew: San Francisco is in the Bay Area, which is where you are. You ended up working at Google for six years. That led to Scalyr. Can we talk about the revenue from Scalyr? I’m looking at it here on my screen, but I won’t divulge it because you told us in private. Can you say publicly? I think it’s something to boast about.

Steve: We don’t get too specific, but I’ll say we’re in the and I do want to make sure I don’t make an off by one error in my head now. We’re in the low eight figures,

Andrew: Low eight figures. Eight figures is 10 million and above annually recurring revenues.

Steve: Exactly, exactly.

Andrew: Can you give me a use case, an example of how somebody would be using Scalyr and then we’ll get into the story of how you built it up and got it so fast to this level.

Steve: So the product like this comes up in a lot of places, but really the core use case is troubleshooting, firefighting. So there’s a problem. You know, hopefully, we’ve alerted you to the problem, but it might be that you’ve stumbled onto it, but one way or another you’re aware that there’s an issue with your service, an issue with your website. Something’s down, something’s broken, something’s running slowly, something’s wrong.

And at first what, you know, the symptom, right? This page won’t load or this thing is slow, and you have to get to a root cause so you can do something about it. And that may be a long chain. You know, the site is slow because the application server is backed up because the database server is bogged down because some other system is overloading the database or maybe six steps from the thing you can see to the thing you’re going to fix. And that’s where we come in. You know, people can go in and dig through their log data. It is usually the most detailed source of information about what’s actually happening under the hood in all these systems and figure out what’s wrong.

Andrew: I mentioned earlier that you came up with this idea largely because of your experience working at Google. What happened at Google that made you say this is a problem and eventually led you to solve it?

Steve: So I was working and I should really say we, because there were a number of us who are working together at Google who are here at Scalyr now. We were working on a fairly complicated database infrastructure project and had a lot of moving parts. We were building this layer that supported a lot of applications and, in turn, we were supported by a bunch of lower level services for data storage and user authentication on all kinds of parts in the system.

And with all these moving parts, there were a lot of places where problems could come up. And when a problem would come up, it was a lot of work to track down. You know, I was talking before about how what you see and where it came from may be six steps removed. And the tools we had for pulling data out of the system and analyzing the data we’re hard to work with and very fragmented. We actually, we realized that we started making an inventory and we realized we were using 17 different tools just for visibility, just to dig into what’s happening in the applications.

And it was just becoming an enormous time sink for us as we weren’t the operations team. We were the development team, but as happens a lot, I think nowadays the development team is, you know, has responsibility for making things work. And we were spending more than a third of our time as a development team just in these investigations. And so that was sort of the, you know, the original pain point.

Andrew: Was this on Google Docs, Google Drive, the G suite package? It was.

Steve: So the actual project, there were Google had at this point, all of these different applications that sort of had the same structure. They worked with basically files and folders, but they were all implemented separately. Google Docs, Google Sheets, Google Slides, Google Photos, Google Drive, all separate backends. And this was a project to unify that. So basically a single internal file system.

Andrew: And you spent that much of your time trying to hunt down bugs, hunt down issues and Google has tons of software and engineers to help you do that, but it was too much all over the place. You then eventually left Google, and I mentioned it didn’t occur to you while you’re at Google to come up with this idea. Actually did it or are you . . . or was it afterwards?

Steve: I think the seed was planted, but kind of the details of the idea came after we left.

Andrew: [inaudible 00:07:36]

Steve: But I really trace it back, it was an actually a specific meeting at Google I trace it back to. Not to mean we didn’t have the idea, but we sort of confronted the problem head-on. Wow, this is ridiculous. Like 17 tools and no one knows how to use all of them and yeah. So then I trace it back to there, but that was the question, not the answer.

Andrew: It wasn’t until you left Google. Why’d you leave Google?

Steve: The short answer is I had actually four great years there, and I had gotten to the point where I wasn’t learning that much new. Google was initially, especially was a huge education. And you know, I’m so glad to have been there. It was a very valuable step in my career, but sort of gotten to the point where I had learned what I felt I was going to learn. And I’m a startup guy. Scalyr is about my eighth startup, and I set a record at Google. I’ve never been longer anywhere else, but it was time to move on.

Andrew: What was your company name? It was like the official company name that most people didn’t even know unless they looked at the About. It was something like Upstartle or was that it?

Steve: That’s right. Well, actually . . .

Andrew: This was before Writely?

Steve: Yeah, yeah. Writely was Upstartle. And so we actually, we had the name before we had a product or even an idea. Upstartle was two friends and myself, one of whom I’ve known since college and we’ve done about seven of those eight startups together. And then another great engineer we picked up along the way. And so we just knew we wanted to go do something. We had no idea what. And so we’re like a startup, but we’re upstarts. I think the story is basically we want an upstart.com, but it was taken. So then we went with Upstartle.

Andrew: I like that name a lot. And so how’d you come up with the idea for Writely?

Steve: So it was a very definite moment where that happened. And it was a college friend of mine, Sam Schillace, who’s, by the way, back at Google now for the third time. There’s a whole story there. But he just walked into the office one day. We were working on some other idea we’d had. Frankly, it wasn’t very good. And a few things had come together in his head, four elements of which I can only remember two. One was Gmail, so everyone thinks about Gmail. Oh yeah, that was a big deal. One gigabyte free.

But another thing that was kind of unique about Gmail was rich text editing in the browser. It was simple, you know, bold and italic or whatever, but you could, it was more than just plain text and you edit it in the browser. And I don’t know whether Gmail was the first mainstream app to do that, but it was the first place I had seen it and I think any of us had seen it.

And so I remember Sam, like a few things clicked together in his head and, “Wait, if they can do that, it must be possible for us to do that. I don’t know how they do it, but what we’ll go figure it out.” And Sam and I actually for years had wanted to solve what we called the emailing a Word file problem. You know, you’re collaborating with someone on a document. Inevitably that means emailing the file back and forth and, “Wait, who has the latest copy?” and, “Oh, we both edited it,” and we’d swung and missed at that problem a few times over the years as desktop applications that you’d have to install. And this was at often, pre-internet and it’s harder to make it work, but he sort of clicked, “Wait, you can do this as a webpage.”

Andrew: And so you saw that Google could do this with Gmail. You said this is possible on the web. You remembered the problem that you’ve been trying to solve all this time and you said let’s connect it. Did you have conversations with customers to see if they were willing to put their stuff online? I think privacy was a big reason in your blog post about why you sold to Google. I think privacy is something that you brought up a lot. I’m wondering if at the time you did any research to see if people felt safe writing documents online. If they wanted it or was it just, “I see the technology. I know this is a big enough problem. I’m going to go.”

Steve: We went through an extensive process of, “Hey, that sounds really cool. Yeah, let’s do it.”

Andrew: That was it?

Steve: That was pretty much it.

Andrew: And then how did you know you were on the right track?

Steve: So I remember so we did a very lean startup, you know, quick prototype and viable process. I remember I just read some blog post someone had written that basically said any new idea, any new product you should launch in 90 days. If you spend more than 90 days, you’re overthinking it. And we decided that we would go for that. I remember we missed slightly. So I think it took us about 100 days from, “Hey, here’s an idea,” to we did a very soft launch. We just, you know, if a launch falls in the forest and no one is there to hear it, we just sort of put the page up and told two friends. But we thought of this as an early beta, not ready for prime time. And but we started spending literally like $3 a day on AdWords. We’re trying to get your stuff, few people to come look at it and tell us why it stinks.

And that went on and then on and then our analytics were, I had a script I could run against the database that would tell us how many registered users we had. That’s the analytics. And I went to bed one night and the number was about 100 and I woke up in the morning and it was a thousand and climbing fast. And what turned out had happened in between was TechCrunch had blogged about us.

Andrew: Michael Arrington, right? And he loved it.

Steve: Yep. And this was early days. Yeah. I mean TechCrunch was Michael Arrington period.

Andrew: And at the time I read this in preparation for talking to you. At the time he was basically looking at a new web 2.0 type software and talking about it, just the fact that it existed and there were minor reviews, but he clearly thought you guys were onto something and he saw the big picture — a replacement for Word.

Steve: Yeah. And he did us a huge service. And so, you know, I think what you had asked was, you know, how did we know we were on the right track, that was how we knew.

Andrew: Because both he said it was good and he was smart about this stuff and people who went to sign up.

Steve: Yeah.

Andrew: I thought of you guys at the time is like a Wiki. And he did too in that post I think mentioned that you guys were like, he said, it’s like what you see is what you get, WYSIWYG. Actually, it’s like Wiki wig or something, he called it. Did you guys think of yourself as a Wiki? Like Wikipedia or a replacement for Microsoft Word?

Steve: So what we thought of ourselves as a collaboration tool. You know, because it was the original idea of, “I’m tired of emailing the Word file back and forth.” Now, and bear in mind, you know, this was 2005. Browsers were much more limited in their technical capability than they are today. It was, you know, we thought, well, if Gmail can do it, like this will be easy. We’ll do it.

But so Gmail, you know, think about how long a typical text message or a typical email message is. Think about the level of complexity of formatting that people expect in an email message and there’s only one person working on it. And you sort of open it up, you type a little bit and you’re done. This was people were writing, I think maybe literally Ph.D. thesis in it, you know, you would collaborate back and forth. It turned out that Internet Explorer and Netscape and all the different browsers handled formatting very differently and we’re trying to synchronize formatted documents between these very different browsers and all that. So, you know, we got a little carried away with if Gmail can do it, then no problem.

And so, but, you know, we didn’t even get just to do the basic bold and italic and fonts, to actually make it work turned out to be a huge challenge. So we never approached and never intended to approach anything like what the full feature set of Microsoft Word. And so we didn’t really think of it as a Word placement. It’s not a rich, you know, beautiful document creation tool, and it wasn’t really a Wiki, although people use it for that. We thought of it as a collaboration tool.

And what people actually wound up using it for . . . So despite all that, people looked at and they immediately said, “Oh, so that’s like Word, but it’s on the web. Word on the web, great.” We’re like, no, no, it’s not Word. Word is huge. Collaboration and turned out what a lot of the early users really wanted was it’s just my documents wherever I’m at. I’m on the lab machine at the university, I’m on my parents or wherever I am, my stuff is there. That was a lot of the early usage.

Andrew: There was a post by a blogger that you guys referred to a lot even. I think Michael Arrington linked to it. I went back and I read it. It’s down, but I found it in the Internet archives, and I’m so glad that I did because there’s a picture of what that early editor looked like. And I could see now why I thought it was closer to a Wiki, because you had these tabs at the top with like a history and so you guys had history versioning in there in the beginning and you had authors and you even had a tab for viewers. So I get it. You guys knew because of that . . . Yeah. What were you going to say?

Steve: No, I’m just kind of, I’d forgotten some of this. So I’m then remembering. Yeah.

Andrew: The only thing that looked a little bit off with it was the logo at first was just very basic. It was like, it had clip art in it.

Steve: Yeah, I do. I mean, the whole thing, like I said, you know, we launched it as a prototype almost, and for a few weeks, you know, we had like three users and then TechCrunch and boom. And you know, from my friend Sam saying, “Hey, I’ve got an idea,” to closing the acquisition, not signing but closing was 10 months. So that just gives you an idea of how compressed the entire process was in.

And so our other partner, Claudia, may kill me for saying this, but I’m going to share something I don’t think I’ve ever shared publicly before. When I called her to tell her TechCrunch 1000 users and climbing, the first thing she said was we have to make them stop because it wasn’t ready. It didn’t look right. She, you know, she’s the UI genius and like, and the design wasn’t great and, or at least wasn’t where she wanted it to be and we weren’t ready for prime time and here it is 11:30 on Saturday night

Andrew: And then everyone knows it. And I know Claudia because I went back and I looked at earlier blog posts. Thankfully you guys did a blog for some period on Blogspot and that stays up forever. And so I could see Claudia was the one who was writing up on there. Why did you guys sell to Google? You were onto something. The world loved you. You could have become an office suite on your own. Why sell?

Steve: So frankly, a big part of it was, so we did this blog post around the acquisition about, we realizing our dream together with Google and they’re the perfect partner to carry this. And you usually see something like that, an acquisition and you know, one presumes that often there’s a certain amount of spin there, but in this case, there was a lot of truth. We really thought Google was going to be a great platform to . . . And I think in retrospect, that kind of worked out. We knew that Google was going to be a great place to really build this and take it broad. It was going to keep the pressure for a business model off, which would really allow us to grow it. And frankly, you know, going and working at Google for a while sounded like it would be fun and interesting and it was.

Andrew: I’m surprised you think that. As an entrepreneur, I didn’t do you justice in the intro with like your entrepreneurial record, and I feel like your LinkedIn doesn’t do it justice. You talked about the companies that acquired you and how you had a job with them. You never, as far as I remember, even in your LinkedIn, you don’t say I started this company acquired by Google and then the next entry would be Google. It’s I worked at Google for six years in there. Now I understand why, because you’re including even your Writely years. So you did start a few different companies. Why’d you want to go be an employee somewhere? It feels like so many entrepreneurs don’t ever want to be employees.

Steve: One of the great things that Google did for me was, and this actually segues into Scalyr, it was an incredible education in engineering at scale. As most of my career prior to Writely actually was almost all my career was desktop software, and Writely was the first serious piece of server work or browser-based work that any of us had done. And, you know, and then we got to go to Google and actually learn how to do servers and scale and distributed software and applicant and all of that. And I mean, I remember I spent my first month at Google just clicking through the Wiki and reading design document after design document after design document.

Andrew: The internal Wiki.

Steve: The internal Wiki. The Google Internal Wiki.

Andrew: Okay. Let me take a moment, talk about my first sponsor and then we’ll get into now that we understand your background and how you came up with this idea, let’s continue with the story and talk about what you’re up to. The first sponsor is a company called Toptal. Before we started, I said, “Is it okay if I talk about Toptal?” You said,” As a matter of fact, we use them.” How do you guys use them? And I know that you’re not directly connected to the hiring from Toptal, but what do you know about them, if anything?

Steve: Yeah, so as I said, I can just tell you a little bit, but one of the parts of our product is we need to pull data from a lot of different sources. Whatever systems you’re running on, whatever cloud environment, whatever software you’re integrating with, we want to be able to integrate with all of those things, and so that we have the data to then let you explore. And we don’t have the resources in-house to go and build all those integrations one-on-one, and so that’s where we look to them.

Andrew: That’s a great use case. If you’re out there and you’re looking to hire phenomenal developers and do it fairly quickly, I urge you to go to toptal.com/mixergy. Steve, I’m a little embarrassed to say this to you because if you would’ve used that URL then you would have gotten 80 hours of Toptal developer credit when you pay for your first 80 hours in addition to a no-risk trial period. And you had that no-risk period. If you didn’t like them, you wouldn’t have to pay. They still pay the developer, but you wouldn’t be billed. Anyone out there wants to go check it out, go to top and top of your head, tal, as in talent. That’s Toptal, toptal.com/mixergy.

All right, so I understand now why you left Google as a person who is constantly moving and thinking of new things and wanting to be challenged. What process did you go through to finally decide, I think I’ve got the idea that I’m going to spend the next few years of my life on.

Steve: So there were a few things that came together. One is I had felt and a number of us had felt this problem very directly at Google. So, you know, we felt there was a very clear problem to be solved. It hadn’t been sold. We knew it was a real problem and we knew it in detail. We weren’t just aware of the problem, we were the customer. So, you know, from that perspective, we felt like that’s always a good place to start from. And also it was, and I can start to talk about maybe in a little bit of, we could talk about what makes our product special, but you know, as we thought about what’s wrong with the solutions out there and how can we do better it, it was a very technical challenge.

A lot of the challenges were around to the amount of data that you need to work within a solution like this. The complexity of analyzing that data and the need for speed. If you’re firefighting, the site is broken and I have to fix it and it’s going to take me 50 questions to get to the end of that trail, I want to answer each of those questions real fast and that’s an engineering problem and that played to our strengths as a team. So it was a combination of I am the customer and this is the kind of problem I’m good at solving.

Andrew: Okay. And did you do any research to see if anyone else was interested in this or at this point you understood. You worked at Google, you knew it because I know at some point you reach out to your network. Did that happen before or after you committed?

Steve: Both. More after. When I’m starting something, it’s a little bit of ready, shoot, aim. You know, start doing it, you’ll quickly find out as you do it, whether it’s worth doing or not. At the beginning we didn’t take investment, we didn’t do anything big. So it really wasn’t big. The research was trying to do it and try to get people interested. And I had a strong intuition that this was going to be a good idea. And that’s not always correct, but it felt like enough to take a start with.

Andrew: Where were you not correct? Have you had a big failure of recently? Let’s say before Scalyr?

Steve: Well, this is going back a ways, but building a new web application framework into the teeth of the Dotcom crash, that didn’t turn out too well. But actually I guess kind of directly to your question, the process that especially Sam and I, you know, my friend Sam who I mentioned, have gone through many times is well, like when we did Writely. As I mentioned that we formed this company, Sam, Claudia and I started Upstartle, and we probably went through three or four product prototypes all over the map. We had a peer-to-peer distributed bug database I think was the project we are working on that we then threw out to do Writely. And there had been a couple of other ideas for that.

So our process is we start working on something, and it either quickly becomes apparent that it’s a bad idea or if we fail to find that it’s a bad idea, then we keep going. And so we’ve had a lot of sort of early failures, but we sort of like start something and fail fast or press on.

Andrew: Sam is his last name pronounced, Schillace?

Steve: It depends on what part of the world you were born in, but I think that’s what he usually uses. Yeah.

Andrew: The reason I bring it up is, I saw this article from August 2005, a blog post. He commented. He goes, “Hey, I’m one of the guys who was working here at this company and thanks for the write-up.” And then he came in with more info. You’re working with him now too?

Steve: Sam? No. Claudia? Yes.

Andrew: Claudia? Yes. Okay. And then there was one other person who I saw on the About page. Another Steve. Steve, what’s his last name?

Steve: Czerwinski.

Andrew: Czerwinski. So it’s also a guy that you worked with in the past?

Steve: Yeah. So I met Steve at Google. He was working with me on that project I was talking about, the kind of file system layer. There’s actually, there are three of us at Scalyr who were in the meeting where we were tearing our hair out about how to keep up with all these productions fires.

Andrew: The reason I bring this up is it seemed like from your conversation with our producer that you said you wanted to get the band back together. You knew you had a good group of people. You jived well with them. You created well with them. You wanted to get them back. How soon after you said this is it, did you get them back together and say, “Let’s do it”?

Steve: So it kind of trickled in over time. I think on probably about day three, I reached out to Steven Czerwinski, but he was in the middle of a project. I had gotten to a point where it was like a good stopping point for me to leave Google. He was in the middle of a project and loved leading a project and just didn’t feel right leaving at that point. So I kind of kept wheedling him and he said when the project’s done when’s that going to be and now it’s slipped. A project slips sometimes. But, you know, kind of as soon as he got to a point where he felt like he could leave in good conscience, he came over.

Andrew: Our producer also asked you what did that first version look like, and you said it did give a lot of data, but you could look at it, but it worked and it had them . . . Oh, here. The tool actually worked. It wouldn’t give you a lot of data and it didn’t look a beautiful, but it worked, right?

Steve: Yeah, it, you know, so Claudia had not joined us yet, and Claudia is the UI expert of the team. So this looked a lot like any, you know, ugly backend internal tool some engineer has hacked together because that is exactly what it was. And it was very raw. It worked, the engine was there. It wasn’t as scalable as it is now, but it worked, but it was definitely an expert mode tool.

Andrew: And what could it do? What could that first . . . How did you know even what to include in the first version?

Steve: So that was fair. That was pretty clear from the beginning. The basic idea, you know, so I mentioned how at Google, we realized we were juggling 17 different tools for operational visibility, for understanding what’s going on in these systems. So why was it 17 or why do you need 17 different tools? And it came down to there are a lot of different kinds of data that you work with, logs and metrics and error reports, crash reports, all these different kinds of data and different ways you may want to search and filter through it. You may want to graph some kind of a trend. There’s different ways, different kinds of data, different ways you analyze the data. And each one of those 17 tools, you know, basically it was a certain analysis on a certain kind of data.

The original idea for Scalyr was, like, this is a pretty good set of tools, but it’s insane that it’s split into 17 parts. Let’s just build one tool that can take all those different kinds of data because they’re not that different and do all the different kinds of analysis, you know, in one place. And so the original Scalyr, it was very raw, the UI wasn’t pretty, it didn’t guide you by the hand, but it could take in most of those kinds of data and it had basic versions of all those different analyses.

Andrew: Got it. Okay. So that was the first version. You took it out to your network and the network said what?

Steve: Some of the networks said . . . you know, I mean, you always get a mix and some people, “Oh no, I have no problems in the world. Yeah, I don’t this. Everything’s great.” But a lot of people were excited but then actually when they, you know, they would go to use it, some of them would bounce off because, you know, I’d assumed early on, you know, our users are engineers. Engineers are very sophisticated. They’ll learn anything. They’ll put up with it. Yeah, this UI is bad, but they’ll put up with it. And that turned out to only be sometimes true.

In hindsight, I didn’t get this at the time, but in hindsight, a lot of the people who are using a tool like this, who are going in and investigating issues, that’s actually not the heart of, you know, they’re not operations people, they’re developers and they’re only dabbling in this. Every now and then they get pulled in to fight a fire and they’re not going to climb the learning curve. They’re capable of climbing it, but it’s not worth it to them because this is only a side part of what they do.

And that was actually a lot of what people are wrestling with. There’s other solutions often because of the performance and the complexity of the query language, and in us it was the rawness of the UI and people would bounce off of that, but some people would get into it and when people did . . . so we had a lot of people bouncing off, but when they would get in, they were fanatics and that, for me is . . . I remember reading once and I think this is really true if you have a lot of people who think your product is kind of neat, you’re not in a good place, but if everyone hates it except a few and the few love it, you’re not done but you’re onto something.

Andrew: Yeah. I think Paul Graham said some variation of that a few times. So that’s how you knew that you were onto something. I’m wondering how you even had a network. I feel like once you get into Google, especially if you’re working on an interesting product, you don’t go out and develop your network or did you?

Steve: So I am not the greatest networker in the world and I did not have the greatest network in the world. But, you know, you do enough startups, you know, you connect with people along the way, and other people also leave Google and so is the Google Alumni Networks.

Andrew: Yeah. Okay, so you’re not necessarily staying in touch with them, but you’ve got a few people who are a lot of people that you’ve been around. You’re taking it out to them. It starts to actually make sense. You also told our producer, “But some of them said there some missing features.” What was missing that led you to improve the product?

Steve: So a lot of it came down to, as I touched on, it was very raw. Basically, if you knew what questions to ask, if you knew exactly which set of data you wanted to filter and exactly what view of that you wanted to produce, the initial product worked well, but almost no one knew that. And so we wound up doing a lot of work and we would do a little and think that might help and then we find out, do a little more, do a little more. There was this endless succession, not endless, long succession of improvements to make move it from type your brilliant query and we’ll give you the answer you need to explore the data. And that actually turned out, and I wish I could say this is the plan from the beginning, but it wasn’t, but it turned out this fit really well with.

One of the other key things we did that really set us apart was performance. You know, often you have a huge amount of data coming at all these systems you’re running and it in other solutions, it may take minutes, sometimes an hour to get an answer to your query, you know, scan the data and produce the result you’re looking for and we were able to do that typically in less than a second. And one of the things that turned out to enable is basically we can start answering questions we think you’re going to ask and sort of present speculative views, free to drill down.

And so one thing we would do is you’re looking at a certain set of logs. We added a feature where every search you run, every, you know, filter, you type in, set of logs you look at, we show you what you asked for. And then off to the side, we show what’s called a facet list, and that’s like when you’re shopping on Amazon and you search for toasters and they say, well there’s 18 brands and here are the price ranges. And you know, here are the three sizes of toaster. There’s all those categories.

And even if you don’t really know what you’re looking for or even what there really is, you can use that to explore. So we did that for the log data, and because the engine is so fast, we can afford just go in anything you look at will put that up off on the side. And then what that means is, even if you don’t know what’s in your logs, you can start exploring and you don’t have to type a query and you don’t have to know a query language. You’re just clicking on, I want the Braun toaster or I want the large, you know, the equivalent of that in your log data. Whatever the categories are there, you can see them without knowing what they were and that I think that was what really made the difference. That took it from people love it when they get in, but it takes a very special person to get into something that was much, much more accessible.

Andrew: And I could see a little bit of the way that you think in your ability because someone with more of a sales background would say, we’re going to have customer success people get on calls with everyone and walk them through it. Someone with your development background can say, “How do I come up with an interesting technical solution that only we could create that will show off?” That’s what it was. You’re nodding.

Steve: Yeah.

Andrew: Okay. Let me talk about my second sponsor. It’s, I love, by the way, I told you hardly anyone watches the video anymore, it’s the podcast that’s taken off. I insist on having the video for the interviews because I want to see you nodding that I’m on the right track or hear you say yes, yes, but see in your eyes it’s not really yes, and then I could backtrack and correct myself so it helps.

All right. My second sponsor is a company called DesignCrowd. I got to tell you, Steve, I was not going to include this reader story because while here, let me tell you the reader story and then tell you why not. It’s from Ron Sherbert who emailed me and said, “Andrew, I’ve got four designers on my team, but what I like about going to DesignCrowd is that I could just tell them what I’m looking for and describe it in really simple terms, and then I end up with a lot of designs that really will shake things up for our company.”

Well, the story that he told me, though, had nothing to do with his company and design team. He says, “I was president and coach of a youth football organization and I’m still treasurer of the team.” The organization is called the Worcester Vikings, and they’ve always used a logo that belonged to a college team. And so they said they need a new logo, so he said, “All right, let’s try going to DesignCrowd.”

I’ll continue reading. He says, “We spent around $250 and got over 100 designs to choose from and decided on the one that we liked best, and now we have their very own logo that we can be identified with,” and so he likes and recommends DesignCrowd and I’m proud that he sent it over to me. The reason I wasn’t going to read it, Steve, was, it’s not like a business need. I wanted to show off how you can create brochures, websites, how if you have any design things like cover art for my podcast, you’re doing an event and you need to have good signage, you could take it to them and get 100 plus designs.

And so I wasn’t going to read because it wasn’t business need, and then it occurred to me, Steve, if I could show you, if I could show our listeners that even someone who needs a logo for a local small team can get a great collection of results for a reasonable price and be excited about it, then someone who has an idea for a new design is not going to hesitate and go, “Oh, this is too small. This is not important enough. I don’t know. I’ll just make it my own,” right?

So I’m going to say this to you, Steve, and to everyone else out there, if you need a design and you want not just one person’s take on it, but basically I was going to call it an army, but an army works together. This is not an army. It’s a crowd of people from all over the world, all competing against each other by creating stuff, you’ll get it at DesignCrowd, so simple that you can do it fast. Get a bunch of designs that you give them feedback. They’ll adjust for you and then you get in brand new designs. Then they’ll adjust them again. Anyway, over and over. Don’t go to designcrowd.com because frankly, I will not get credit for it, but you don’t care about me. You care about yourself.

You are not going to get a discount for using just DesignCrowd. If you want to get a big discount from them and see the design I got from them, go to designcrowd.com/mixergy. For some reason, every time I do it, I’m getting a lot of typos right now because I’m doing it fast. When you go there, you’re going to see the big offer that they’re giving you, which is a lower price than you are going to get anywhere else substantially and they already have a price that’s super low.

In fact, Steve, I’ve been saying this but they’re one of the few sponsors that’s not coming back for 2019 and I was thinking why? And I realized, they’re charging too little. When you charge that little and your margin is so small, you have to get like 1,000 customers every time I speak about it for you to pay off. So it doesn’t make sense.

The problem is not that it’s not a good fit for us, it’s just that they’re charging too little. Go on and enjoy the fact that there’s a low price. One of the last times I’ll talk about at it, designcrowd.com/mixergy.

All right. The articles that I see about Sinclair on TechCrunch all have to do with how much funding you raised, which is not what it used to be, which was all about the features and the way you’re changing the world. But it did change your business. You told our producer when you went out for your first round, one of the VCs did something that helped you find a major client that’s still now on your website. I saw it a couple times. What happened?

Steve: Yeah, there was Return Path. And that was actually, I’m telling the story out of order.

Andrew: That’s fine.

Steve: So when we went out to raise our seed round it and it turns out that before a venture fund is going to give you money, they like to learn something about you and learn whether what you’re doing is any good. And one of the ways they do that, especially with a product like ours that really is used by other companies and often tech companies, is they’ll go to their portfolio and, you know, they’ll go to some of the companies in their portfolio and here’s this other company we’re thinking of investing in. What do you think? Would you use these guys?

And so one of the seed funds that we were talking to had invested in this company called Return Path. They asked one of the folks there to take a look at us and long story short, pretty soon Return Path was our first large customer.

Andrew: First one because they were introduced that way.

Steve: Yeah.

Andrew: And then does it allow you then to get the credibility to bring in more customers because people know Return Path and they’d been around for so long? Yes.

Steve: Absolutely. That helps you.

Andrew: You told our producer she’s so good at digging in. We’ve got like the set of questions that we’re looking for that the next big way that you grew, was something called land and expand. This is big in the business space today. What is, land and expand? How do you explain that?

Steve: So it’s really a simple idea. You get someone using your product, they may not be the whole organization. So, you know, for example, like, one of our customers today is Discovery Networks and initially, we didn’t go to the head of technical operations at all Discovery Networks and do a big . . . you know, it was one team that we worked with and we signed a deal with that team and they started sending their data, their logs to Scalyr and started using us. And then we spread through the organization.

So, you know, the idea is partly that it’s, you know, it’s just a lot easier as a sales process to close a team rather than a whole company. It’s a smaller check, fewer approvals are needed. You only have a few people to work with. They want to do it, they do it great. It’s a complicated process and then you’re in the door. And, if somebody is using something good, they’ll tell the person who sits next to them or they’ll tell their counterpart on the team across the hall who they have lunch with from time to time. So that’s it. The land is, you get that first team and then they expand is a sort of goes viral ideally within the organization.

Andrew: Yeah. Like one of your clients is a company called Zalando. I didn’t know anything about them, but I’m on their website now. There are only a European brand, not any like options for the U.S., but boy, I think my wife would look great in this stuff.

They have 100 different development teams. Once you get one engineering team internally to try it, then a second and a third all start to take over. Here’s the part that made me want to bring this up. I don’t think of your type of company as being the type of company that starts from the ground up that anyone who’s interested in solving a problem would go in and sign up for. It seems like more of a top down. Like this is going to be part of our toolset, we’re going to put it in internally and put it down and make sure everyone uses it. And is that because you’re only analyzing the data that I’m already creating so I don’t have to implement anything into my website in order to get to get this insight or am I misunderstanding?

Steve: So an interesting question. You know, it’s interesting you asked that because I’d never really considered the question from that angle before. We were just sort of living this and we haven’t taken a step back to examine it, but I think that’s a good point you’ve made. It is relatively easy to adopt a product like ours. It doesn’t require a lot of other changes. It certainly doesn’t require. You know, if I’m one team and I want to do this, I don’t need to ask any other team to do anything to make that possible.

Andrew: Do I need to put anything into my site? Do I need to put anything into my app? Nothing. It’s analyzing the data that’s already being put out there.

Steve: Yeah. Typically you’re dropping what we call our agent onto your servers, but then it’s basically just a small configuration change in your server. You’re not changing code. You’re modifying the script that sets your servers up to add one more small piece of software. And if your team wants to do that for the servers that run your applications, it doesn’t affect the teams around you.

Andrew: I’m hunting around for info about your traffic and your customers on SimilarWeb tool that I use all the time. I’m wondering if at this point in the story you started buying ads. Were you doing any ad buys?

Steve: I can’t give you a great timeline off the top of my head, but that’s definitely a component of what we’ve done. Interestingly I found lately that LinkedIn ads have worked fairly well for us, which kind of surprised us because that’s not the first place you think of where engineers go to spend their time, but for some reason, it’s been working. The truth is I’m not close enough to that side of things to say too much.

One thing that has worked very well for us and really helped us in the beginning, it got us Zalando, it got us some of our other big customers after Return Path, was just technical blogging. We wrote some very detailed articles about . . . I talk about how we differentiate because we’re much faster and much better at working with these large data sets. We had to do some really interesting engineering to accomplish that kind of borrowing some ways of thinking about things that we learned at Google. We wrote some pretty detailed blog posts about it early on and that did really well for us because that’s interesting to the kind of person who also would then use our products.

Andrew: That makes sense. And then to create that kind of team, this was after you raised money, right? And then you started hiring for it. It doesn’t seem like it was a skill that you came to Scalyr with. Am I right?

Steve: You’re right. Yeah. That was definitely something I’ve had to learn though. We’ve all had to learn on the job.

Andrew: Like I feel like even your name when I look it up, I guess there’s some politician from Virginia or something who has your name and he out SEOs you like crazy. Where there a lot of founders, they’ll out SEO everybody because they’re kind of obsessed with it. What I found about your earlier writing, your company’s earlier writing was it was like a charm of like homemade products, charm of being proud of the work. And now it’s insane. It’s good.

You told our producer though that at one point, I’m constantly mentioning the producer to let people know we do our homework here. We’ve got a team. That in the beginning it was hard to get customers. And we kind of passed over that a little fast and I failed to help you explain the difficulty. What was it like when you were just starting out after having succeeded so much, trying to wander around and find customers? Take me to that.

Steve: Yeah, it’s hard. And you know, earlier we talked a little bit about in hindsight some of the reasons why it was hard. The product really wasn’t very approachable and we misestimated how important that was going to be to this target audience. But so the effect is, you know, I would go around to my network, I would go around to the kinds of people who, you know, we were convinced, you know, you’re who were building this for. The whole hypothesis is that you’re going to love it. And 9 times out of 10 we’d bounce off.

Maybe people would get excited when they first heard the idea and then we would send them the invite link and then they wouldn’t really go anywhere. And that’s always hard. But, I’ve been through it before. And after you’ve been through this a few times, you learn that just because you haven’t connected right away doesn’t mean that you need to give up. You need to figure out why and you need to iterate. But there’s an expectation. I think that it’s going to take you a few turns of the iteration crank to actually put all the pieces to something people want, that you’re explaining in a way they can understand and that actually, you know, the product experience actually works when they open it up and you’re talking to the right people. And so, but you have to have, I think, some courage of your convictions to sweat through that period.

Andrew: And you were convicted. The conviction was what? You were convinced that what was needed and what was going to happen? What did you see?

Steve: That systems are only getting more complex. There are these big technical macro trends, the cloud and microservices and they’re the only things going on that make it easier and easier to build the complicated systems. And we’re reaping the benefits of that, right? That, you know, every day there’s 83 new apps that launch to do incredible things is because there’s all the supporting technology. But behind the scenes, you know, these more and more sophisticated apps are being built in very complicated ways and that’s with technologies like the cloud where, you know, they really are changing things.

So the behind the scenes, the problem of what’s going on. Why did this break? You know, so many places for things to go wrong. Like, you know, we had felt that problem at Google. There were these macro trends that said, you know, everyone is going to be. Google was ahead of the curve on the inside, but like everyone is going here.

And as we were then left and started building Scalyr and everything that we could see that bearing out and these existing tools just aren’t cutting it. We felt that inside Google, talked to people outside Google, you know, what are you using to work with your logs? How long does it take you to run a query? How long? So what do you do about that? Oh, you don’t use it if at all possible. Got, you know.

So you know, the basic validation was there. It was a really hard problem. It’s important to people. It’s important to keep your site running. It’s only getting worse and the tools aren’t doing it. So that was the conviction. We knew there was a real problem. We knew it hadn’t been solved and the leap of faith was that we would know how to solve it.

Andrew: Which you feel confident about as a team of developers. I saw somewhere on your website, there it is. For engineers by engineers and you’ve highlighted the by engineers part. You’re really proud of that that is your skill set. Let me close it out with this. The other thing that I see on your about pages, that you learned to program at eight years old and you’ve been a lifelong engineer. I also noted earlier that you’re a guy who started multiple companies. I think Scalyr is your sixth company. At what age did you start to sell? What’s one thing that you created that you programmed yourself as a kid that you started to sell if anything?

Steve: So I forget the exact year, but I was probably about 11 or 12. And so this was an I’m going to very clearly date myself, but this was 1978, 1979 and one of the hot things in computing back then was the Commodore PET and I used to laugh. My wife and I were watching “The Americans” last year and you can see that in the background every Commodore in their office. Commodore PET.

Andrew: That computer that’s got the monitor and the keyboard connected to each other.

Steve: Yeah. And it’s sort of a funny trapezoid shape and the keys are safe for an 11-year-old. So I liked it, but all the grownups hated it. And I wrote up a little pinball simulator and software on the PET was distributed on audio cassette tapes and I borrowed some way or someone had a tape duplicator, ran off about a dozen of them, photocopied the one page instruction sheet that I typed in my mother’s typewriter, and my father had a retail computer store and he put them up on the counter there and we sold a few copies.

Andrew: Wow. And because you are an entrepreneur and developer from an early age, that’s what I’m sensing in that story. And, or am I reading too much?

Steve: I wouldn’t have known those words at the time. But yeah, the itch was there.

Andrew: The itch was there and the itch is continuing for anyone who wants to go check out the website, it’s Scalyr, scalyr.com. And I want to thank the two companies that made this interview possible. The first is a company both Steve and I have used. If you’re hiring developers, you should check them out too. No obligation, just go to top toptal.com/mixergy. They’ll have a conversation with you and you’ll know whether it’s a good fit or not. Frankly, for a lot of people, it’s not a good fit and they turn you away and so people complain to me, but I’d rather they could turn people away if they’re not a good fit.

Second is a company that will help you get your designs done right. Go to designcrowd.com/mixergy. One of the last chances are you’ll get to use that and get that incredible discount.

And finally I want to close out by mentioning somebody who was on Mixergy a long time ago, and it was a longtime Mixergy listener. Gabriel Weinberg is the founder of DuckDuckGo. Do you use them? Do you know them? You do now, Steve.

Steve: Oh, yeah, no, I think they do neat stuff.

Andrew: Yeah, he was this dude. He was sitting at home, I think in his basement. I think he was like the house husband. While his wife was working and he was an entrepreneur. Had success, so could stay home, but he was just coding up a competitor to Google. And while you were working in Google, I remember saying to him, dude, what are you doing taking on Google? There are more people in Google’s bathrooms, no exaggeration, working on search than you got in a whole fricken company, but he just kept at it and the thing that he discovered was privacy. He had to create the privacy search engine.

He then started building it up and building it up, building up. I’m looking@duckduckgo.com/traffic. I see his traffic and I’ve been following that page for years. It’s growing and shooting up. Partially because people care about their privacy and he’s growing it, partially now it’s built into Safari. It’s built into Firefox. It’s grown insanely fast and he’s a real competitor in the search space.

He’s not taking down Google, but he’s creating a successful search engine company. And the reason I bring it up as he was with Mixergy for so long that he created one of the first Mixergy courses, all about how to get traction.

If you’re at all interested in how to grow, especially if you’re like a guy like him who cares about his product and maybe you’re taking on a tough battle, you should go check out his course where he teaches you how to get traffic, traction for your product, and he’s a guy who not only did it before, he taught, he’s doing it since. Go check him out at mixergy.com/premium. Learn from him. He’s fricking brilliant. There’s a venture capitalist in the Bay Area. He gives this out to the entrepreneurs he invests in because he believes so much. He gives a Gabriel Weinberg’s book out because he believed so much in this process.

If you want to check out the course, it’s mixergypremium.com or mixergy.com/premium. Steve, I just had to get that off my chest. I was so proud of that, that he’s doing so well. All right. Thank you so much for doing this, Steve.

Steve: Yep. Thank you. This was fun.

Andrew: Thanks. Bye, everyone.

Who should we feature on Mixergy? Let us know who you think would make a great interviewee.

x