RSS feed
Eric Ries: I met the Anthropic team when they first left OpenAI. They were really committed to the idea that this new generative AI should be used and commercialized for the benefit of all humanity. The irony of this whole situation is one of their early backers was FTX. Vibe coding era is gonna be remembered for a Chernobyl-style disaster, is my prediction.
The transformer technology that is the basis of all modern LLMs was invented at Google. If you look at the co-authors of that paper, they all, every single one, had to leave and do it elsewhere. I know– When someone sends me an AI-generated thing, I always know it instantly. It’s always garbage. Too many people are excited about using AI to replace human creativity instead of augmenting it.
Andrew Warner: Eric Ries, who helped so many entrepreneurs build phenomenally successful businesses based on his lean startup philosophy, is back with a new book called Incorruptible, where he talks about why some companies succeed, do well over the long term, and others just keep dwindling. And that’s what we’re here to talk about today, specifically related to AI startups to see what’s working, what’s not, and also what he’s building himself.
Let’s get into it. Presented by Zapier, the AI automation company. What’s Anthropic’s mission, and how have they been able to do well because of it?
Eric Ries: I met the Anthropic team when they first left OpenAI, and they, you know, they left over a dispute with OpenAI over, um, you know, exactly how to pursue the, the question of AI safety.
So they were really committed to the idea that this new generative AI should be used and commercialized for the benefit of all humanity, and they were very worried about certain specific safety scenarios that at the time seemed like far-out science fiction. I remember being like: “Whoa, are we really imminent?
Is this, is this imminent, or is this like a long…” They were like: “No, man, it’s imminent. We need to be working on this right now.” So they were farsighted in caring about that stuff. And again, everyone wants to be like, okay, put, put– AI safety is now a super polarized debate. And of course, I have my own opinions about AI safety.
I’m happy to talk about that. But I would ask for the purpose of this conversation, like put that aside for a second. They were committed to it. So my position with them was like: “Let me help you encode these commitments into your company structure.” ‘Cause that’s really what they were worried about. It’s like: “Well, we’re gonna raise all this money.”
And I remember they at first were like: “We think the solution would be to raise money from really, really values-aligned people,” which they did. They, they’re a neighb- They were in a, in a position to really curate their initial cap table. They were very selective about who they took money from. But I remember we had this conversation about, “But what if you’re wrong?”
Like, what if somebody who seemed really values-aligned turns out not to be? And what if, like what if investors who are aligned, like naturally, when the amount of money we’re talking– Because they were talking about, you know, AGI before that was in the news every single day. We’re talking about s- uh, technology that could be worth trillions of dollars, but maybe hundreds of trillions of dollars.
And I was like: “Are, are you really so confident that these people will be able to maintain their principles even in the face of this overwhelming temptation?” And you know, they were– they took it really seriously. Okay, we need to figure out how we’re gonna build the structure that is, um, resilient even if Even if someone tries to betray the values, even if someone turns out to be unaligned.
Now, the irony of this whole situation is one of their early backers was FTX. So they thought those guys were super aligned because of their, you know, supposed commitment to effective altruism, and it actually turned out to be a total disaster when that company imploded. Huge chunks of Anthropic stock were sold at auction to any investor who wanted them, including people who are super, super unaligned.
So it actually turned out to be extremely important. And, and I mean, we can talk about the structural stuff, you know, the, the elements of building a company that can be truly strong.
Andrew Warner: Did, did you help them do that? Did you help them actually codify? Yeah. You did?
Eric Ries: Yeah, but I mean, I don’t wanna take credit for what they’ve accomplished.
I played a very bit part, um- Okay, and
Andrew Warner: so- …
Eric Ries: in this, uh, in this whole story. Yeah.
Andrew Warner: So you helped them codify their, their beliefs. They then said, “We only want investors who align with what we stand for.” They ended up with an investor who said he did, Sam Bankman-Fried, very famously. Turns out he didn’t, and then random people were able to buy his shares.
Why were they still able to maintain their, their I mean, why were they able to still continue standing for what they meant, for what they did before?
Eric Ries: Yeah. So there’s two components to it. What, what– I would think of this now like the inner part of it and the outer part of it. And I know, again, it’s so natural that now, now people see Anthropic as this mega company.
I totally get it, and so it can seem inaccessible. But remember what– to me, they’re just a couple guys in a garage. Like I, I met them when they were no- like it was not a big deal except to people who were very, very in the know about this inside baseball stuff, um, related to AI. And they did, they made the critical commitments then before billions and hundreds of billions of dollars were, um, were at stake, which is a really important part of it.
So the inner part of it is really about alignment, coherence. Can we get everyone in the organization really committed to some set of principles or some kind of thing, some kind of vision? And then the second part is what I call integrity, which is not some vague thing. I know people hear these language, they’re like, they worry we’re gonna talk about morality.
What I mean is more like structural integrity, the ability to make and keep promises, and the ability to be trusted by employees, by customers, by partners, and even by investors. So Anthropic’s structure is not your typical investors run the show, what’s called shareholder primacy, uh, set of practices. Uh, their, uh, company is governed by something called the Long-Term Benefit Trust, which is an outside set of trustees that are not like a vague advisory board or like remember the Trust and Safety Council that, uh, that Facebook famously had and then totally ignored.
Um, the LTBT has the actual power to appoint members of the directors of the board of the for-profit entity of Anthropic. So they have real, uh, effectively veto power over things Anthropic does, and their job is to act as guardians, they call them mission guardians, uh, to make sure that Anthropic never deviates from that mission.
Andrew Warner: And so that’s why when the government asked for carte blanche over their technology, they said, “We can’t. This just goes against what we bel- Mm-hmm … what we believe.”
Eric Ries: Yeah, and you know, it’s– people now, now we know how it turned out. Obviously, like the, the district, district court or the, the, yeah, so the district court just gave a preliminary injunction in which they, they absolutely, uh, demolished the government’s case, saying this was a case of, of overreach.
Um, and not only that, but like they h- they’ve reaped already, even though they g-gave up $200 million, a $200 million contract, even for a big company like OpenAi, is actually really a lot of money. And, and you can tell because their competitors all raced to go get this contract as soon as they gave it up.
Um, but like Claude went to number one the next day, like that was a huge boost. I saw a video, some friend of mine sent me a video. People had come out to chalk up the sidewalk outside of their headquarters, thanking them for doing this. And believe me, like that is not typical for tech companies in San Francisco right now.
If you’re being chalked up, it is not usually- Right … in a positive way. Like they, it cau- when you do the right thing, it, it causes these positive ripples sometimes which redound to your commercial advantage. But the key to the whole thing is you have to be willing to do what’s right whether or not you know that you’re gonna be rewarded for it.
You do it the right thing for its own sake, and you trust that, that and the chips will fall where they may. And I think that kind of strength is only made possible if you’re not always looking over your shoulder like, “Uh-oh, I’m worried I could be fired. Uh-oh, I’m worried investors might not like it. I’m worried about the next activist campaign,” or whatever.
Um, so I think, I think that that courageous stand that they took, again, leaving aside the politics of it, just looking at it from a pure business perspective- Mm-hmm … is an example of a very savvy business strategy that sees trustworthiness as an asset, like a business asset that can be intentionally cultivated and acquired.
And in fact, I write in the book, I think it’s one of the most underrated assets in the world today.
Andrew Warner: But are they doing well because of that? Or are they doing well because Claude Code is good, because they kept shipping Claude CoWork and so many other tools? Honestly, does that even really matter? Isn’t it just, is the product good at the best price possible, and the rest is just- Well-
we like you?
Eric Ries: Okay. Yes, you can say that it’s b- uh, it’s because they have the best product, but the question you gotta ask, I think, is the deeper question of, well, why do they have the best product? You look at who chose to work for them. Like, part of the reason they have such good products- Mm … is ’cause they’ve been able to attract such incredible talent.
Why? And especially at the beginning, they, they’ve been the, the– You know, they were behind, percei- widely perceived by a lot of people as being behind OpenAI for a really long time. And even b- in some cases, I’ve heard people say that they’re, they’ve been behind, you know, behind Google, behind Grok, behind whoever.
Like, they, they were not the, uh, odds-on favorite to win this race. And part of the reason they’ve been able to do such a good job is simply because they’ve been able to attract capital and, um, uh, a- and employees. But then the other question you gotta ask is, they’re ahead in enterprise. Why? People always say, “Well, they have the best product,” but, like, is it your experience that enterprise procurement departments have a history of, like, being really able to meritocratically judge the best product, and it’s so reliable?
Like, we, we kind of n- we overlook sometimes the facts that are staring us right in the face. I think it’s extremely obvious if you look at the data that’s been published so far, that part of the reason they have an enterprise advantage is because they have a trust advantage. If you are worried about the liability of adopting a platform- Mm
like that, it matters who your vendor is. And, you know, the ones that have a kinda cowboy kamikaze ethos, that’s very scary for enterprise customers.
Andrew Warner: But let me give you, let me give you a counterexample. From your book, you talk about Google.
Eric Ries: Oh, yeah. This is the real- Yeah. Yeah, this is not my, like, personal judgment of Google.
And listen, Google’s a great company. That, that’s not– Again, it was not about absolute right, absolute wrong. Um, I made a study of people who have blogged about what it was like to work at Google, who had been there for 10 years or more, and who left. So there’s like, there’s so many of these blog posts out there.
B- Google people are very prolific in their blog posts, uh, after they leave. So it’s, it’s, uh, unique only insofar as we have this beautiful glimpse into what it was like. And a very recurring theme of those blog posts is people talk about this creeping mediocrity and kind of loss of something that made the company special.
They, they, they talk about it like a grief, like something precious was lost, and they can’t figure out where it went. Like, they, they don’t say Google’s leadership sucks. They don’t say Google’s a bad company. They said Google had great culture, great leadership, great intentions, and yet despite all of those advantages, plus Larry and Sergey have dual class shares, remember.
Despite all those advantages, this thing was lost. One of the, one of the, um- Former employees put it this way. He said, “Over my tenure,” I think he’d been there like 13 years, “decisions went from being made for the benefit of the customer to being made for the benefit of Google to being made for the benefit of whoever’s making the decision.”
So yes, Google’s still around. They’re doing fine. But over time, this kind of corruption is corrosive to their ability to create value. And of course, we’re talking about Anthropic and OpenAI and these guys. Don’t forget that the transformer technology that is the basis of all modern LLMs was invented at Google.
And if you look at the, um, co-authors of that paper, it has a ton of co-authors, not a single one commercialized the transformer at Google. They all, every single one had to leave and do it elsewhere. So this is not just a matter of morality or virtue signaling. It has real, tangible business consequences
Andrew Warner: The reason I was going to use them is that Gemini is not a bad product.
It’s a good product. NotebookLM is something we use on a regular basis. I could keep on going through all their products that, that are good. Oh,
Eric Ries: yeah. I– listen, I use Google products every day. Like I said, Google’s a great company. Now, you have to– but you have to ask the counterfactual. Since we’re, since we’re saying, “Oh, it doesn’t matter,” we have to ask, what would it have been like?
What– how would, what– how would the– how would Google be today if they had been the leaders in this category instead of allowing their own employees to leave and do it elsewhere? So how much was that head start that they gave to the rest of the industry? How much was that head start worth? Um, now, I think if you do it, if you do the math, that’s a bigger corporate debacle from a value creation, value loss perspective than Kodak’s inability to commercialize a digital camera, just because the opportunity cost of that is so immense.
We’re talking about companies with hundreds of billions of dollars of valuation that had, that, that were created by ex-Google people, that could have been created at Google, but had to be created elsewhere. So again- Mm … we can’t know for sure. Uh, it’s hypothetical. You can never know with a hypothetical.
But the good news is the book is not about hypotheticals. We have so many examples, practically in every industry, of companies that have bucked this trend. And yet when you study those companies, you’ll notice they all pretty much, every single one, violate many of today’s supposed best practices about how companies are supposed to be run, created, and governed.
And I basically think today’s best practices are a value-destroying mess, and part of my goal in writing the book is to replace them with better best practices. So we mentioned Costco. Costco was repeatedly attacked over its decades of life by activist campaigns that have, that have accused them of having bad governance.
But you also have companies like Patagonia and Vanguard and Novo Nordisk and IKEA and John Lewis Partnership in, uh, in the UK. These companies that a lot of them have been around for 40, 50, 60, 80, 100 years and are still basically true to the ethos that remained, even though we’re living in a time when average corporate lifetimes are collapsing.
Andrew Warner: So what else? What else goes into this?
Eric Ries: So when we talk about an organization that does the right thing, we’re talking about an organization whose character is consistent and aligned with human flourishing. So a big part of the book is how do you create such a thing? How do you instill that deep down into the bones of a company?
And, um, what you have to realize is that for the vast majority of the history of time there have been joint stock corporations. It was seen as completely obvious that corporations should be incorporated to do a specific thing. And in fact, in the 19th century, to convert a company from a mission of doing any specific thing to just, “I’m just gonna enrich my shareholders,” would’ve been seen as a crime.
And your corporate charter would be voided. Like this is not, this is not actually what is the foundation of capitalism. So anyway, because of the shareholder primacy thing, shareholder primacy basically says that anyone who’s rich enough can take over any company they want at any time. And most founders are hopelessly naive about this point.
They think that they’re in control of the company, and they’re always so betrayed when they find out that their own founding documents basically say that they can be removed at any time. And they don’t realize that that’s a choice. But rule by the richest would be bad enough. This is actually kind of more like rule by whoever can borrow the most money.
‘Cause you don’t actually have to be that rich to take over a company, as banks will loan you the money. And we’ve seen lots of examples, you know, in recent years of people borrowing a lot of money, taking over a company, and doing whatever they want with it. And, you know, there are people who defend that on the basis of free market, free markets, and you should be allowed to do any crazy thing that you want.
Okay, this is not a book about policy and politics. To me, the question is, we who build organizations of all sizes, do we think that’s a good idea? Do we think that’s actually value-creating? And almost every pe- person I know who works for a living, who builds things for a living, has an intuitive sense that that cannot be right.
And I think we have to, um, adjust our formal categories, our formal definitions to bring them more in line with this intuitive understanding that all builders share.
Andrew Warner: All right. Can I talk about minimum viable product for a moment?
Eric Ries: Sure, yeah.
Andrew Warner: Okay. It was the most exciting idea in the startup world for a long time, because in– you basically brought us down from trying to build too much to simple.
The thing that I wonder though, Eric, is We talked about Claude. Is Claude going to take over every little MVP? Is it– Is the idea of an MVP still possible? Will it still be possible a year from now to be able to do that? Or are we going to now, in every turn, be competing with Claude Code and OpenAI and customers who are expecting perfection because that’s what they’re getting already?
Eric Ries: Well, I have a real contrarian view here. I know this is not the hotness at the exact– at this exact moment, but I think too many people are excited about using AI to replace human creativity instead of augmenting it, and I don’t think that’s gonna work very well. Like this, the vibe coding era is gonna be remembered for a Chernobyl-style disaster, is my prediction.
V- people are vibe coding software that they do not understand, and it’s only a matter of time, with all the hype that’s being poured into this right now, it is only a matter of time before somebody deploys a vibe-coded solution to a mission-critical system that not only wasn’t reviewed, cannot be reviewed because nobody– there’s no human being who could possibly understand what the code does.
I’ve had a lot of experience with vibe coding. I’ve, I’ve done a lot of stuff in Claude Code and in other tools, so I know what I’m talking about. The, um, the, the fallacy, the error that people are making, uh, this is a really important thing for people who wanna use these tools for MVPs. If you focus your energy with these tools on artifacts, if you see like, “Oh, Claude Code is awesome ’cause it can make me a perfect artifact,” then you’re making a double mistake.
First of all, you don’t know how good the artifact is. We have a lot of good evidence now that when people get enamored with their own creations, they way overrate how valuable they are. So you don’t really know what you’re talking about. So the, the part of MVP that is about getting feedback from human beings is more important than ever.
Don’t delude yourself. The, uh, we, uh, we talk about the research that’s been done in this area. But the second, I think even more important part, is when you’re using Claude Code to create artifacts, you are actually causing skill atrophy in yourself. You have gone from f- the flow state of building stuff into what we call dark flow, which is the state you get into when you’re using a slot machine.
And you just see it. Like I, I have apps where I’m just like, I’m hitting next, next, next on Claude Code, and I’m like, “Ah, just whatever you do, whatever. Yeah, it sounds good, sounds good, sounds good.” And I notice over time, Claude will start to make design suggestions. “Hey, I think this would be a good design,” and you’re like, “Oh yeah, it sounds pretty good.”
And like when I review, where did the design end up compared to what I originally envisioned? I’m like, “Wait a minute.” That’s not the app I was even trying to create. So I think, I think we have to be, um, much more careful about how we use these technologies, and especially for MVPs. Now, I think it’s not– but it’s not the underlying models that are the problem, it’s the way that they’re– that the harnesses that they’re being embedded into.
Now, I’m talking my own book here ’cause I helped start an AI research lab around this contrarian thesis, so obviously, you know, uh, I, I, I’m a believer in that thesis. But I think, I think we have really good evidence that it’s right.
Andrew Warner: Uh, because we’re disconnecting ourselves from the final customer, and we’re getting stuff that we feel we made, and anything we make we feel more love for.
And so we’re twice delusional. Yeah. Once for m- having made it- We’re
Eric Ries: twice delusional. If you– I don’t, I don’t know if you’ve ever seen this, this psychology research. It’s incredible. They put people in an MRI machine and scan their brain, then they ask them to do various tasks to see what lights up. And they’ll have them do something like, um, read their favorite poem.
People who, like, love poetry, and they read your favorite poem that you ever read in your life. And they’ll have them read it in the MRI, and you’ll see the pleasure centers of the brain just go foosh. You know, it’s like your favorite– You’re remembering your favorite thing. You’re reading your favorite poem.
Then they’ll be like, “Great. Can you take two minutes and write a poem yourself?” And the person will be like, “I suck at poetry. I can’t…” Just like, just anything. Just whatever you can do. Write a poem. And they’ll be like, okay. They’ll take two minutes writing the crappiest poem you ever heard. They’ll put them back in the MRI machine.
Now be like, “Now read the poem you just wrote.” And the brain centers light up just the same as they were reading Keats or whoever else. You can’t imagine how addicted you are to the feeling of the thing as your precious thing that you created. It is s- causes such incredible delusion.
Andrew Warner: I find that myself, too.
You’re right. Like if I create it, I design it, there’s some beauty in it. Uh, like a Suno song that I made is just going to hit me so hard and emotional. Might even make me cry. I thought part of it is because it’s tapping into my own use. You’re saying in addition to that, the fact that I created it makes me love it.
Eric Ries: The f- yeah, the fact that you yourself created it, it makes it so much, uh, more valuable to you. And unfortunately, um, vibe coding gives you the simulacrum of having created it. So you feel, uh, the same sense of ownership over it, so you cr- it lights up the same pleasure centers in the brain, even though in a lot of cases you don’t even understand what you’ve created.
And like how many of us have received a vibe coded proposal or email? Like I know, I know the Claude co-work. I know all the major tools. I know their house style of how they like to style things. So I know– I– When someone sends me an AI-generated thing, I always know it instantly. And like, it’s always garbage.
So like think about it this way. How can the things that other people are vibe coding be garbage, and the things that you’re vibe coding are great? What are the odds, right? Like what’s happening is you’re using different parts of your brain to evaluate. So like, so at AnswerAI, we’ve built these tools that are human-in-the-loop, um, that are designed to help you learn how to produce the artifact.
And this has always been the insight of Lean Startup going back many, many years. We called it validated learning. The learning is the asset, not the artifact. So this has been like one of the biggest, you know, like crusades of my career is starting to get people to value actual scientific learning. And what’s cool about LLMs, I think, is not that they’re artifact-generating machines.
They’re okay at that. But like take them out of their distribution, and all of a sudden you’re in big trouble. But they’re incredibly good teaching machines. They’re maybe the best teaching technology we have ever developed in history. And so if you just get into the habit, and of course, if your tools are designed for learning primacy, it’s better.
But using any tool, s- instead of saying, “Make me an artifact,” just say, “Teach me how to build the artifact Teach me how to do it. And every, everything it creates for you, insist that it walks you through step by step exactly what it does. Make sure you– make it quiz you to see if you actually understand.
And people hear that, they’re like, “But I, I have Claude Code generating thousands of lines of code a second. There’s not time for that.” I’m like, “Right, that’s what I’m talking about.” So you wanna be responsible, morally, ethically, and economically responsible for deploying these armies of robots that you don’t understand?
You’re much, much better, I think in the long run, people who invest now in craftsmanship, in a deep understanding of software principles, architectural principles, artistic principles, whatever the thing is, are gonna be so much better off. They’re gonna wind up being, like, incredibly super productive cyborgs that are gonna run circles around the vibe coders.
Andrew Warner: Because they know how to code or because they understand how a vibe coder– No, it seems like you’re saying because they know how to code.
Eric Ries: Well, I don’t care if you can type the physical lines on your keyboard, right? Mm-hmm. Like, so if you say, “I’m a great writer,” you’re like, “Really? You have excellent penmanship?”
Andrew Warner: Okay.
Eric Ries: No. My-
Andrew Warner: No.
Eric Ries: You should see my handwriting. My handwriting is just terrible. Like, to be a great writer is to understand the craft of writing, and to be a great programmer is to understand the craft of software engineering. That’s what we’re talking about. So I think these tools will make a, make that, that elite level of performance available to far, far, far more people than is currently possible with the way we teach people programming.
So I think a lot of vibe coders can graduate to this skill. All that is required is you just have to be determined to understand what you’re doing, and if you have that hunger to understand, you will become far, far more powerful.
Andrew Warner: You know, let’s take it to writing, ’cause writing is universally understood.
I’ve never found that AI can write well for me. No. What you’re saying is don’t even try. Instead, maybe give it the transcript from this interview and have it help me figure out what I should write. Not even create a first version, but like- Yeah … ask me the right questions to understand the meaning?
That’s-
Eric Ries: Yeah, you got it exactly right. And, and it’s so it can critique too. So like, so I– listen, I used AI. I know this is like not popular to admit this kind of thing, but it’s so funny. Vibe coding, you’re supposed to be like, “I vibe coded, I’m vibe maxing,” and whatever, token maxing. But in writing, you’re supposed to be like, “Oh, no, God forbid anything AI helped.”
So like I use AI a lot with the writing and research of this book. It was incredible because I had a, I had a fully custom rig that kept me in control. So AI did not write the prose for me, but like when I would write something, it would have available to– it would create a context for me. So we had what we call shared context at AnswerAI, where I and the AI have the same information, and we see the same steps and can, we can go back and modify any steps if a wrong turn was made.
And it was the context was not just what I had write my previous draft, but also I had like hundreds of test readers. I had 10,000 comments from test readers on this book, and I had access to all of them while I was writing, but that’s too many to– you can’t look at 10,000 comments. The AI would find the comments that are related to the thing I’m writing to make sure I have that context available.
Same with research. I had this massive research archive, more than I could possibly keep in my own brain at any given time. So it would help me make sure I was aware of what information was relevant. And then it has, it’s been trained on all the literature that has ever been written by a human being. W- unfortunately, speaking of someone who’s in the training data without any of us being compensated, by the way, which I think is atrocious.
Um, even still It has knowledge about writing. So it’s like, you can be like, “Critique this for me. Is this really good? Is this 10 of 10 good? What, what– how could it be improved?” And again, don’t listen to its suggestions ’cause it will often go back to it’s not X, it’s Y, and all this other, like… It’ll start loading it up with em dashes and other garbage.
You can’t do that. Instead, you could say, like, “Is this really good?” And what I would find is basically I would get into an iteration cycle where, um, I would bring in the research, I would synthesize. Sometimes I would have it help me do an outline. Sometimes we would just go paragraph by paragraph. And after, this is the key, I would work with it for a while until it was convinced we had the best possible thing we could make.
It’s like this is 10 of 10. When the AI says it’s 10 of 10, now our work- now the work begins. Okay. See, people stop there. But no, now we have reached the limits of its training data. Now it’s time to actually do the creative act of writing, which often would be like, “Okay, now I see how it’s like this is a very basic way of arranging this information.
Now let me try to take it to the next level and bring my own unique skill and creativity to bear.” But from a– as a writer, for me, the hardest part of being a writer is the blank page or the feeling of like, “What do I do next?” And AI’s just extremely good at you’re like, “Look, I have 100 things I gotta do.
Will you just pick one for me and let’s do it?” And like help me, like or, you know, I would get interrupted. I have young kids. I get interrupted. I was in the middle. I was doing great, the truly great writing, and I get interrupted. Now I can’t reme– And it’s like you can just be like, “Where was I?” Like, “Get me back up to speed on what I was just doing.
Help me make forward progress.” So for me, it was extremely powerful as a tool, and I think, I think we only scratched the surface of what it’s capable of. But again, because I was 100% focused on having it improve my own craft and skill, not on having it do the artifact for me.
Andrew Warner: Can I see it? Can I see what setup you used, what tool you used, and like how you used it?
Eric Ries: Sure, yeah. Yeah. Um, I don’t know if it’s public yet. Uh, the, the tool is called, it’s called Solve It at AnswerAI that I do to, to build all this stuff. I just don’t know if we have made it public. Um-
Andrew Warner: I think the video is right on the page.
Eric Ries: Oh, if the video’s right on the page, then it’s already public, and yeah, go to solveit.com.
Sorry, I just, I didn’t wanna, I don’t wanna overpromise when I’m not 100% sure what’s public yet.
Andrew Warner: But ul- ultimately, when you were writing, what were you looking at?
Eric Ries: Solve It is based on Jupyter Notebooks for those that know what that is. Mm-hmm. Which is basically like a messaging system where instead of a chat, we create structured messages, and the messages can be content like markdown, you know, just text.
They can be prompts to the AI to do something, or they can be Python code.
Andrew Warner: Is this what it looks like?
Eric Ries: Yep.
Andrew Warner: And so this is what you were staring at as you were writing.
Eric Ries: Mm-hmm.
Andrew Warner: And so what am I looking at here? The top is what, and then the bottom is what?
Eric Ries: Yeah, the red pro- the red boxes are prompts, so the, the, the brain i- little brain icons tells you that, that Claude is responding in thinking mode.
Andrew Warner: Okay.
Eric Ries: Um, and so, uh, the green boxes are notes. That’s just raw markdown. That’s just the information. Um- Okay … so like you were talking about, like, taking the, uh, an audio of this, um, interview and make a blog post out of it. Like many- Yeah … AnswerAI blog posts are done that way. We just sit down, we record a discussion about it, and then we transcribe it and bring it into this, uh, into this thing.
And the key is– So what happens is, um, LLMs, like, people make AI into this magical thing, and I just really, really encourage everybody to, like, learn about how large language models actually work, so that you can learn to reason about what the technology can and can’t do. It is not a magic trick, although it is remarkable.
So l- large language models are auto-regressive, meaning they learn from examples. That’s really all it is. People talk, “Oh, it’s just a token predictor.” But– And that’s true. It’s just a token predictor. It’s just trying to figure out what word comes next in the sentence. The fact that it evolved a, a world model is an incredible feat of engineering and, and pokes so many holes in what we usually th- we used to think about our theory of mind and what is intelligence.
Like, it raises huge philosophical questions. But put that all aside. Because it’s a token predictor, like, in– If you’re in a, if you’re notice if you’re in a chat with Claude or, or OpenAI or any of these tools, if it makes a mistake and you correct the mistake… So you, let’s say you guys say, “Oh, here’s the transcript.
Can you write me a summary?” It writes a summary and you’re like, “Oh God, that summary is terrible. It doesn’t include these important… Like, this is stupid. Like, please do it over again.” It does it again, and you say, “Oh, that’s even worse.” And you, you know, you go back and forth, right? You might notice, like I, I noticed that- chats with these tools, they either get better over time or they get worse over time.
Andrew Warner: Mm-hmm.
Eric Ries: You’ll just be like, “God, it’s like it’s getting dumber the more it talks to me.” And that has to do with the mechanics of the attention mechanism. As the context gets larger, you’re literally spreading the attention over more and more and more data. But also, it’s learning from all the bad examples it gave.
So it’s actually much more likely, even though you corrected it, the fact that the bad example is still there in the context makes it more likely to give you a bad example. So when we would take a, uh, transcript, we would say, “Look, give us a, a summary of the first section of the discussion,” or whatever, and it would write a paragraph.
And instead of saying, “No, that’s wrong,” this tool allows us to go into that box and change the AI’s output as if it got it right.
Andrew Warner: Ah, okay.
Eric Ries: So now the AI thinks that’s what it generated. Now it’s learned the style of how you want the rest of the blog post to go, and the next paragraph is far more likely to be what you want.
You do that two or three times, and now you can say, “Great, now please write up the rest of the blog post for me,” or like, “Write it to this outline,” or you know what? You could kinda go back and forth with it. So yeah, so that’s, that’s what we’re looking at. You can see here there’s, like, section headers and, um, text going on, and then the red boxes are the, the actual interactive bits with the, um, uh, with the, with the LLM.
And, and there’s a
Andrew Warner: video- Where in here are you writing? What’s that? Like, where’s the a- where’s the actual writing that you write or that it writes? Where’s the output?
Eric Ries: Yeah, this is a technical, um, this is, you know, this is being used for writing code, so the- Okay … the writing i- is not– I don’t think it’s being demoed in this one.
But yeah, my– A lot of the writing would either be in these boxes myself, like if I was just working on a small section. Interesting. Or I would keep a separate document with my markdown files per chapter, and if I was just– If I just wanna do raw writing, I would just go over there and blah, you know, barf a bunch of stuff on there.
Yeah. But once I had a complete first draft, that was very rare. Most often I would be like, even if I wanna rewrite a section, I would do it in this environment because it’s just so convenient once it’s there to be like, “Wait, is this rhetor-” Like, I often use a metaphor or a certain rhetorical move, and I’d just be like, “Is that effective?
Can you follow what I’m saying? Would, would my reader be able to understand it?” Like, do people know… Like, just like, I could just ask it these questions in the process of doing the writing. It was exceptionally helpful.
Andrew Warner: Can I see it? Like, can you log in and do screen share?
Eric Ries: It’s funny, ’cause we were just talking about, um- Uh, trustworthiness is an asset.
See this?
Andrew Warner: Oh, yeah. So there’s a section of the book as you wrote it, and so red means- Yeah, right … that the AI came up with the writing?
Eric Ries: No, no, no, no. This is, this is at the end. I’ll, I’ll show you, I’ll show you. This is just, like, this is the end of the process where it’s… This is a diff.
Andrew Warner: Okay.
Eric Ries: So this, the red are the parts that are being removed.
Andrew Warner: Ah,
Eric Ries: okay. Diff. So I had this section about, um… that was, like, trustworthiness research. It looked like it was a bit of a stub.
Andrew Warner: Mm-hmm.
Eric Ries: And I had written this section, which at the time was called Master Using It, and you can have this, um, for those that get the Zelda reference, although I think this is no longer in the final manuscript.
Andrew Warner: No.
Eric Ries: Um, and it’s like, here is, uh… Oh, yeah, it’s actually here, the metaphor I just gave you about the friend who’s a drug addict. Oh, that’s funny. This is, this is when I w- I wrote this section. And, um, it was riffing on the fact that, that, uh, Jim Sinegal, the founder of Costco, made this quote about raising prices as a form of heroin.
Once you do it and get away with it, you can’t stop. Anyway, so if you go back, you can see my, my dialogue with the AI. It goes on for a while. So for example, here I brought in some research called Trustworthiness as a Catalyst for Strategic Benefit. So this is a note which I can pull up for you. You can kinda see it here.
This is just, like, tons and tons and tons of research of different studies and information about the value of trust. Okay. I pulled in so much research for this book, you can’t even imagine it.
Andrew Warner: No, I felt
Eric Ries: it. Um, here’s a whole bunch of it. So here’s-
Andrew Warner: And so you’re, you’re saving the note just so the AI has more research in this step, okay?
Yeah,
Eric Ries: yeah. So I was like, “Okay, good. So give me this… Consider this section of research.” Mm-hmm. “What are some ways that we could use this specific research in this specific section?” And so here it’s making different ideas about what we could do. Um- Yeah.
Andrew Warner: What did you respond to it? To, uh… You said now
Eric Ries: show changes.
Well, so here you can’t… Yeah, you can’t actually see. Unfortunately, for historical reasons, this isn’t great because, because I… Remember I mentioned that when it does something wrong, I rewrite it?
Andrew Warner: Mm-hmm.
Eric Ries: Okay. So what happens is we brainstorm and go back and forth for a while. I then delete all the brainstorming and back and forth and collapse it down to a revised version that it thinks it wrote.
Andrew Warner: Okay.
Eric Ries: So it doesn’t know that we did… I erase all that from context because con- basically, when you’re using LLMs, context is everything. And remember, this is, this is old, Ri. This is old work. This is from last year when Solve It was very primitive and when context windows were crazy short. Now they’re a million tokens.
This is, uh, this is probably back when it was, like, 100,000 or 200,000 tokens. Um, anyway, so here you can see, like, we’ve now added these different stats that are from that research document. Um- And here’s some additional things. Anyway, so we go back and forth. We h- I get an evaluation from it right away. I love this part of it.
Oh. I find evaluating my own work extremely difficult as a writer because, of course, think about the MRI thing. It’s not that I want the L- it’s not that I trust the LLM’s evaluation to be correct. This is the hardest thing to understand. But just having something to react to when you’re alone trying to write and you’re feeling stuck is incredibly powerful.
Andrew Warner: And so you said to it, “Evaluate this new version. Is it better than the previous one?” It evaluated and said, “Here’s the strength. Research- Yeah … based credibility, added statistics, and research finding provide concrete evidence for claims that previously felt more anecdotal.” Okay, and it’s giving you all of this.
Where do you then take it and rewrite based on what it’s given you in the chat?
Eric Ries: Great, do it again. Let’s streamline this version for both length and tone. Da, da, da, da, da, da, da, da, da. Um-
Andrew Warner: And so then it comes back with an answer with, like, what the streamlined version looks like, and you then
Eric Ries: go
Andrew Warner: in and edit it Well,
Eric Ries: again,
Andrew Warner: every
Eric Ries: time you see a new version here- Uh-huh
you’re missing the fact that we went back and forth on it.
Andrew Warner: Got it. Meaning, like- So c- … you went into this and you edited. It came up
Eric Ries: with something- Yeah. Yeah, exactly … and then you- This is not… Yeah, exactly. This is a mix of, of its writing and mine as we went- Yep … back and forth and back and forth. Um, and let’s see.
Um, yeah, like it has… It, it is noting things that it thinks could be elsewhere in the chapter. It’s just like, it’s giving suggestions without telling me what to do, which is so valuable.
Andrew Warner: I see.
Eric Ries: Yeah, and then like, okay, then it’s funny, like I, I, I know where this all wound up. Like this, this giant section, a lot of this is not, is not in the book anymore, but like this quote from Fred that we added here, that was added for the first time in the manuscript right here.
This, this sentence is in the final manuscript. I know this ’cause I just read the audiobook. So, like this is extremely valuable, this quote that we, that we surfaced from the research. And like, it’s funny, like I know Fred’s w- work extremely well, but I didn’t know this quote. This quote is from a, like a paper of his, I think, that I had not read at the time.
So it’s like surfacing things for me that like I know a little bit about it, or I’m, I know adjacent to it. It’s like, look, no, you should quote him with this specific quote ’cause it’s just the perfect one for this paragraph. Those kinds of suggestions are super valuable.
Andrew Warner: And the difference with Solve It versus like if I’m using Claude or OpenAI with their artifact d-, uh, section, I would be editing on the right and asking questions on the left, and we might edit together on the right.
But here, it’s conversation back and forth- Yes … and the bad part just gets removed because you’ve edited the, the answer didn’t exist. Yeah,
Eric Ries: I always, I c- I prune the bad part out of the dialogues. With Cloud Code, with CoWork, these things, there’s a huge amount of work happening behind the scenes that you can’t see.
And like how many times have you been through like a compaction, co- context compaction in Cloud Code?
Andrew Warner: Yeah.
Eric Ries: You have no idea what it did, and then now all of a sudden it doesn’t remember what you told it to do anymore ’cause it’s decided what’s important. Here, the human always decides what’s important.
The human is in control of the context. So it’s slower, but it’s way better.
Andrew Warner: I love that you shared it. I love that you did the screen share.
Eric Ries: Yeah, no problem. No problem. So you can see, I hope people will, when they read the book, I hope they’ll be able to feel the level of care and effort that has gone into it.
Andrew Warner: So here’s what I liked about In- Incorruptible, the book. I feel like you’re on a mission. I f- and I can see it from even the Long Term Stock Exchange. I could totally see the mission from there through the book, the stories. And the mission is this: Hey, folks, stop caring about the quarterly earnings. Stop caring about the person who’s the loudest with a few shares in your company.
Think about what it is that you stand for, and I’m not even gonna take a position and tell you that you should be for or against anything. I’m just telling you, earn or feel, feel good about standing for something. And once you do that, here’s examples of all these companies that, that you’re proud of because they did that.
All right. Thank you so much for doing this. Thank you. It’s a great book. Oh, my pleasure. Well written
Eric Ries: Thank you very much. I feel bad now showing how the sausage is made, but for you, for you and your audience, I think, I hope people will appreciate it. Yeah.
Andrew Warner: They better. I do. Thanks. Hey, my agent says that if you watch this far, you’re gonna wanna subscribe, and Google thinks that if you watch this far, you’ll wanna watch that video
