Transcript Reader Lenny's Podcast
Library
Builder transcript 中文已完成

Why experts writing AI evals is creating the fastest-growing companies in history

> 当前版本已完成全部 chunk。

ChannelLenny's Podcast
Language中文
SourceYouTube
Coverage100%
0% 章节 01
Video Source Why experts writing AI evals is creating the fastest-growing companies in history

Lenny's Podcast

https://www.youtube.com/watch?v=ja6fWTDPQl4
Reading Mode

默认显示中文,缺失的章节会自动回退到英文原文,保证这页随时可读。

章节 01 / 07

第01节

中文 译稿已完成

> 当前版本已完成全部 chunk。

English Original transcript

Brendan FoodyThe wealthiest companies in the world are willing to spend whatever it takes to improve model capabilities.

Lenny RachitskyWe are entering the era of evals.

Brendan FoodyWe started working with all of the top AI labs. What the labs need is labor marketplace. They actually need extraordinary professionals that can measure model capabilities.

Lenny RachitskyThey found this pocket, maybe the biggest business opportunity in history.

Brendan FoodyWe grew from 1 to 400 million in revenue run rate in 16 months, fastest ascent in history.

Lenny RachitskyWhy is this so valuable?

Brendan FoodyThe market is bound by the amount of things where humans can do something that models can't. The lab's primary bottleneck to improve models is how they can effectively have some way of measuring what success looks like for the model.

Lenny RachitskyThere's this tweet that you retweeted. "If you really think about it, we were put on Earth to create reinforcement learning training data for labs."

Brendan FoodyIt's highly likely that the entire economy will become an aural environment machine, building out all of these worlds and contexts. And I think the narrative in AI over the last three years has almost entirely been one of job displacement, but very few companies and people have talked about this new category of jobs that's being created.

Lenny RachitskyI talked to a lot of people about what should I be studying? Where should I be getting better?

Brendan FoodyHow can they leverage this technology to do so much more? We'll give people interviews where we say, "Use whatever tools are available to build a website and let's see what product you're able to build in an hour."

Lenny RachitskyToday, my guest is Brendan Foody, CEO and co-founder of Mercor. Mercor is the fastest-growing company in history to go from 1 to $500 million in revenue. They did this in 17 months, less than a year and a half. Brendan is also the youngest unicorn founder ever. They just raised $100 million at $2 billion valuation. Mercor, if you haven't heard of them, helps AI labs and AI companies hire experts to help them train their models using AI. They've never had a customer churn, their net retention is over 1,600%, and they're on a nine-figure revenue run rate.

In our conversation, we talk about the increasing value and importance of evals, the landscape of AI training companies like Mercor, and why they've become so important and valuable, how Brendan discovered this opportunity, his insights on what product market fit looks like, the core tenets he's instilled within his organization that have allowed him to build the fastest growing company in history, what people writing evals for labs are actually doing day to day, which skills and jobs are going to last the longest with the rise of AI, why he doesn't think we'll see AGI or superintelligence anytime soon, and so much more. This episode is incredible. You need to hear this.
WorkOS also recently acquired Warrant, the fine fine-grained authorization service. Warrant's product is based on a groundbreaking authorization system called Zanzibar, which was originally designed for Google to power Google Docs and YouTube. This enables fast authorization checks at enormous scale while maintaining a flexible model that can be adapted to even the most complex use cases. If you're currently looking to build role-based access control or other enterprise features like single sign-on, SCIM, or user management, you should consider WorkOS. It's a drop-in replacement for auth zero and supports up to one million monthly active users for free. Check it out at workos.com to learn more. That's workos.com.
You fell in love with building products for a reason, but sometimes the day-to-day reality is a little different than you imagined. Instead of dreaming up big ideas, talking to customers, and crafting a strategy, you're drowning in spreadsheets and roadmap updates and you're spending your days basically putting out fires. A better way is possible.
Introducing Jira Product Discovery, the new prioritization and roadmapping tool built for product teams by Atlassian. With Jira Product Discovery, you can gather all your product ideas and insights in one place and prioritize confidently, finally replacing those endless spreadsheets. Create and share custom product roadmaps with any stakeholder in seconds. And it's all built on Jira, where your engineering team's already working so true collaboration is finally possible. Great products are built by great teams, not just engineers. Sales, support, leadership, even Greg from finance, anyone that you want can contribute ideas, feedback, and insights in Jira Product Discovery for free. No catch. And it's only $10 a month for you. Say goodbye to your spreadsheets and the never-ending alignment efforts. The old way of doing product management is over. Rediscover what's possible with Jira Product Discovery. Try it for free at atlassian.com/lenny. That's atlassian.com/lenny.
Brendan, thank you so much for being here. Welcome to the podcast.

Brendan FoodyThank you so much for having me, Lenny. I'm a huge fan, and so excited to have a conversation.

Lenny RachitskyI'm really excited to have this conversation as well. I'm a huge fan of yours. I'm excited for more people to learn about you and what you're building.

I want to start with a tweet that you have pinned at the top of your Twitter feed right now, and here's the tweet. "We are now working with six out of the Magnificent 7, all of the top five AI labs, most of the AI application layer companies. One trend is common across every customer. We are entering the era of evals."
The reason this caught my attention is that's one of the most recurring trends on this podcast, people talking about the increasing value of learning how to do evals well and the value of evals for companies. It feels like still people don't know what the hell this is what we're talking about, why this is so important. Talk about just what you think people are still missing, what they need to know, what this era of evals means.

Brendan FoodyIf the model is the product, then the eval is the product requirement document. And the way that researchers' day-to-day looks is that they'll run dozens of experiments where they'll make small improvements on an eval set. And reinforcement learning is becoming so effective that once they have an eval, they can help climb it. If you look at just how fast people were able to saturate Olympiad Math once they focused on it, how fast we're even saturating SWE-bench once we focus on it. And so in many ways, the barrier to applying agents the entire economy to automate every workflow is how do we measure success? How do we eval it? And write the PRDs for everything that we want agents to do, which Mercor is obviously a huge part of doing.

Lenny RachitskySo people hearing this, they're like, "Oh, yeah. Okay, shit. I got to really pay attention to this eval stuff." Any advice about learning how to do this well? What companies that are doing this well are doing differently? Help people get better at this thing.

Brendan FoodyYeah. I think that for enterprises especially, the core way to think about it is how can they build a test or systematic way to measure how well AI automates their core value chain? So if it's an architecture firm that's producing these architecture diagrams of what they provide to their end customer, how can they effectively measure that? And each company has its own value chain or maybe a handful of them if it's a multi-product company. And just thinking about how they measure that is the prerequisite to really effectively applying AI throughout their entire business.

Lenny RachitskyI saw you talking about this on the No Priors podcast with Sarah and Elad, and I don't know if it was after this or before this, but Sarah tweeted, "Evals equals your new marketing." What does that mean? What do you think she's saying there?

Brendan FoodyYeah. Well, it ties to what I said earlier about how if the model is the product, evals are the PRD, but also subsequently the sales collateral, right? Because evals are what you give to researchers to show them what they should be building and going on, but they're also the way that you demonstrate the efficacy of capabilities.

And historically, everyone's been pointing to these academic evals of PhD level reasoning with GPQA, Humanity's Last Exam, or Olympiad Math, but now it's moving towards the capabilities that people practically care about of how do we get models to automate the way that we build a software platform or automate the way that we do an investment banking analysis. And I think labs as well as application layer companies will increasingly use evals to demonstrate the capabilities of their models and their products.

Lenny RachitskyOkay. So let's build on this and zoom out a little bit and talk about the landscape of the market that you're in. And I was just reflecting on this as I was preparing for this conversation. If you think about the companies growing faster than any company's ever grown in history, there's essentially three buckets. There's the foundational model companies, there's vibe coding apps, Cursor and Loveable and Bolt and Replit and all these here, and then there's data labeling data companies like you. So I've had the CEO of Handshake on the podcast. I have the CEO of Scale coming on. There's also Surge. There's you guys. Help us just understand the landscape of what this is all about because I think people don't really know what the hell's going on and see all these companies growing like crazy.

Brendan FoodyYeah, I'll give a little bit of the origin story, incorporate in that and how it frames the landscape. Because when we started the company, I met my co-founders when we were 14 years old. We started the company together when we were 19 initially, in January 2023, initially hiring people internationally, matching them with our friends and automating all the processes of how we did that. So similar to how a human would review a resume, conduct an interview, and decided to hire. We automated all of those processes with LLMs, bootstrap the company to a million dollar revenue run rate before we dropped out of college.

And then a handful of other things happened, but we met OpenAI and we saw that there was this enormous transition in the human data market where it was moving away from this crowdsourcing problem of how do you find low and medium skilled people that can write barely grammatically correct sentences for early versions of LLMs and moving towards this sourcing and vetting problem. How do we source and assess the best professionals, the experienced? Think software engineers, the investment bankers and doctors and lawyers that can actually help to evaluate and interpret all of the capabilities that people want models to have.
So from there, we start working with all of the top AI labs. We grew from 1 to 400 million in revenue run rate in 16 months, and it's been an extraordinary journey and super exciting.

Lenny RachitskyOkay. First of all, that is out of control. I don't know if people understand. I think this is the first time you're sharing that number. I know we're recording this, you'll have announced it by now, but 1 to 400 million in revenue in 16 months.

Brendan FoodyExactly. So fastest ascent in history, which is an exciting statistic we're very proud of.

Lenny RachitskyOkay. So something big is happening here. Why is this so valuable? What is going on here? So it's just to try to summarize what you guys do simply is you help hire people for labs to help them train their models, and you help them find not just generalist labor, but experts, helping them with very specific gaps in the model's knowledge.

Brendan FoodyYeah, precisely. And so it really ties to your first question around the era of evals that's framing all of this, which is that the lab's primary bottleneck to being able to improve models is how they can effectively have some way of measuring what success looks like for the model, both to use it as the eval for the tests that they're measuring their progress against, as well as the verifiers in an RL environment to then reward the model, improve capabilities, et cetera. And they need this across every domain for every capability that models don't know how to use. And the wealthiest companies in the world are willing to spend whatever it takes to improve model capabilities where Mercor is sitting at the forefront and the primary bottleneck.

Lenny RachitskyOkay, what are these people actually doing? So what's an example of a kind of person that is sought after? And then what are they doing sitting there at the computer?

Brendan FoodyEffectively, the market is bound by the amount of things where humans can do something that models can't. So I'll make that very concrete. Say you have a model that you want to write a red line for a contract in the way that a lawyer would, and it makes a handful of mistakes, misses a bunch of key points in doing so. What you could do is have a lawyer create a rubric similar to how a professor might create a rubric to create a deliverable for what are the things we want the model to be able to do?

So it can effectively score that, right? Plus however much of it identifies this or XYZ key point. And that's really the foundation to measuring what does progress look like for models? Is this model achieving the capabilities that these professionals want? As well as how do we use this as training data to reward and to reinforce a lot of the capabilities that people want models to achieve.

Lenny RachitskyOkay, so they're essentially writing evals just to connect it back to original conversation.

Brendan FoodyExactly. Well, that's an interesting thing is everyone talks about RL environment. I feel like the two hot button things are like RL environments and evals, but one thing like Andrej Karpathy's tweeted out about a bunch is there's not actually a nuance. It's in the data type. It's more just a different semantic way of describing what it's being used for. But ultimately, it's just some stasis point for how do you measure what good looks like? And you can use that either as the benchmark to the sales collateral, as Sarah was saying, to say, here is why are models the best model in the world and here's the capabilities that we've been working towards, or you can use it on the post-training side to reward certain model trajectories and achieve those capabilities.

Lenny RachitskyOkay. So say this lawyer, this person is writing, "Here's what a great red line contract looks like and here's the rubric of what excellent is." Then are they also providing data, like actual examples of red line documents as a part of that?

Brendan FoodyThey may. The data landscape historically has included two kinds of data. The first is supervised fine-tuning data, which is input/output. When people think about fine-tuning in the historical sense, that's what it is. The second is RLHF where the model will generate a couple of examples. We'll choose which is the most popular example.

What everyone is generally moving towards is reinforcement learning from AI feedback instead of human feedback where you have instead the human defined some sort of success criteria, some way to measure that. And examples in code, it could be a unit test. We can scalably measure success and other domains that could be a rubric. And then you use that to incentivize model capabilities. And it's far more scalable and data-efficient, and so that's why a lot of the broader trend in the market across the board is moving towards RLHF to both eval models as well as improved capabilities.

Lenny RachitskyI had one of the co-founders of Anthropic on. He said exactly the same thing. That's what they've done at Anthropic, is move towards AI-driven reinforcement learning.

So essentially, if I can understand this correctly, I'm the lay person here trying to understand this on behalf of the audience. So essentially a lawyer is like, "Here's what correct looks like for redlining," and then it's AI is just on its own almost, just like, "I'm going to try to get this. I'm going to try to improve on this and I know if I'm heading the right direction based on this eval/rubric I've been given."

Brendan FoodyExactly. Applying all of the criteria of what good looks like similar to how the TA might apply the professor's criteria of does the student's response meet this criteria or this criteria plus however many planes, et cetera.

Lenny RachitskyAwesome, okay. Let me shift to talking about the broader labor market here. So there's two parts to this question as we talk about this. One is just how long will we need to do this? You guys grew so incredibly fast. Is there a point of like, "Okay, we don't need humans. We're tapped out." So let's start there and then I'll ask a broader question.

Brendan FoodySo the key question is how long there's going to be things in the economy that humans can do that AI can't do? And I think there's certainly a bucket of people that say we're going to have superintelligence within three years and humans won't play a role in the economy. And that's one school of thought.

Our perspective is very different. Our perspective is that these models are extraordinary and automating a lot of things very quickly, but there's a lot of things that they're horrible at. Even still, it can't schedule time on my calendar. It can't draft emails for me. It can't use basic tools. And we need evals for everything. For everything that the models can't do, we need evals for the tool use, evals for the long horizon reasoning.
Imagine in 10 years when we want models to be able to go out and build a startup for 30 days. We need evals for that to effectively reward it. And I think that that road to improving models will last for as long as there is anything in the economy that humans can do which models can't and be a huge portion of what the future of work looks like. And so our mission is creating the future of work, and I think that this is a really exciting industry and giving us a glimpse into the direction that everything is headed towards.

Lenny RachitskyThere's this tweet that you retweeted that I want to ask you about. "If you really think about it, we were put on Earth to create reinforcement learning training data for labs."

Brendan FoodyYeah.

Lenny RachitskyWhat does that mean to you? What is this person implying? And it's basically what you're saying is we're just helping train models.

Brendan FoodyIt speaks to conversations I've had with a lot of researchers and executives at top labs, which is that it's highly likely that the entire economy will become an aural environment machine, building out all of these worlds and contexts for us to then have rubrics or other kinds of verifiers. And that is really exciting in so many ways.

Because I think let's draw an analog to other revolutions where when we had the industrial revolution, everyone was freaking out about losing their jobs, but there was this whole new class of jobs of how do we build the machines? How do we have knowledge work? How do we create everything new? And I think that the narrative in AI over the last three years has almost entirely been one of job displacement, right? Sure, there's ChatGPT is growing fast and it's very cool that everyone loves using it, but from an economic standpoint, people talking a lot about job displacement. But very few companies and people have talked about this new category of jobs that's being created and what that's going to mean and how people can prepare and upskill for that. And I think that the most exciting thing possible is creating that future of how do humans fit into the economy and how will that evolve over time?

Lenny RachitskyI talk to a lot of people about just what should I be studying? Where should I be getting better? People in school right now are just like, "What is even going to be valuable in the future?" You're at the center of a lot of just what jobs are most in demand, how hiring is evolving. So let me just ask you a very concrete question. What jobs do you think will remain in the future/what skills are still worth investing in for younger people, especially?

Brendan FoodyIn terms of jobs, I would respond with a category of things that have very elastic demand are going to be super exciting. Because when we make people 10 times more productive, we'll build 10 times, if not 100 times as much software as an example. And so I think the product managers that can now do so much more are going to be extremely well-positioned. And so far as the skills, I think it's people that can leverage AI to do whatever their day-to-day workflows are.

I have had a couple conversations with teachers where they get my thoughts on how they should be assessing their students because we originally started out curating all of these AI interviews and assessments for people and have thought about this immensely. And what we realized is that you don't want to fight against them using the models. It's similar to when the calculator came out, you don't want to give people all of this arithmetic work of how do you get them to do it and not use the calculator. You want to tell them, "Use the tools and let's see what you can do."
And so we'll give people interviews where we say, "Use ChatGPT and Kodak. Use Claude code. Use whatever tool cursor and whatever tools are available to build a website and let's see what product you're able to build in an hour." And so I think that I give that an example in so far as talent assessment because I think it pertains also to the skills that people should be honing in on of how can they leverage this technology to do so much more in whatever industry or vertical they're operating in.

Lenny RachitskyWhen you talk about elastic, being elastic, is it generalists being good at just a bunch of different things, or what do you say? What do you mean when you think elastic?

Brendan FoodySo I more mean how much capacity for demand there is in that industry. So I'll give a couple of examples. In accounting, I think realistically we only need so much accounting in the world. Maybe there's areas where we can do more and that'll be good, but it doesn't feel like the world needs 100 times more accounting.

On the other hand, in software development, I think we can ship 100 times more features for our products, move 100 times faster, build so much more. There's just it feels like there's unlimited demand for the industry. And I think Mark Andreessen tweeted about this recently, that software is the most elastic industry of all where when we increase productivity, there's so much more that will be built. And it's definitely characteristic of a lot of other domains as well. And so I would focus on those domains where if we make everyone 10 times more productive, that'll increase demand, not reduce it.

Lenny RachitskyOkay. So you're in the bucket of learn to code, still useful as a skill. You take computer science. And so in terms of elastic categories of jobs, sounds like engineering, product management is in that bucket. Great. A lot of people listening to this are PMs. What else, like design users? I don't know. What else do you feel is in that bucket from what you've seen?

Brendan FoodyYeah, I think that there's a lot of things where the whole value chain of building companies has a lot of these variable costs, even large portions of operations or consulting. Imagine if we could have 10 times as many McKinsey consultants, what would be possible in so far as the research we could do, the analysis, et cetera. But I think the companies and people that are going to succeed are those that lean into this narrative of abundance of how do we do so much more rather than fighting back against it of how do we try to stop displacement.

Lenny RachitskySo along those lines, I think about your second bucket, which is the people that will be most successful. It's not like a specific skill, but it's being good with AI, using AI to become better at what you're already doing. This reminds me of Elon's whole thing with Neuralink, which I don't know if this is how we put it, but the way I've always heard it is you wanted to build Neuralink because in the future when AGI and superintelligence is around, we need a way to compete and the best way to compete is plug our brains into a superintelligence so we have a chance. And it feels like that's what AI is. Getting good at AI tools is essentially is having this super superpower.

Brendan FoodyFiguring out how to leverage them and incorporate it will definitely be of paramount importance.

Lenny RachitskyIt just comes back to this almost cliche quote now. It's, "AI won't replace you. People that are really good with AI will replace you."

Brendan FoodyI think it's totally spot on. And I've definitely seen this at the enterprise level as well where there's certain enterprises we talk to that are almost fearful not wanting to engage, not wanting to eval their businesses because that'll provide the evidence that their value chain is being automated. And there's others that... Literally some of the most recognized sophisticated Fortune 500 businesses that have this mentality and there's others that are leaning into it of if we have the ability to do 10 or 100 times more, what will that mean and how do we lean into that future? Because there's so many things that are going to change over the next 10 years, and I think those are the kinds of businesses that are going to be successful.

Lenny RachitskyLet's talk about labor markets more broadly. You guys, so it's interesting though. You started not feeding people to AI labs, not training models. It was just like help people find jobs, help companies hire, and then you're like, "Oh wow, this whole opportunity." You have this really interesting view on the future of just labor markets and hiring. Talk about that.

Brendan FoodyYeah, it's interesting. I remember when we started the company, as I mentioned, we were 19, and just had this gut intuition that it felt so wildly inefficient that labor markets are so disaggregated. And what I mean by that is when we would hire someone internationally, they would apply to a dozen jobs. When we as a company in the Bay Area were considering candidates, we would consider a fraction of a percent of candidates that were available in the market. And the reason for that is that there was this matching problem that everyone's solving manually where they'll manually review resumes, they'll manually conduct interviews, and manually decide who to hire. But when we're able to automate that matching problem at the cost of software, it makes way for this global unified labor market that every candidate applies to and every company hires from facilitating a perfect flow of information in the economy.

And I think that that future is undoubtedly what we're heading towards, but what we've realized over time is that the nature of work is also changing dramatically. And part of building that future over a 10-year time horizon is creating that future of work and all of the more tactical things we do and building these incredible data sets across evals and RL environments for our customers.

Lenny RachitskyWhat I've seen in how hiring has changed, I'm doing research on this with a partner, Gnome, it's so much easier to apply for companies that everyone's just applying now, to hundreds of companies. AI is just making it easy to adjust their resumes and cover letters and make it feel like, "Oh, I applied to more of course very specifically, but it was one of 100 places." And then on the flip side, hiring managers are getting flooded with applications and so now they need AI to filter. So even if we didn't want to get to this place, we're almost being pushed into this direction of so much volume on both sides. We need something really smart at filtering and helping us hire and select, and this is exactly what you guys have been building for a long time.

Brendan FoodyPrecisely, yeah. And the fascinating thing a lot of people ask, do we think about ourselves as a labor marketplace or do we think about ourselves as a data company? And I think that the reason it's an interesting question is our realization on from what the labs need is that they actually need a labor marketplace. They actually need these exceptionally high caliber people. And of course we'll layer on some project management and some software platform associated with it. But the really core thing that they want is how do they find these extraordinary professionals across all of these different domains that can measure model capabilities and work to build that future work together?

What makes Enterpret unique is its ability to build and update a customer-specific knowledge graph that provides the most granular and accurate categorization of all customer feedback and connects that customer feedback to critical metrics like revenue and CSAT. If modernizing your voice of customer program to a generational upgrade is a 2025 priority, customer-centric industry leaders like Canva, Notion, Perplexity, and Linear, reach out to the team at enterpret.com/lenny. That's E-N-T-E-R-P-R-E-T.com/lenny.
Going back to just how this all works and what you guys do for models, I was talking to a friend who had an ankle sprain or his foot was hurting and he got an x-ray and he fed the x-ray into ChatGPT and then asked him, "Give me this specific x-ray." And it's like, "Okay, sure." And then it gave him, "Here's what you have." And he was talking to me, he's like, "What is out there on the internet that trained this model to know this stuff?" And I was like, "No, it's actually somebody sitting there helping the model understand this. Once they recognize, it doesn't fully understand this. Humans are actually helping them learn these things."

Brendan FoodyExactly. Well, so the way it works, at least what most people's understanding is there's a lot of complexity in how the models work, is that pre-training gets a lot of the knowledge into the model of what are all the different things that see into the world. And then post-training and reinforcement learning is for all of the reasoning of what are the pieces of knowledge that are accurate, what are inaccurate, and what to prioritize at any given time to make a decision. And so behind that, there would've been radiologists that worked on the post-training data set to create some stasis point for here's the diagnosis and rewards and penalties associated with it. And it's really the quality of those people that went into the quality of the decision and recommendation that ChatGPT ultimately made.

Lenny RachitskySo let's actually follow that, right, because that's really interesting and I don't know how many people understand it. I understand it. So the work that you do and these experts do is post-training. It's not feeding data into the model that it's trained on. It's, "We have this model GPT-5. Now here's all the things that's missing. Let's add to it."

Brendan FoodyExactly, yeah. It's really unlocking, allowing the model to focus on all the right tokens, from pre-training all the right things in model context, up weighting the effective reasoning chains to enable the models to reason better in a more generalized way.

Lenny RachitskyWhat's the scale of people just working on the stuff. It's like thousands, tens of thousands, hundreds of thousands?

Brendan FoodyTens of thousands at any given time, hundreds of thousands more generally. It's huge. And the most exciting thing is that it's growing really quickly. I think that to your question also about the competitive landscape, historically there were all these crowdsourcing companies that would get these super high volumes of low-skilled people. I think Scale and Surge were the primary companies that pioneered that industry. And then in this transition to higher-skilled labor, what people realized is that actually you can go a lot further with just getting higher caliber people even in smaller amounts initially, and now subsequently scaling that back up once they're able to meet the quality bar.

And I think that there's a bunch of companies that after our success and very rapid revenue growth that started early last year have chased after that, which makes sense. And seeing that the market was changing very quickly, we were taking off, and trying to pursue a similar thesis on the market.

Lenny RachitskyIt's interesting. There's always been these companies, AlphaSights and GLG, that did this before AI or is paid to connect to an expert and ask them questions about stuff. And essentially, okay, it turns out this is really useful for models. We don't need the person in the middle.

Brendan FoodyExactly, yeah. Well, but one core difference is that AlphaSights would generally be a one-off call versus a lot of our work is really hiring people for projects of how do they work on something for a longer period of time. And so that's, I think, one of the reasons that some of the traditional expert networks have struggled to get into this. And also how do you retain those people and think about all the incentives where it actually looks more similar in some ways to one of the traditional labor marketplaces of an Uber or DoorDash, just with much higher-skilled talent that's treated exceptionally well?

Lenny RachitskyIt's such a good opportunity for me to learn so much about this, so I'm going to ask questions.

Brendan FoodyYeah.

Lenny RachitskyIt's so interesting to me. How much of the experts are focused on specific concrete knowledge versus personality and softer skills? How much of it's like, "Here's how you do an exam. Here's how you do an x-ray"?

Brendan FoodyIt depends on the lab. It's a lot of both. I think that previously it might've been more softer skills, but now a lot of the labs are focused on their business models of what are the economically valuable capabilities that drive revenue and leaning a lot into these professional domains. But I think the creative side is also still really important to everyone. And so we're seeing a meaningful amount of both. We hired all the people from the Harvard Lampoon a couple of months ago, their comedy club, to help with making models funnier. And so do all sorts of stuff like that, hiring Emmy award-winning screenwriters and everything across the board on creative capabilities that you'd look for.

Lenny RachitskyThat is amazing. What a cool story. I'm excited for this to kick in. How fast do these things turn around? Say you hired this team, how fast are we going to see the impact potential? Is like months? Is it years?

Brendan FoodyWell, so it depends, because some models or some labs will release iteratively where they'll just improve the model behind the scenes.

Lenny RachitskyWithout announcing a new model?

Brendan FoodyExactly. Every couple of weeks versus others do these big releases. And so it depends a lot. We're behind all of them, but we move really fast. It would be a customer gives us a request of we need these award-winning screenwriters, and within 24 hours we'll turn around the experts. And then there's also this really interesting dynamic where in a set of 100 people that we hire, oftentimes the top 10% of people will drive majority of the model improvement. It's like a company. If you have 100-person company, oftentimes the top 10% of the company will drive majority of the impact. And what that means is that when we're able to build proprietary advantages in identifying who are those top 10% of people, both in so far as how do we have them on our platform but also identify and match them effectively, it creates so much value for customers that it's difficult to compete against.

And so it really does tie back to the founding thesis of the company, which is how do we find these extraordinary people and identify them so that we can reliably deliver these top 10% or top 10X experiences for our customers.

Lenny RachitskySo on that, so is the idea, you hire Jane. She's incredible at coding and she now works for Anthropic and that's her full-time job doing this? Or is this a part-time thing? Is this a project thing mostly?

Brendan FoodyIt would sometimes be part-time. Sometimes it would be full-time. I would say most often it's part-time where it's like someone might work at a thing company where they're underemployed, maybe one of the ones that's moving slower where they have an extra 20 hours a week and then they're able to do this on the side or whatever the equivalent is across a bunch of different industries. But we also do a lot of 40 hour a week roles as well.

Lenny RachitskyAnd how much are they making? Is it meaningful enough for a AI engineer to spend time on this?

Brendan FoodyYeah, very meaningful. So our median pay rate in the marketplace is $95 an hour, but it can flex up, well up into $500 an hour based on the depth of someone's expertise. And one thing that highlights this difference relative to a lot of the crowdsourcing companies is if you look at the economics of the crowdsourcing companies, oftentimes they would pay $30 an hour to town as the average. And so think about the people that you can hire, the undergrads for $30 now versus the Goldman bankers, the McKinsey analysts, the Fang software engineers. And ultimately it comes down to what are the capabilities that labs want their models to have? And it much more falls in the latter bucket than the former one.

Lenny RachitskyI know there's only so much you can talk about with this stuff, but so Anthropic, Claude has been so good at coding so much better historically than other models. I also use it for writing, giving feedback on writing. What is it that allowed them to get so good at this and continue to be so good at this?

Brendan FoodyWell, I can't go too much into detail about customer work, but I think that it's this trend of reinforcement learning and being very thoughtful about defining the right rewards that we're releasing across the board. And how we could mitigate reward hacking, set up the right rewards, that's super impactful.

Lenny RachitskyEvals. Again, evals is all you need.

Brendan FoodyBack to evals.

Lenny RachitskyYeah.

Brendan FoodyOne of my favorite quotes from customers is that, "Models are only as good as their evals," which has always held true.

Lenny RachitskyI think Greg Brockman tweeted this once. "Evals are all you need."

Brendan FoodyYeah, truly.

Lenny RachitskyLet's talk about Mercor a little bit more. One of the maybe, not even maybe, I believe the data tells us it's the fastest growing company in history.

Brendan FoodyYeah.

Lenny RachitskyI want to understand what you did to make this happen. So let me just ask, what do you think are some of the core tenets of how you built Mercor that most contributed to being this successful?

Brendan FoodyI think the most important thing is looking at the leading indicators in fast-moving markets. I remember when I used to think... Everyone in venture talks about the why now, and I used to think about the why now of how from a product standpoint, less from a market standpoint of now we can automate the way that we review resumes or the way that we conduct interviews, et cetera. But ultimately there is this legacy market that's has all these incumbents and it's relatively stagnant. But what matters a ton is actually figuring out what are the new markets, the new pockets of demand that are changing very quickly where the wealthiest customers in the world are willing to pay whatever it takes to improve model capabilities, and how do we focus on the leading indicators of those markets to make sure that we have the best solution for the flagship customers in the market and optimize everything around that.

And that's what I found has been most impactful in building the business. I think maybe that's one thing is leading indicators in markets. If I had to choose another, it's customer obsession. We have had for the last... We're starting to have a couple of product managers help out with go-to-market, but for the last year and a half of the business, we've had no one in sales and marketing. And so we're immature from a sales and marketing standpoint because we focused 100% of company resources on how do we build great products and experiences for our customers. Just getting word of mouth, the people that have worked with us at other businesses want to keep working with us and leaning into creating those great experiences. And so that's where I spend all my time. And I think that some founders can get caught up in how do they get really good at marketing before they've figured out the thing that really drives a lot of customer love and creates the six-star experiences that you're used to building.

Lenny RachitskyI'm going to go back to that first point, which is like, okay, you found this pocket, maybe the biggest business opportunity in history. How did you first find... What was that moment of, "Wait, this could be really big"?

Brendan FoodySo there's some crazy stories here. I remember we started the company as I mentioned in January 2023. And then in August 2023 when I was still in college, one of our customers introduced us to the co-founders of xAI over a Zoom call saying how we had these really smart Indian software engineers that were great at math and coding. So we met them and we explained how the software engineers we had were really good at math and coding because they weren't distracted by all the humanities. They didn't have to study history and English and all these other things, and they loved it. So they had us in two days later to the Tesla office and we met the entire xAI co-founding team except for Elon, while I was still a college student. And xAI was just getting started at that point and they were super excited about our focus on the quality of the experts.

And so while they were still doing pre-training, they weren't ready for human data at the time and we didn't start working with them at that point. We just knew from that point forward before we even dropped out that the market was about to change radically and we needed to be at the frontier of that. And so then fast-forward a few months, one of the crowdsourcing players came to us and actually used our platform to hire over 1,000 people where this is very interesting experience because we started getting flooded with support tickets about how those people weren't getting paid. And we obviously felt horrible because we had referred them to this opportunity. It was this reputable company. And we realized that a lot of the incumbents were resting on their laurels with respect to what was needed in the experiences they were creating for talent in their marketplaces to help improve models. And there was this opportunity to work directly with the labs in a way that kept the dignity of the experts in the marketplace, paid them extremely well, and cut out the middlemen.
And so we started doing that in May of last year, and then the rest is history.

Lenny RachitskyWow, okay. Hundreds of millions of dollars in revenue since. So what I'm hearing here is you were very open to looking for poll. You saw some poll, you explored it. And then once you saw that there was something really meaningful there, you just went deep on making that an incredible experience as amazing as possible.

Brendan FoodyExactly. I think if I had to distill it into advice for founders, one thing I've realized is that I spent a lot of time trying forced product-market fit. And in some ways you should be persistent. You should have these theses that you have conviction about how the world will change. But sometimes you just need to sheer it from the market and know that it's there, the poll, to know the right places to focus. Because if it's difficult to sell, if it's extremely difficult to sell the marginal customer, you're not going to be able to grow a huge business. What you actually need to find is the customer that's surprisingly easy to sell into where you're going to be able to grow with them. You know that it's a large pain point. And so it's some combination of being stubborn with respect to your thesis around how the world will change, but also very open-minded with respect to exactly what form that takes and how the market's developing and how your company will fit into it.

Lenny RachitskyThat's an amazing insight. In the moments you described, felt like it was a combination of this xAI meeting feeling like, "Oh wow, they really, really want this thing that we have. We're now doing an amazing job," and then it's 1,000 people hiring in the platform. Was that those two moments that are like, "Wow"?

Brendan FoodyExactly. And those happened, keep in mind, while we were a seed company, right? Well, so the first one was before we even raised any seed funding, we were totally bootstrapped because we bootstrapped the company to a million dollar revenue run rate and have always remained super capital-efficient. We've never burned money. We were lifetime profitable. And then we raised our seed round in September from General Catalyst, and it was the other experience after we raised our seed round where we really knew that there was an enormous amount of demand in this market where we saw the volume and we saw that the incumbents were sleeping with respect to how the market was changing and the kinds of people that were needed to make that change happen.

Lenny RachitskyIt's one thing to see this opportunity and start to execute on it. It's another to actually succeed at this scale and consistently win. You guys have very specific values within the business. Talk about those. It feels like that's a big part of your success too.

Brendan FoodyIt totally is. So I'll give the three and maybe a brief story associated with each of them.

So the first one is having a can-do attitude, which everyone gives me a little bit of a hard time for because it's a funny saying, but we've always set these ridiculously ambitious goals, and then somehow the trajectory of the company forms around those goals. Where I remember when we were talking to Benchmark before they led our Series A, we were at 1.5 million in run rate. And I said we'd be at 50 million in run rate by the end of the year. And they said we were absolutely insane, right, as anyone would. And plus or minus two weeks, we hit it. And then we've now well blown past the tracking to 500 million in run rate, which was initially our goal for this year. So setting these incredibly ambitious goals with respect to the revenue scale of the business, the caliber of experiences for talent, all those dimensions is super important to first have a can-do attitude.
The second thing is really high standards, which is who we hire and what we expect of them. We have an incredibly high hiring bar where we hire tons of former founders, people that have incredible experiences. We just hired or partnered with Sundeep Jain who joined us as president. He was previously the chief product officer and chief technology officer at Uber and joined our relatively small in the grand scheme of things company to help scale up all the processes where Uber is of course the largest labor marketplace in the world. So super high standards is of paramount importance.
And then the third one that we really lean on significantly is intensity. And that if you look at the early cultures of the legendary companies, thinking of Meta or Google, they have these incredible, intense early-stage cultures of people just moving heaven and earth and doing whatever it takes to push the frontier of model capabilities. And so still very much output-oriented of what do people achieve rather than input-oriented of the specific hours they work, but recognizing that it takes a lot to build a legendary business, and that's ultimately what we're optimizing for.

Lenny RachitskyI could see why this works. Can-do attitude plus high standards plus intensity, I could see how that leads to success. There's a lot of talk these days about this 6-9-9 culture, working six days a week, 9:00 AM to 9:00 PM. A lot of people are like, "Why? That's terrible. Why would you make people do that?" But at the same time, I'm just constantly hearing this from the most successful AI companies. This is just the way it is to be successful. Things are moving so fast. This is an opportunity you'll never see again. Just talk about your thoughts on that.

Brendan FoodyYeah. Well, to clarify, we've never mandated hours. It's more been a byproduct of people that care a lot where we care a lot about the trajectory of the business. And so a lot of people come into the office and stay late. But if they need to leave early and get dinner with their kids or travel on the weekend, of course that's totally fine. And for us, it's much more about finding people who have a lot of ownership and are really bought in, less so about the specific hours in the office, even though we found that oftentimes it's the people that are most bought in, not always, but oftentimes it's the people that are most bought in and that burn the midnight oil with us.

Lenny RachitskyWhen you say high standards, is there something you could share that gives us an example of what you mean there? Because a lot of people think they have high standards and they don't.

Brendan FoodyIf you are very patient, there's always some trade-off between speed and quality when hiring. And I remember especially for our first 10 people, we were just so patient and disciplined about finding some of the best people in the world. Half of them are... Our second employee, Sid, as an example, our second employee in the US, Sid was previously the head of growth at Scale who joined us when we were a seed stage company. Daniel who joined us was previously scaled to consumer apps to over 100,000 users and all sorts of just extraordinary backgrounds of our first 10 hires. And I think that that initial talent density shaped so much of what the rest of the org looks like as you scale it up.

Lenny RachitskyI know you also have this perspective that people talk about waiting to hire, to hire really slowly, but it's actually not necessarily the right advice. Talk about that.

Brendan FoodyIt's painful because it's a double-edged sword. On one hand, I'm thrilled that our first 10 people are so phenomenal and I think that that has paid dividends for the business. But on the other hand, I think that companies do get to the point where you just need to hire really fast. And there's some things where you need a lot of people to do them and you need to recognize that there's going to be some variants associated with hiring, but moving quickly is the priority.

And I think that in some ways, we move too slowly with how we scaled out the team. And so the benefit is that everyone is extraordinary. We have this super high bar and we want to maintain that over time. But I think the downside is that while the company has grown incredibly quickly, we likely could have grown even faster if we had moved a little bit more quickly with especially ramping from call, like 10 to 100 people.

Lenny RachitskyOkay, I was going to ask. So it sounds like the first 10, be very careful, take your time, 10 to 100, maybe speed up a bit.

Brendan FoodyYes, though I wouldn't say it's necessarily 10. It's determined by the point where you know it's really working. And I know that's still not a bright line, but it's like once you know that there's so much more demand than you can handle, that's when you want to step on the gas and optimize for speed in a lot of ways. But I think especially until then, it's important to be patient, be disciplined. Get the best people is always important, but speed becomes more important once you find the market opportunity, the market vacuum.

Lenny RachitskyI know you've started a couple companies in the past, much smaller scale. In this new role as CEO of this massive hyper growth company, what surprised you most about where you spend the time most or just what the role involves? Because a lot of people want to start companies dream about being in your shoes. What are they maybe not understanding about where a lot of your time goes?

Brendan FoodyYeah, it's actually not too surprising. The top two buckets are always working on hiring and time with customers of how do I really deeply understand what customers need and how we can support them? And then how do I build the team and a lot of the processes around that? Of course, there's all of the ad hoc things I didn't expect of dealing with the people questions of how do we set up our levels and our comp bands and all of that, which you learn as you scale a business. But I think that the core places that I spend my time are in line with what I expected as well as what I love doing, which is very fortunate.

Lenny RachitskySo these two companies you've started in the past, maybe share what they work because they're fun, and then how do they help you be successful in this? What's something that they taught you that helped you in your current role?

Brendan FoodyYeah, so there's been like a dozen, but I'll choose my favorite two. So when I was in eighth grade, I started Donut Dynasty where I saw that Safeway Donuts were selling for $5 a dozen, and I was amazed because I felt like as an eighth grader, this was such an incredible deal. And I started to bike down to Safeway, buy Safeway Donuts for $5 a dozen, and then go back to my middle school and then sell them for $2 each, running really good margins of course. It sold out super quickly. And so then I need to scale up. So I would pay my mom $20 to drive me in her minivan down to Safeway, buy 10 dozen donuts, go to my middle school, sell them all out.

And then the school tried to shut me down because I was selling food on school campus, which they didn't like. So they had me in the principal's office asking me to not do that. And then I moved my donut stand over 50 feet, so it was off school campus, saying that they could no longer police me. I remember we had competitors pop up where the competitors were charging. They bought these Chuck's Donuts, which if anyone in the Bay Area knows, are higher end donuts than Safeway Donuts, but they have a higher cost basis. They cost a dollar per. And so I dropped my prices to $1 for two weeks to run them out of business before I knew what anti-competitive practices were. And I'd hire all my friends, paying my friends in donuts because they perceived the donuts as $2 each where they could sell them throughout the school and I could have a lower cost basis on them.
So I had all of these fun experiences in selling donuts, and then I could talk more about my high school business as well, which was a more significant scale. But I think the takeaway from that was just like you can just do things. So many people have ideas, but the barrier to more companies being built, I think, is just initiative and taking the steps to build the product or experience that customers want and investing the time and the ambition to scale that up. And so I think it was really getting reps of that that enabled me to realize that I should do it later on at a much larger scale.

Lenny RachitskyAmazing story. I love how wholesome that is versus drugs, selling donuts.

Brendan FoodyThen my mom was very worried. She was like, "Oh, is there any pot of these donuts?" I was like, "No, mom, I assure you these are pure donuts."

Lenny RachitskyI love that you paid your mom $20 to drive.

Brendan FoodyYeah. She was adamant it couldn't be a handout that she was taking her time to drive me, so she needed to make a little bit of money off of it. We haggled over her title where eventually she wanted to be head of global operations, which we found very entertaining.

Lenny RachitskyI hope that's on her LinkedIn.

Brendan FoodyNot yet. Maybe she'll have to add it.

Lenny RachitskySo you said that you've started a dozen companies?

Brendan FoodyYeah.

Lenny RachitskyWow. Okay.

Brendan FoodyWell, a dozen projects, but I think it was that, and then my AWS company were the two that I scaled up.

Lenny RachitskyWhat's the story behind Mercor as the name?

Brendan FoodyMercor means marketplace in Latin or to buy, sell, trade. And we want to build the largest marketplace in the world, the marketplace for how everyone finds jobs, and that was really the draw to it.

Lenny RachitskyOkay, maybe a last question. This is going back to earlier in discussion because it's something I've been thinking about as we're talking. There's been this shift from data as the fuel for models, and now it's experts. Do you think there's a next step, or is this just will take us to AGI, superintelligence?

Brendan FoodyI don't think it's necessarily changing from data to experts. It's more just the paradigm of realizing that labs need this close collaboration with experts to help understand what are the evals that they're building and how can they push the frontier. But I think it's very clear that evals are evergreen, that so long as we want to improve models, we'll need experts to create evals for them and to create the post-training data for them to learn those capabilities. And of course there might be changes in the exact way that people do training with RL or otherwise, but they will always need an eval to measure what does success look like across every domain that they want to build.

Lenny RachitskyOkay. So then building on that, a question that comes up a lot these days is, and I know we're talking about fun stuff but I'm getting to serious stuff again, scaling laws and just progression of model intelligence. A lot of people are feeling like, "I don't know, it's slowing down. We're not going to really get to superintelligence at this rate." What is your sense?

Brendan FoodyI totally agree with that. I know there's been some executives to big labs that say we'll have superintelligence in three years, but I think the truth is that it's a longer road. And that's not to diminish from how extraordinary the models are. I think we'll be able to automate a majority of knowledge work tasks in the next 10 years for sure, but that long road is paved with all of the evals that help to make those capabilities possible. And it's not going to be 10X more pre-training data that gets those capabilities. It's much more going to be all of the post-training data sets that are far more data-efficient and thoughtful that help us get there.

Lenny RachitskyDavid Sachs tweeted this interesting point that the situation we're now is almost the best case scenario where AI is not in this fast takeoff to superintelligence. There's a lot of competitors keeping each other in check. Models are already very valuable and only getting valuable, more valuable, but there's not just this winner superintelligence taking over the world situation.

Brendan FoodyYeah, I think that's true. I think a lot of the super intelligence fearmongering is probably overrated, but at the same time a lot of people's framing around that is even if there is a 5 to 10% chance of this P-Doom, then we should be careful, which seems logical. But I think that it's going to be an extraordinary 10 years for all of Silicon Valley and all of the world as this technology is able to create abundance and giving everyone better medical treatment, the best access to legal recommendations, and the ability to build great products more than we've ever seen before.

Lenny RachitskyAnd education feels like is transforming.

Brendan FoodyAbsolutely, right. I even have felt bits of this over the last 10 years where I remember ever... My parents would give me a hard time for not going to classes in college and I'd be like, well, there's way better lectures on YouTube. Why not just listen there? But I can only imagine as the models get extremely good at conveying information, better than the best professor, what that'll mean and access to all sorts of information to better forward humanity and upskill everyone.

Lenny RachitskySo I'll use that as a segue to a final question. I'm going to take us to AI Corner, which is a recurring segment on the podcast. What's some way that you personally use AI to do better work to help you in life?

Brendan FoodyWell, let's see. I use it a lot to write documents, as you would expect. I also talk to get advice on problems. I find it helpful to just reason through almost as a thought partner because, yeah, I don't know. I find I think better sometimes when I'm talking something through, but I can't talk through everything with colleagues or people around me.

Lenny RachitskyAnd so this is like ChatGPT Voice Mode mostly or something else.

Brendan FoodyYeah, I like ChatGPT Voice Mode a lot. There's stuff-

Lenny RachitskyMe too.

Brendan Foody... or room for improvement, but I am very excited about the future of Voice.

Lenny RachitskyLet me show you something I built, actually. I wasn't planning to talk about this, but there's this guy, Eric Antonow, who's been recommended by a lot of people to get him on this podcast. He's this creative product person that's under the radar now. He's at Facebook for a long time. He built this project called Pirate GPT, which is you basically put ChatGPT into a stuffed animal to talk to it. So built a little wise owl. I don't have it on right now.

Brendan FoodyWow.

Lenny RachitskyBut basically you sew in a little speaker right here and you put a little magnet underneath and you can put it on your shoulder and then you just talk to it.

Brendan FoodyThat's so cute. Wow. I love it. I'll have to get one of those. Because I have some of the voice assistants in my apartment, but I really want a ChatGPT voice assistant, so I'm excited for-

Lenny RachitskyI was just thinking that. Yeah, just come on. Why can't we have a ChatGPT voice just sitting around listening to us all the time. And you can't on your phone because it goes to sleep and it's like, "Hello, what?"

Brendan FoodyExactly. Yeah.

Lenny RachitskyYeah, so it's what this is trying to be. Well, there's a kickstarter he started that we'll link to that. You could help out.

Brendan FoodyThere we go.

Lenny RachitskyThat's really easy.

Brendan, is there anything else that you wanted to share or touch on or maybe leave listeners with before we get to a very exciting lighting round?

Brendan FoodyTying to the point around initiative and that you can just do things, I encourage everyone, especially with AI and it being so much easier to build, just take the initiative to go out and build products and talk with customers and take that leap of faith because I think that that is in so many ways, the largest barrier to more innovation, the economy in any way that we can support that.

Lenny RachitskyYeah. There's so many people that just, let's not bash the podcast, but just listen to podcasts, read posts, just keep reading and listening and don't do anything with that information. And there's never been an easier time to actually build stuff and try stuff.

Brendan FoodyTotally.

Lenny RachitskySo definitely take that advice. Just you can do things. You can move your donut stand 50 feet and get out of their jurisdiction.

Brendan FoodyYeah.

Lenny RachitskyOkay, Brendan, with that, we've reached a very exciting lightning round. I've got five questions for you. Are you ready?

Brendan FoodyAll set.

Lenny RachitskyWhat are two or three books that you find yourself recommending most to other people?

Brendan FoodyLet's see. I would say in order, High Output Management is a phenomenal book on running companies. Second is Zero to One, which of course is a classic. And then third is Shoe Dog, where I just find it to be a really inspirational story.

Lenny RachitskyWhat is a recent movie or TV show you really enjoyed?

Brendan FoodyI really liked Oppenheimer. My favorite TV show of all time is Suits, so I know not recent, but if I had to choose a recent one, probably Oppenheimer.

Lenny RachitskyVery cool. Suits, first time someone's mentioned that. Favorite product you recently discovered that you really love?

Brendan FoodyI love using Codex, like the new version. I know it's sort of new in terms of version. Yeah, I think it's incredible and just a huge, huge improvement. So yeah.

Lenny RachitskyDo you have a life motto that you find yourself coming back to, sharing with folks, finding useful in work or in life?

Brendan FoodyI think it's you can just do stuff, what we were talking about earlier. Take the leap of faith.

Lenny RachitskyI thought you were going to say can do, which is in your Twitter profile.

Brendan FoodyCan do as well, yeah.

Lenny RachitskyTwo great ones. Final question. So we were chatting before this about things that we could talk about and you shared this interesting thing that you haven't shared anywhere else, which is that you're dyslexic. Why don't you share that with folks? And just how do you get around that having built the fastest-growing company in history?

Brendan FoodyI don't hide it at all. I think a lot of my colleagues know. And I think on one hand it definitely makes it difficult to go through 1,000 emails a day or read every document that I'm supposed to, but on the other hand, I feel like it helps me to think a little bit differently, to be more creative, and perhaps see that markets are changing that not everyone sees. And so it's turned out okay so far. And so I think one thing it's helped me realize from a management standpoint is that we focus much more on how we can leverage people's strengths rather than helping to improve weaknesses, because there's some things that I'm not great at and I'll never be the best in the world at, and there's others that I can hopefully refine and strive to be.

Lenny RachitskyThat's such a also recurring theme on this podcast of just focusing on strengths and not focusing over all your focus on weaknesses.

Brendan, this was incredible. I learned so much. I have a billion more questions, but you got shit to do. Two final questions. What should people know about what you're doing and roles you're hiring for? And then how can listeners be useful to you?

Brendan FoodyAbsolutely. We're hiring a ton across the board on our team. We're hiring strategic project leads on our operations team, software engineers in our engineering team, as well as researchers. And so please go to mercor.com and we would love to work with you, and that's the largest way that you can help us. Share it with your friends as well. Over half of people in our marketplace come from referrals because we have a platform of people that love us. And so any jobs that you want to apply to or send your friends to, we would love to have you.

Lenny RachitskyBrendan, thank you so much for joining me.

Brendan FoodyThank you for having me.

Lenny RachitskyBye, everyone.

Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.

章节 02 / 07

第02节

中文 译稿已完成

### 中文译文

Julian Shapiro为什么好点子总要等坏点子耗光以后才冒出来?因为你一旦经历了一堆坏点子,大脑就会本能地识别出,到底是哪些元素在制造“坏”。这样你会越来越擅长避开这些坑,也会越来越能靠直觉把新点子和这些模式对应起来。很多创作者其实都在抗拒自己的坏点子。你如果坐下来,只是在空白文档里随便写了几句,然后因为没立刻闪出金光就走人,那你根本还没完成创作过程,也就不可能真的写出金子。

Lenny欢迎来到 Lenny's Podcast。我是 Lenny,这里主要是想帮你提升做产品、做增长的手艺。我会采访世界级的产品领导者和增长专家,听他们讲一路摸爬滚打后,怎么把公司做大做强。今天的嘉宾是 Julian Shapiro。我在开场已经认真介绍过他了,所以这里我只说我们今天会聊什么。我们会聊他所谓的 product-led acquisition,也就是产品驱动获客,这是他和几千家公司打交道、帮他们梳理增长策略时总结出来的一套方法。
接着我们会聊怎么提高产品留存,再往下会聊很多写作相关的东西,比如写作里为什么“新意”这么重要,写之前怎么选题,Julian 还提出了一个叫 Creativity Faucet 的框架。Julian 是个特别有意思的人,我很期待把这期节目带给大家。

Julian Shapiro这是我人生中最光荣的一件事,谢谢你。

Lenny哇!

Julian Shapiro听完你这个介绍,我都快哭了,太好了。

Lenny这就是效果。对我来说这也是人生中最光荣的一件事,我们扯平了。

Julian Shapiro太好了。那我们就彼此抵消一下,看看这样会不会更有意思。

Lenny没错。话题很多。我知道你在 Twitter 上大概有 25 万粉丝,你在 Twitter 上很强,但我注意到你这一年只发了三条推。怎么回事?

Julian Shapiro有几个原因同时在发生。第一,很多人都在写 threads,我发现这东西真的很尬。那种“这里有 21 个方法帮你重做创业公司”之类的鸡汤串,尬得要命。它们的实际效果,是把那些觉得这类信息有价值的人吸引过来,同时把你真正想维系关系的人劝退。我记得 thread 刚兴起那会儿,我自己在试,也看到有人开始取关我,因为那些内容惹恼了我很在意交流的那批人。
慢慢地,我就没了写那类东西的动力和热情。现在我只在写一些东西时顺手发推,基本上是我网站文章的反映版或者浓缩版。我知道它质量高、是原创的、是经过思考的,不是为了博眼球。
粉丝质量这件事其实可以分成两种:一类人是因为你的脑子关注你,另一类人则是把你当成一个“高级内容搬运工”。如果他们是因为你在做搬运工式的劳动而关注你,那我叫他们 labor followers。他们追的是你帮他们找有趣梗图、发冷笑话、写鸡汤 thread 这些劳动成果。相反,如果他们是因为你的脑子关注你,那就是第一类,也就是他们关注的是你的原创想法、洞见和你对世界的看法。像 Y Combinator 的 Paul Graham 就是在输出原创观点,他不是为了涨粉去写 thread,而是为了写出有趣的新想法。这样一来,粉丝对他和他的大脑的认同感就更强了,因为他们会想:“哇,这个观点太原创、太有意思了。”
他们关注的是你的脑子,不是你在网上整理一个虚拟 BuzzFeed 的劳动。这样的 mind followers 认同感更高,忠诚度也更高,会更认真看你说什么。要是你真想让他们支持你,比如办线下活动、卖东西,或者推动某个你在乎的事情,他们会更愿意买单。
如果他们只是因为你的劳动关注你,那你就会和一堆 meme 账号没什么区别,他们也不会对你这个人本身有真实认同。我更在意粉丝质量,而不是数量。

Lenny这个提醒太好了,不能只盯着 follower、follower、follower。我很好奇,如果一个人刚开始玩 Twitter,你会给什么建议?

Julian Shapiro而如果你只发一句聪明话,别人会想:“哦,这可能只是碰巧。谁知道这人是不是稳定能产出聪明东西。” 但如果一个 thread 有 30 条推,而且每条都好,大家就会想:“哇,这人是个机器。只要持续关注他,就会一直拿到好内容。” 这会重新强化别人应该关注你的判断,这也是 threads 更容易带来关注的原因。说白了,你确实应该发 threads,老实讲,这就是核心。再加上一个足够吸睛的开头,这事就成了。

Lenny很棒。我本来没想到会聊到 Twitter 策略,但这确实有意思,因为你真的很擅长这件事。也像你说的,你的内容是很认真思考过的,不是那种为了粉丝和转发而硬凹的 thread。谢谢你分享这些。

Julian Shapiro这事其实一开始也是那样做的,因为我和几个朋友算是最早大规模玩 threads 的人之一。后来我们意识到它最后会变成什么样,就停了。

Lenny我懂你的意思,那些尬 thread 真的很烦。总之我想做的,是与其问一堆散乱的问题,不如聚焦五个大主题,顺着这五个主题深入聊。可以吗?

Julian Shapiro当然,我很愿意。

Lenny好。先说点背景。你写了很多很深入的手册,内容涵盖增长、写作、健身等等。首先你能不能解释一下,这些手册到底是什么,为什么你要做这些?

Julian Shapiro这些手册对我来说,就是一种强迫机制,让我在学东西时对自己负责,同时逼自己学得足够彻底。比如我想系统学增长、学写作,或者学别的什么主题,我就会去做大量研究,把能找到的东西全读一遍,再做很多实验,尽量形成一些你从别人研究里不容易直接看到的新洞见。然后下一步,就是尽量把这些东西压缩得足够简洁、足够可执行,好让我自己以后查阅。
等这些工作做完后,通常只需要再花个几十个小时,把它打磨成公众也能看懂、也愿意看得下去的版本。既然我已经为自己做了这么多工作,为什么不顺手公开呢?到了这一步,它其实也就变成了获客素材,能帮我建立受众,把我的观点传播得更远。
我最骄傲的一点是,这些手册虽然谈不上彻底独一无二,可每一本里都有很多原创内容,是普通人之前没怎么听过的。我最有成就感的,就是把那些夹在字里行间的洞见提炼出来,让原本看起来很复杂的事情变得特别容易理解。对我来说,如果我能把一个大家通常会觉得“超级复杂”的东西,变成大家都能顺着看懂的内容,我就算成功了。
没几个人会去翻自己邮箱里那篇超长、超级难找的健身指南邮件。第一,这是一个 UX 选择。第二,我能拿到 SEO 流量。第三,这其实是一个可以持续更新的活资产。它不会死在别人邮箱里,也不会被打印成纸。也许我和很多线上写作者不太一样的地方在于,我花在回头重写旧文章和旧手册上的时间,和写新东西差不多。
如果你一年或一年半之后再回来读我写过的任何东西,它都会被更新,因为我把所有写作都看成 evergreen 内容。我尽量不写那种一过就没的东西,比如追热点、写新闻。那类我几乎都避开,我更想写能长期有用的东西。

Lenny这点我之前还真不知道,挺酷的。我很喜欢你这样做。你最好把这事写清楚一点。很有意思,这并不是“过时内容”,而是“上次更新还是上周”。你现在可能已经这么做了吧?

Julian Shapiro没有,我其实还没这么标,可能我确实应该这么做。有人也跟我抱怨过,说我没写更新时间,所以我也许哪天会加上。

Lenny好吧,至少我们今天有了一个好主意。

Julian Shapiro那就行。

Lenny好。今天第一个想聊的点,是你叫做 product-led acquisition 的概念。我觉得这是你任何一本手册里最受欢迎的页面之一。这个概念来自你和几千家公司一起工作、帮他们梳理增长策略的经验。我很好奇,这个概念到底是什么,大家怎么拿它来帮产品增长?

Julian Shapiro对很多人来说,这个词更常被叫做 product-led growth,但我觉得这个词有点名不副实。它经常被拿来泛指 SaaS 公司用自助式销售漏斗,不需要销售人员介入的那种增长方式,也就是绕开销售,让产品自己长大。这样说也没错。但从增长营销的角度,我们真正关心的是 product-led acquisition,也就是“产品的使用本身会推动产品增长”。
比如我在用 PayPal 给别人转 1000 美元,那对方为了收这 1000 美元,几乎不可能不去注册一个 PayPal 账号。因为我在日常场景里使用 PayPal,借它来结清一笔债务,而我在得到价值的同时,也自动、而且非常强烈地在诱使对方成为 PayPal 用户。这就是 product-led acquisition。
我识别出了几类不同的情况。如果你的公司能靠 product-led acquisition 增长,那当然不是所有公司都行,毕竟有些企业级产品只能靠销售,但它就是远远最好的增长方式,因为用户邀请用户的边际成本接近于零。它很可扩展,而且通常会形成网络图,带来复利效应,既能造出护城河,也能更快获取更多客户。说白了,它就是 viral。
另一个很有意思的点是,它的外部依赖少得多。假设你的公司主要靠内容和 SEO 增长,那你就得看 Google 算法更新的脸色。比如一年两次的算法更新,偶尔会把你的流量直接打穿。经历过的人都知道那种感觉,太糟了。或者如果你是靠付费获客长大的公司,比如靠 Facebook Ads、Instagram Ads,那你同样得看 CPM 波动和平台各种奇怪更新、删掉某些定向选项的脸色。你的整个获客策略都压在一个完全不受你控制、而且波动很大的东西上。
快速背景就是,我们前面举过一些例子。我很喜欢 Paul Graham 说过的一句话:不要去做那种你得先经过别人才能拿到用户的创业公司。这里的核心是,第一类 PLA 就是用户帮你清结债务。
比如我通过 Venmo 给你付我欠你的晚餐钱,或者我用 PayPal 付供应商款,只要我必须借助某个产品完成支付,对方为了把该得的钱拿到手,几乎都会去注册那个产品。这个就是最典型的 user-led growth / product-led growth。
第二类,是你邀请别人加入你正在使用的产品,好参与一段否则他们没法接触的对话。为什么 Telegram、WhatsApp、iMessage、Discord 这些聊天应用能长得这么快?原因很简单:如果你和一小群朋友的聊天都在这个 app 里,那现实生活里也在你朋友圈中的某个人,只要还没装这个 app,就必须装上它,才能跟你继续聊天。Slack 也是同样的逻辑。
如果我是做产品的人,在做 roadmap 时,我会想:我的产品里有没有什么东西,能支撑前面说的那两种功能之一,也就是结清债务,或者在产品里引入聊天?如果有,那你可能就打开了一个非常惊人的增长渠道。顺便说一下,很多人会把 product-led acquisition 和 referral program 混为一谈,但它们其实不是一回事。
Referral program 本质上是在产品外面加一层奖励,试图把用户往外推;而 PLA 是你在自然使用产品的过程中,邀请别人本身就能让你获得更多价值。它不需要人工奖励来推。
第三类我叫做 billboarding,也就是产品的使用本身对周围的人可见,产品自己给自己打广告。最经典的例子是 Hotmail 和 iPhone。你用 Hotmail 发邮件,结尾会自动带上 “sent via Hotmail”;iPhone 也会把自己变成一块免费的广告牌。
还有一种更直白:你在现实世界里做出一个一眼就能认出来的东西。Tesla、Nike、AirPods 这些产品,别人一眼就能看到,所以它们本身就是移动的广告牌。现在 Twitter 上最典型的例子,就是头像换成某个 NFT 系列,或者 Bitcoin 的 laser eyes,本质上都是在给某个东西打广告。
Telegram 现在也有点类似的玩法。你是高级用户,名字旁边会多一个小星星,所有人都能看到:“哦,这是什么?这是 Telegram Pro。” 再比如 Calendly,你要创建日程,就必须把自己的 Calendly 链接分享出去。Dropbox、GoFundMe 也一样,分享本身就是曝光。
最后一类是 UGC,也就是 user generated content。用户在 YouTube、TikTok、Instagram 上发内容的时候,平台本身也会嵌进去一起曝光。或者像 Quora、Reddit、Stack Overflow、TripAdvisor 这种产品,你鼓励用户生成公开可索引的内容,这些内容还会被 Google 收录,帮你吃到搜索流量。说白了,PLA 的骨架就是这些:你不花钱,也能扩得很快,而且不会过度依赖第三方流量。

English No English text found
No English transcript text was found for this chapter.
章节 03 / 07

第03节

中文 译稿已完成

### 中文译文

Julian Shapiro真正的教训是,如果你作为创始人,根本不知道公司以后会怎么增长,那就别去创业。当然,这不适用于所有公司。深科技、生物科技、气候科技这些方向另当别论。可对很多打算做 SaaS、而且想在 B 端或 B/C 两端快速增长的人来说,我真正想表达的是:如果你眼前有三个点子,其中有一个特别适合 product-led acquisition,那就可以优先选它,尤其是当你判断“增长能力”会决定谁赢谁输的时候。
这样做能让你轻松很多。因为如果我们太依赖 SEO 和内容,而这两样都极度拥挤;或者依赖付费获客,也就是广告渠道,而这些渠道同样拥挤,尤其当你的 LTV 很低时,我们根本扛不住付费 CAC 的波动,甚至连这些 CAC 的成本本身都很难接受,那你就必须从产品层面更战略性地思考。它不是简单地以后再补一个功能,有时候当然会自然长出来,但如果它足够有机,效果会非常好。
另一个常见误区,是很多人把 product-led acquisition 和 referral program 混为一谈,但它们不是一回事。Referral program 是在产品外面加一层奖励,试图把用户往外推;而 PLA 是你在自然使用产品的过程中,邀请别人本身就能让你获得更多价值。比如你结清债务时,收款人需要注册;比如聊天里 Jack 也进来了,你的对话就更有价值。这里不需要用人工奖励去刺激。
我对 referral program 一般没什么兴趣,因为它往往会筛出那些只想要奖励的人。然后他们邀请的对象也可能只是想拿双向奖励,根本不是为了 app 本身,拿完就走。之后他们通常也不会再拉其他人。它没有 PLA 那种复利式、粘性强、留存好的特性。当然,如果你能把它做成,那也很棒。
第三类我叫做 billboarding,也就是产品的使用本身对周围的人可见。产品自己给自己打广告。这个词的来源,其实是我在旧金山看高速公路上的 billboard 时想到的:我看到那些真正挂 billboard 的公司,会在 billboard 上既展示自己卖的广告,也顺手把自己的 logo 一起打出去。他们是在拿自己的展示面给自己打广告。
最经典的例子就是 Hotmail 和 iPhone。当你通过 Hotmail 发邮件时,结尾会自动附上一句 “sent via Hotmail”。iPhone 也是一样,几乎每一封从 iPhone 发出去的邮件,除非你手动删掉签名,都会变成一块免费广告牌,给 Apple 自己打广告。
如果你的应用里有发邮件、对外发短信、给供应商开 invoice 这些功能,那你就可以在签名里写上“由某某创业公司提供”,把自己的展示面用起来,增加品牌曝光。
还有一种更直白:在现实世界里做出一个一眼就能认出来的东西。Tesla、Nike、AirPods 这些东西,周围的人一看就知道,所以它们本身就是移动的广告牌。现在 Twitter 上最典型的热点例子,就是你把头像换成某个 NFT 系列时,你就在给那个 NFT 系列打广告。Bitcoin 的 laser eyes 也是同理。
Telegram 现在也有点类似的东西。你是付费高级用户,名字旁边会多一个小星星,所有人都能看到:“哎,这是什么?哦,这是 Telegram Pro。那我也去看看。” 还有一个我很喜欢的例子,就是 Calendly。你要用它,就必须把自己的链接分享给别人。Dropbox、GoFundMe 也是一样,分享本身就是曝光。
最后一类是 UGC,也就是 user generated content。你在 YouTube、TikTok、Instagram 上发内容的时候,平台本身也会嵌进去一起曝光。或者像 Quora、Reddit、Stack Overflow、TripAdvisor 这种产品,你鼓励用户生成公开可索引的内容,这些内容还会被 Google 收录,帮你吃到搜索流量。

Lenny我最近也在研究招聘市场的变化。现在大家都在疯狂投简历,AI 又让人很容易改简历、改求职信,好像投了很多公司。另一边,招聘方又被海量申请淹没,所以现在也需要 AI 来筛选。就算我们不想走到这一步,现实也在把我们推向这边。两边的量都太大了,所以必须有很聪明的筛选和招聘系统,而这正是你们一直在做的事。

Brendan Foody没错,这正是我们看到的情况。很多人会问,我们到底把自己看作一个 labor marketplace,还是一个数据公司?这个问题有意思的地方在于,我们后来意识到,labs 真正需要的是 labor marketplace。他们需要的其实是那些极高水平的专业人士。当然,我们会在上面叠加一些项目管理和软件平台,但最核心的东西是:怎么在各个领域里找到这些卓越专业人士,去衡量模型能力,并一起搭建未来的工作方式。

English No English text found
No English transcript text was found for this chapter.
章节 04 / 07

第04节

中文 译稿已完成

### 中文译文

Brendan Foody是的,不过我澄清一下,我们从来没有硬性规定过工作时长。更准确地说,很多人因为真的很在意业务走向,所以会自发投入很多时间,常常在办公室待到很晚。但如果有人需要早点回去陪孩子吃饭,或者周末要出门,那当然完全没问题。对我们来说,更重要的是找到有 ownership、真正投入进来的人,而不是盯着他们具体在办公室待了多少小时。虽然我们确实发现,通常最投入的人会陪我们一起熬到深夜。

Lenny Rachitsky你说高标准,这里能不能举个例子?很多人都说自己标准高,但其实并不是。

Brendan Foody如果你足够有耐心,招人这件事里永远都是速度和质量的权衡。我记得最开始那 10 个人,我们就是非常耐心、非常克制地去找世界上最优秀的人。前 10 个里有一半都特别强。比如我们的第二位美国员工 Sid,原来是 Scale 的增长负责人;还有 Daniel,之前把消费级应用做到了 10 万以上用户。前 10 位加入的人背景都非常夸张。我觉得正是那批最早的高密度人才,塑造了后来组织的底色。

Lenny Rachitsky你也提过,大家总说招聘要慢一点、晚一点再招,但这未必是对的。讲讲这个。

Brendan Foody这件事确实很纠结,因为它是双刃剑。一方面,我很高兴前 10 个人都这么强,这对业务确实带来了很大的回报。但另一方面,公司做到某个阶段后,确实必须开始更快地招人。有些事情就是需要很多人来做,你也得接受招聘里会有一定波动,但更重要的是速度。
我甚至觉得,我们在团队扩张上有点太慢了。好处是,大家都非常优秀,我们也把标准守得很高;但坏处是,虽然公司增长已经很快了,我们其实本可以在从 10 人涨到 100 人的阶段更快一点,整体跑得更猛。

Lenny Rachitsky所以听起来像是,前 10 个人要非常谨慎,慢慢来;到了 10 到 100,可能就要加速了。

Brendan Foody对,但我不会说“10 人”是绝对分界线。真正的分界点,是你已经确认它真的跑通了、而且需求远远超过供给的那一刻。那时你就该踩油门,把速度放在第一位。只是到了那个阶段之前,耐心和纪律特别重要。高质量的人永远重要,但一旦找到市场机会、找到那个需求真空,速度就变得更重要。

Lenny Rachitsky你之前也创业过几次,规模都小很多。现在作为这个超高速增长公司的 CEO,最让你意外的是什么?大家都很想坐在你这个位置,但可能不太知道你大部分时间到底花在哪。

Brendan Foody其实没那么意外。我的前两大时间桶永远是招人,以及和客户待在一起,深入理解客户到底需要什么、我们怎么支持他们。然后就是组团队,以及围绕团队建立各种流程。当然,随着公司变大,会有很多我没预料到的杂事,比如层级、薪酬带宽这些管理问题,但我花时间的核心区域,其实和我预期的一样,也正是我喜欢做的事,这点很幸运。

Lenny Rachitsky你之前创过的那几家公司也挺有意思的,能不能讲两家最有代表性的?它们怎么帮你在今天成功?

Brendan Foody好啊。我就挑两个我最喜欢的。第一个是我八年级时做的 Donut Dynasty。我发现 Safeway 的甜甜圈一打卖 5 美元,我当时觉得这简直是八年级学生能碰上的神仙生意。于是我就骑车去 Safeway 进货,买回来后在学校里一个个卖,两美元一个,毛利非常漂亮。很快就卖光了,所以我得扩大规模。后来我会给我妈 20 美元,让她开面包车送我去 Safeway 进 10 打甜甜圈,再运回学校卖掉。
后来学校不让我在校园里卖食物,就把我叫到校长办公室,要求我别这么干。我就把摊位往外挪了 50 英尺,站到校外,告诉他们现在管不到我了。我还记得那时竞争对手也冒出来了,他们卖的是 Chuck's Donuts,湾区的人都知道那是比 Safeway 更高级的甜甜圈,但成本更高,一只要一美元。我就把价格降到两周 1 美元一只,直接把对方打出局,那会儿我还不知道这叫反竞争行为。
我还雇了所有朋友,拿甜甜圈当工资付给他们,因为在他们眼里一个甜甜圈值两美元,他们可以在学校里继续转卖。我就这样把成本压得更低。那段经历特别有趣,后来高中我还做过一个更大规模的生意,不过这已经足够说明问题了。最大的收获其实很简单:你就是可以去做。很多人有想法,但真正阻碍更多公司出现的,往往不是创意,而是主动性,是你愿不愿意动手去做产品、去做客户想要的体验,并愿意花时间、花野心把它做大。

Lenny Rachitsky这个故事太棒了。我喜欢它比卖毒品健康多了,就是卖甜甜圈。

Brendan Foody后来我妈还很担心,她问我:“你这些甜甜圈里不会有什么乱七八糟的东西吧?” 我说:“妈,我保证,纯甜甜圈。”

Lenny Rachitsky我喜欢你还给你妈付 20 美元让她来接送。

Brendan Foody对啊。她坚持说不能白帮忙,因为她花时间开车过来,所以得赚一点。我们还为了她的头衔争论过,最后她特别想叫自己 Global Operations 负责人,我们觉得很好笑。

Lenny Rachitsky我希望这已经写进她的 LinkedIn 了。

Brendan Foody还没,也许她之后会自己加。

Lenny Rachitsky你说你做过十几家公司?

Brendan Foody对。

Lenny Rachitsky哇。

Brendan Foody更准确地说,是十几个项目,但真正做大的是那个和 AWS 相关的公司,以及这个。

Lenny Rachitsky那 Mercor 这个名字是怎么来的?

Brendan FoodyMercor 在拉丁语里就是 marketplace,也有买卖、交易的意思。我们想做全球最大的 marketplace,也就是人人找工作的 marketplace,这就是我们很喜欢这个名字的原因。

Lenny Rachitsky好。最后一个问题,还是回到前面我们聊过的那件事。现在的变化是从“数据是模型燃料”变成“专家是模型燃料”。你觉得下一步会是什么?还是说这一路会把我们带到 AGI、超级智能?

Brendan Foody我不觉得这是从数据转向专家,更准确地说,是一种新的范式:labs 需要和专家更紧密地协作,去搞清楚该做什么 eval,以及怎么把边界往前推。我觉得非常明确的一点是,eval 会是 evergreen 的。只要我们还想继续提升模型,就永远需要专家来做 eval,也需要他们来做后训练数据,让模型学会这些能力。当然,训练方式可能会变,比如 RL 或者别的什么,但无论如何,想知道“成功长什么样”,每个领域都需要 eval。

English No English text found
No English transcript text was found for this chapter.
章节 05 / 07

第05节

中文 译稿已完成

### 中文译文

Lenny Rachitsky现在大家很常问的一个问题是,模型智能的演进是不是在放缓。我知道我们刚才聊了很多轻松的东西,但我想回到这个严肃问题。很多人觉得,可能没那么快到超级智能了。你的感觉是什么?

Brendan Foody我基本同意这种判断。虽然确实有些大模型实验室的高管会说三年内就会有超级智能,但我觉得现实是,这条路会更长。不过这并不意味着模型不强大。相反,我认为未来十年里,我们肯定能自动化掉大部分知识工作任务,但这条长路是由一整套 eval 铺出来的。真正推动能力跃迁的,不会是再多 10 倍预训练数据,而是更数据高效、更深思熟虑的后训练数据集。

Lenny RachitskyDavid Sachs 发过一个很有意思的说法:我们现在其实处于一个还不错的状态,因为 AI 并没有那种快速起飞、直接冲向超级智能的局面。很多竞争者在彼此制衡,模型已经很有价值,而且还在持续变得更有价值,但又不是一个赢家通吃、超级智能突然接管世界的场景。

Brendan Foody对,我觉得这说法是对的。很多超级智能恐慌其实被夸大了,但与此同时,很多人强调的那点也有道理:哪怕只有 5% 到 10% 的概率会出现那种灾难性结果,我们也应该保持谨慎,这逻辑并不奇怪。不过从更大的图景看,这十年会非常精彩。硅谷和全世界都会因为这项技术而进入更高的丰裕状态,得到更好的医疗服务、更好的法律建议,也能比以往更容易做出优秀产品。

Lenny Rachitsky教育这块感觉也会被彻底改变。

Brendan Foody当然。其实过去十年里,我自己就已经感受到一些变化了。我记得父母以前总因为我大学不去上课而说我,但我会想:网上明明有更好的讲座,为什么不直接看 YouTube?而我只能想象,当模型变得特别擅长传达知识,甚至比最好的教授还强的时候,这会意味着什么,也意味着大家获取信息、提升能力的方式会发生怎样的变化。

Lenny Rachitsky那我顺着这个问题,最后问一个。节目里有个固定环节叫 AI Corner。你个人平时怎么用 AI,让工作或生活做得更好?

Brendan Foody嗯,我会用它写文档,这个不意外。我也会把它当成一个可以讨论问题的对象。我觉得它很适合拿来做思维推演,像一个思考伙伴一样帮我把问题捋清楚。因为有时候我确实在和 AI 聊的时候,想得更清楚,但又不是每件事都能跟同事或者身边的人反复聊。

Lenny Rachitsky你主要是用 ChatGPT 的语音模式吗,还是别的?

Brendan Foody对,我很喜欢 ChatGPT Voice Mode,虽然还有进步空间,但我对语音的未来非常兴奋。

Lenny RachitskyBrendan,在进入很激动人心的快问快答之前,你还有什么想补充的,或者想留给听众的话吗?

Brendan Foody回到“你就是可以去做”这件事,我想鼓励每个人,尤其是在 AI 让构建东西变得这么容易的时候,直接动手去做产品,去和客户聊,去迈出那一步。我觉得这在很多层面上都是最大的障碍之一。凡是我们能支持创新的方式,我都很支持。

Lenny Rachitsky是啊。太多人只是听播客、看文章,不会把信息变成行动。而现在从来没有这么容易过,真的可以去构建、去试。

Brendan Foody完全同意。

Lenny Rachitsky所以这建议真的很值得听。就是去做。你甚至可以把甜甜圈摊往外挪 50 英尺,离开他们的管辖范围。

Brendan Foody对。

Lenny Rachitsky好了 Brendan,我们进入了一个非常激动人心的快问快答环节。我有 5 个问题,你准备好了吗?

Brendan Foody准备好了。

Lenny Rachitsky你最常推荐给别人的两三本书是什么?

Brendan Foody我会按顺序说:High Output Management 是一本非常棒的公司管理书。第二本是 Zero to One,当然是经典。第三本是 Shoe Dog,我觉得它特别鼓舞人。

Lenny Rachitsky最近看过什么电影或电视剧,让你特别喜欢?

Brendan Foody我很喜欢 Oppenheimer。我最喜欢的电视剧是 Suits,虽然不是最近的,但如果一定要选最近看的,那大概还是 Oppenheimer。

Lenny Rachitsky很酷,这是我第一次听人提 Suits。最近发现的、你特别喜欢的产品是什么?

Brendan Foody我很喜欢用 Codex,新版特别棒。虽然它刚出没多久,但真的进步非常大。

Lenny Rachitsky你有没有一句人生座右铭,会反复想起、也会常常拿去和别人分享?

Brendan Foody我觉得就是“你可以直接去做”。也就是我们刚才说的,迈出那一步。

Lenny Rachitsky我本来以为你会说 can do,因为这就在你 Twitter 主页上。

Brendan Foodycan do 也算,没错。

Lenny Rachitsky两个都很好。最后一个问题。我们开始前聊到,你说过一个你从没在别处说过的事:你有读写障碍。你愿意和大家分享一下吗?以及,你怎么在这种情况下还把公司做成了历史上增长最快的那种?

Brendan Foody我并不避讳这件事。其实很多同事都知道。一方面,它确实让我每天要处理 1000 封邮件、看很多文档时比较吃力;但另一方面,我觉得它也让我想得更不一样,更有创造力,也更容易看到别人没看到的市场变化。所以到目前为止,这反而是正向的。

Lenny Rachitsky这也是节目里经常出现的主题:把精力放在优势上,而不是把所有注意力都放在弱点上。
Brendan,这期太精彩了。我学到了很多,还有一大堆问题想继续问,但你还有事要忙。最后两个问题:大家最该知道你们在做什么、正在招什么人?听众又怎么才能帮到你?

Brendan Foody当然。我们现在各条线都在大量招人。我们在运营团队招 strategic project leads,在工程团队招 software engineers,也在招 researchers。所以欢迎大家去 mercor.com 看看,我们很希望能和你一起工作,这也是你最能帮到我们的方式。也欢迎转给朋友。我们市场里超过一半的人都是靠推荐来的,因为我们已经有很多很喜欢我们的用户了。所以不管是想投递的岗位,还是想转给朋友的岗位,我们都很欢迎。

Lenny RachitskyBrendan,非常感谢你今天来。

Brendan Foody谢谢邀请。

Lenny Rachitsky再见,大家。

English No English text found
No English transcript text was found for this chapter.
章节 06 / 07

第06节

中文 译稿已完成

### 中文译文
(本集结尾无新增正文,以上已完成全篇中文译稿。)

Lenny Rachitsky我希望这已经写进她的 LinkedIn 了。

Brendan Foody还没,也许她之后会自己加。

Lenny Rachitsky你说你做过十几家公司?

Brendan Foody对。

Lenny Rachitsky哇。

Brendan Foody更准确地说,是十几个项目,但真正做大的是那个和 AWS 相关的公司,以及这个。

Lenny Rachitsky那 Mercor 这个名字是怎么来的?

Brendan FoodyMercor 在拉丁语里就是 marketplace,也有买卖、交易的意思。我们想做全球最大的 marketplace,也就是人人找工作的 marketplace,这就是我们很喜欢这个名字的原因。

Lenny Rachitsky好。最后一个问题,还是回到前面我们聊过的那件事。现在的变化是从“数据是模型燃料”变成“专家是模型燃料”。你觉得下一步会是什么?还是说这一路会把我们带到 AGI、超级智能?

Brendan Foody我不觉得这是从数据转向专家,更准确地说,是一种新的范式:labs 需要和专家更紧密地协作,去搞清楚该做什么 eval,以及怎么把边界往前推。我觉得非常明确的一点是,eval 会是 evergreen 的。只要我们还想继续提升模型,就永远需要专家来做 eval,也需要他们来做后训练数据,让模型学会这些能力。当然,训练方式可能会变,比如 RL 或者别的什么,但无论如何,想知道“成功长什么样”,每个领域都需要 eval。

English No English text found
No English transcript text was found for this chapter.
章节 07 / 07

第07节

中文 译稿已完成

### 中文译文

Lenny Rachitsky现在大家很常问的一个问题是,模型智能的演进是不是在放缓。我知道我们刚才聊了很多轻松的东西,但我想回到这个严肃问题。很多人觉得,可能没那么快到超级智能了。你的感觉是什么?

Brendan Foody我基本同意这种判断。虽然确实有些大模型实验室的高管会说三年内就会有超级智能,但我觉得现实是,这条路会更长。不过这并不意味着模型不强大。相反,我认为未来十年里,我们肯定能自动化掉大部分知识工作任务,但这条长路是由一整套 eval 铺出来的。真正推动能力跃迁的,不会是再多 10 倍预训练数据,而是更数据高效、更深思熟虑的后训练数据集。

Lenny RachitskyDavid Sachs 发过一个很有意思的说法:我们现在其实处于一个还不错的状态,因为 AI 并没有那种快速起飞、直接冲向超级智能的局面。很多竞争者在彼此制衡,模型已经很有价值,而且还在持续变得更有价值,但又不是一个赢家通吃、超级智能突然接管世界的场景。

Brendan Foody对,我觉得这说法是对的。很多超级智能恐慌其实被夸大了,但与此同时,很多人强调的那点也有道理:哪怕只有 5% 到 10% 的概率会出现那种灾难性结果,我们也应该保持谨慎,这逻辑并不奇怪。不过从更大的图景看,这十年会非常精彩。硅谷和全世界都会因为这项技术而进入更高的丰裕状态,得到更好的医疗服务、更好的法律建议,也能比以往更容易做出优秀产品。

Lenny Rachitsky教育这块感觉也会被彻底改变。

Brendan Foody当然。其实过去十年里,我自己就已经感受到一些变化了。我记得父母以前总因为我大学不去上课而说我,但我会想:网上明明有更好的讲座,为什么不直接看 YouTube?而我只能想象,当模型变得特别擅长传达知识,甚至比最好的教授还强的时候,这会意味着什么,也意味着大家获取信息、提升能力的方式会发生怎样的变化。

Lenny Rachitsky那我顺着这个问题,最后问一个。节目里有个固定环节叫 AI Corner。你个人平时怎么用 AI,让工作或生活做得更好?

Brendan Foody嗯,我会用它写文档,这个不意外。我也会把它当成一个可以讨论问题的对象。我觉得它很适合拿来做思维推演,像一个思考伙伴一样帮我把问题捋清楚。因为有时候我确实在和 AI 聊的时候,想得更清楚,但又不是每件事都能跟同事或者身边的人反复聊。

Lenny Rachitsky你主要是用 ChatGPT 的语音模式吗,还是别的?

Brendan Foody对,我很喜欢 ChatGPT Voice Mode,虽然还有进步空间,但我对语音的未来非常兴奋。

Lenny RachitskyBrendan,在进入很激动人心的快问快答之前,你还有什么想补充的,或者想留给听众的话吗?

Brendan Foody回到“你就是可以去做”这件事,我想鼓励每个人,尤其是在 AI 让构建东西变得这么容易的时候,直接动手去做产品,去和客户聊,去迈出那一步。我觉得这在很多层面上都是最大的障碍之一。凡是我们能支持创新的方式,我都很支持。

Lenny Rachitsky是啊。太多人只是听播客、看文章,不会把信息变成行动。而现在从来没有这么容易过,真的可以去构建、去试。

Brendan Foody完全同意。

Lenny Rachitsky所以这建议真的很值得听。就是去做。你甚至可以把甜甜圈摊往外挪 50 英尺,离开他们的管辖范围。

Brendan Foody对。

Lenny Rachitsky好了 Brendan,我们进入了一个非常激动人心的快问快答环节。我有 5 个问题,你准备好了吗?

Brendan Foody准备好了。

Lenny Rachitsky你最常推荐给别人的两三本书是什么?

Brendan Foody我会按顺序说:High Output Management 是一本非常棒的公司管理书。第二本是 Zero to One,当然是经典。第三本是 Shoe Dog,我觉得它特别鼓舞人。

Lenny Rachitsky最近看过什么电影或电视剧,让你特别喜欢?

Brendan Foody我很喜欢 Oppenheimer。我最喜欢的电视剧是 Suits,虽然不是最近的,但如果一定要选最近看的,那大概还是 Oppenheimer。

Lenny Rachitsky很酷,这是我第一次听人提 Suits。最近发现的、你特别喜欢的产品是什么?

Brendan Foody我很喜欢用 Codex,新版特别棒。虽然它刚出没多久,但真的进步非常大。

Lenny Rachitsky你有没有一句人生座右铭,会反复想起、也会常常拿去和别人分享?

Brendan Foody我觉得就是“你可以直接去做”。也就是我们刚才说的,迈出那一步。

Lenny Rachitsky我本来以为你会说 can do,因为这就在你 Twitter 主页上。

Brendan Foodycan do 也算,没错。

Lenny Rachitsky两个都很好。最后一个问题。我们开始前聊到,你说过一个你从没在别处说过的事:你有读写障碍。你愿意和大家分享一下吗?以及,你怎么在这种情况下还把公司做成了历史上增长最快的那种?

Brendan Foody我并不避讳这件事。其实很多同事都知道。一方面,它确实让我每天要处理 1000 封邮件、看很多文档时比较吃力;但另一方面,我觉得它也让我想得更不一样,更有创造力,也更容易看到别人没看到的市场变化。所以到目前为止,这反而是正向的。

Lenny Rachitsky这也是节目里经常出现的主题:把精力放在优势上,而不是把所有注意力都放在弱点上。
Brendan,这期太精彩了。我学到了很多,还有一大堆问题想继续问,但你还有事要忙。最后两个问题:大家最该知道你们在做什么、正在招什么人?听众又怎么才能帮到你?

Brendan Foody当然。我们现在各条线都在大量招人。我们在运营团队招 strategic project leads,在工程团队招 software engineers,也在招 researchers。所以欢迎大家去 mercor.com 看看,我们很希望能和你一起工作,这也是你最能帮到我们的方式。也欢迎转给朋友。我们市场里超过一半的人都是靠推荐来的,因为我们已经有很多很喜欢我们的用户了。所以不管是想投递的岗位,还是想转给朋友的岗位,我们都很欢迎。

Lenny RachitskyBrendan,非常感谢你今天来。

Brendan Foody谢谢邀请。

Lenny Rachitsky再见,大家。

English No English text found
No English transcript text was found for this chapter.