HBAP | STARTUPS - Jeff Bussgang - "The Experimentation Machine"

Todd Brouse:

Good evening, everyone! Thank you for joining us. I’m Todd Brouse, and I’ll be your host tonight, alongside Susan Schultz and Ashley Wilson. Welcome to HBAP Startups.

Let’s quickly walk through tonight’s agenda: We’ll begin with some announcements, followed by brief networking in breakout rooms, and then we’ll return for our featured presentation.

Our Mission:

HBAP Startups exists to support innovation and entrepreneurship within the Harvard Business Analytics Program community. We aim to equip our members with the tools, insights, and support to launch and scale their ventures with confidence.

Susan Schultz:

There are many ways to stay connected with us:

• Our website

• YouTube channel (where recordings are available when speakers allow)

• Email newsletter

• Slack channel (look for the entrepreneurship thread if you're in HBAP)

Feel free to bring a guest, and if you or someone you know would like to present, let us know—we’re always looking for great speakers.

Reminders:

• Sessions are recorded and publicly posted unless a speaker requests otherwise.

• We follow the HBAP honor code and the Francis Frei feedback model, which means focusing on actionable, constructive feedback.

• Please don’t record sessions independently.

• If you want to collaborate or pitch an idea, reach out to me, Susan, or Ashley.

Upcoming Events:

• Next week: Mike Salguero, CEO of ButcherBox, will speak about the claims-based meat movement and building a mission-driven company.

• March 26: I’ll be presenting part two of our series on agentic AI.

• April 16: Our own Nick will explore Oratory Mastery with an Analytics Edge.

• We’re also planning a hands-on workshop on customer discovery in March or April based on your feedback.

Now, let’s head into breakout rooms for a few minutes of networking—we’ll regroup shortly.

[After Breakouts]

Welcome back, everyone! Hope the networking was valuable. Before we begin, a quick favor—when you leave Zoom tonight, you’ll be redirected to a feedback survey. Please take a moment to fill it out. As data folks, we live on feedback.

This event is being recorded and will be posted to our YouTube channel. We’ll also hang around after the formal session for overtime Q&A and conversation.

Introducing Tonight’s Speaker:

It’s my pleasure to introduce tonight’s speaker, Professor Jeff Bussgang.

Jeff is a Senior Lecturer at Harvard Business School and General Partner at Flybridge Capital, an early-stage VC firm with offices in Boston and NYC, managing over $1B in assets. He teaches Launching Technology Ventures at HBS—an AI-forward course for aspiring entrepreneurs—and has authored three influential books:

• Entering StartUpLand

• Mastering the VC Game

• His newest release: The Experimentation Machine—a guide to building a startup in the age of AI.

Before venture capital, Jeff co-founded Upromise (acquired by Sallie Mae) and served on the exec team at Open Market, which IPO’d in the 90s. He’s also a leading voice on equity and social impact, serving on boards like Hack. Diversity and Facing History and Ourselves.

Thanks to Kelly Kimball’s sponsorship, everyone attending tonight will receive a digital copy of The Experimentation Machine. We’ll drop a Google Form link in the chat later so you can claim yours.

Now, as is our custom, if everyone could briefly unmute and help us give a warm round of applause to Professor Jeff Bussgang!

Jeff:

We were the Series A investor in MongoDB, and one of the defining theses that led to that investment was this question we used to ask ourselves:

What if we had a thousand times more information at our fingertips than we do today? What could we do differently? What would the world look like?

Now, the question is:

What if you had a million people working for you — the smartest people in the world — with all the knowledge of physics, engineering, science, chemistry, computer science, AI — and they were working for you for free? What could you achieve?

That’s the moment I believe we’re coming into.

The final thing I’ll say — and this also comes from Leopold’s white paper — is that as superintelligence kicks into gear, we’re going to see incredible inventions in science, breakthroughs in robotics. I was recently at Boston Dynamics, and what they’re doing to merge general intelligence with their incredible robots is extraordinary. And of course, we’re going to see significant developments in military tech — companies like Anduril and Shield AI are doing impressive work, following in Palantir’s footsteps in defense tech.

That’s the essence of what’s in the book. It’s written for founders, but really it’s for everyone.

Before I open it up for questions, and Todd, I’ll let you facilitate — I want to leave you with a few questions:

1. On a scale of 1 to 10, how AI-native do you feel today?

2. How can you harness AI’s power to improve that score? AI can be your tutor. It can help you learn more about AI itself. The book can help, sure, but more importantly, it’s about seizing your knowledge and using these tools to accelerate your learning.

3. How can you develop the skills to become a builder, an opportunity creator, and a 10x leader?

I tell my MBA students: This is the most unique moment in history I’ve ever seen, where to be a creator, you no longer need a tech co-founder, an engineering degree, or a PhD in machine learning.

When we were investing in AI and machine learning companies 15 years ago, we required at least one PhD in machine learning on the founding team. Today, my students are using tools like Cursor or V0 and building rapidly.

Before I open up for your questions, Todd, I want to acknowledge that Scott Ford is in the room. Scott is the founder of SparkRockets — a phenomenal tool that allows founders to generate Lean Canvases with just a few prompts.

Scott introduced me to the tool and generously tutored me on it. I even incorporated it into an assignment for my students. I had them use ChatGPT to ideate startup ideas, design experiments to validate them, and then use SparkRockets to build Lean Canvases to structure their hypotheses.

There are incredible tools being built — even in this community — to help founders go from zero to one faster.

I’ll stop there. I know that was a lot. Todd, let’s open it up for questions.

Todd:

That was fantastic, Jeff. Would you believe it if I told you that Scott presented here at Startups a year ago?

Jeff:

Very cool!

Scott:

Thanks for the shoutout, Jeff. I met Jeff back in November on campus — he's been an amazing supporter. He even just sent me an email yesterday with a bunch of feedback from his students — both positive and critical — and that’s helped us make the product better.

Susan Shultz:

Scott, I need to catch up with you! I use SparkRockets in my own class as well.

Todd:

Alright, folks, please raise your Zoom hands if you have questions. Jeff, I have so many myself. Let me kick it off:

Where do you see the future of work heading? We’re all entrepreneurs — looking forward, if these technologies can augment or even replace the workforce... is there any work left in the future?

Jeff:

Great question. It’s a hot debate right now among labor economists. You’ve probably heard the two extreme views:

1. Techno-optimist view: Every time we’ve had a major technological revolution — from the steam engine to the information age — society has adapted. New jobs are created with every increase in productivity. So yes, work will be transformed, disrupted, but there will be different jobs, often better and safer, and higher-paying.

2. Techno-disruption view: That may have been true in the past, when we had more time to adapt. But this time, the rate of change is too fast. As I showed you in the technology adoption curve, these changes are outpacing our ability to adapt. We might be at peak employment now. From here, we could see a world that needs fewer jobs and fewer workers.

I was listening to a podcast recently — I won’t name names, they’re a bit controversial — but a few VCs argued that tightening immigration policy in the U.S. was justified, because with AI creating so much slack in the workforce, we may not need as many workers.

So... maybe we’re heading into a post-labor economy.

Mo Gawdat has a great quote:

So, if I’m a founder or CEO, and I can hire an AI agent to do the job faster and cheaper, isn’t it my fiduciary responsibility to my shareholders to do that?

Jeff (continued):

My opening line in the book is:

We’re in a hyper-competitive world. If you don’t use these tools, your competitor will.

Craig made a great comment earlier about the “three-day work week.” Hundreds of years ago, economists predicted that increased productivity would lead to shorter workweeks. But… most of us don’t work 20-hour weeks today, do we?

Maybe this is finally the moment where we bank those gains.

Let me give you an example:

My son, who’s a CS minor in college, took a summer programming job. He used ChatGPT to help port an app to mobile and modernize its infrastructure. He worked about 20 hours a week, spent the rest of the time road-tripping with his girlfriend, hiking, rafting… and he still delivered on the job.

That might be the future for a lot of our kids. And that’s… probably okay. If they get the job done and use AI to do it more efficiently, that’s a win.

Todd:

That’s awesome. One last question from me before we go to the audience:

Do you see an advantage for startups vs incumbents in this new AI-native world? Or is it a level playing field?

Jeff:

Good one. I can argue both sides.

When the cloud emerged and we invested in MongoDB, we thought we were going to eat Oracle’s lunch. MongoDB is now worth $20–25 billion. But guess what? Oracle’s market cap has also grown significantly.

Salesforce thought they were going to disrupt SAP. They built a massive business. But SAP also grew.

So… it’s less “winners and losers” and more “winners and winners.” Incumbents like Adobe and Salesforce are doing great by embedding AI deeply into their product suites. But AI-native startups are also thriving.

It’s a rare moment when both sides can win — if they adapt fast enough.

Jeff:

I think the idea of experimentation is incredibly valuable, especially within larger companies. We've seen how this culture has been embraced— Booking.com, for example, runs thousands of experiments daily across sites like Priceline. That kind of scale allows them to iterate quickly and make better, data-driven decisions around more efficient travel bookings.

This mindset of experimentation is powerful for entrepreneurs, too. And increasingly, we're seeing AI-forward companies and executives—like Salesforce's Marc Benioff—"eating their dog food," so to speak. They’re reinstrumenting their companies to be more AI-native, more agentic. That shift is rippling throughout the economy.

Susan Shultz:

Awesome—thanks, Jeff. Let’s go to Ivon next.

Ashley Wilson:

I’ve got a quick question before we move on. Jeff, you mentioned that you have a bot trained on the cases in your book. Have you looked at the answers it generates? Do you ever find yourself disagreeing with it?

Jeff:

Great question—yes, I have. I have two admin interfaces for the bots I use, so I can review and correct their answers and retrain them when needed. What's interesting is that I can see what students are asking ahead of time, which gives me insight before I even walk into class.

For example, I teach on Mondays and Tuesdays, so on Sunday night, I’ll check the bot traffic. Then I wake up Monday morning and see a flurry of new questions—students are working from midnight to 2 a.m., just like you might’ve done before your AB course!

If I don't like the answers the bot gave, I retrain it. And yes, it does get better over time. It’s my second year using the course bot and my first year using an AI clone. Both are improving a lot.

Ashley Wilson:

That’s amazing. So, you've been teaching the class for 15 years but only started using the chatbot recently?

Jeff:

Exactly. It’s my second time using the bot, and I keep refreshing the course content to keep it current—now, it’s fully fused with AI.

Ashley Wilson:

That’s incredible. So, here’s my follow-up: Over the past decade, has your view changed in how you select companies to invest in? Especially with this AI revolution?

Jeff:

Yes—two years ago, our firm shifted entirely to investing in AI-forward founders. We have a set of three criteria we look for in terms of competitive moats. So, our perspective on what makes a winning founder in 2025 is very different from what it was even a few years ago.

Ashley Wilson:

And for people who aren’t looking to start a company, but want to join one, what advice do you have? What does a "winning" company look like now?

Jeff:

Great question. Every year, I put together a list called the Startup Rocket Ship List. I’ll drop the link in the chat. The list outlines the criteria I use to spot high-potential startups. I’m updating the 2025 version right now—it’ll be released this spring.

If you're looking to join a startup, aim for one that’s growing, dynamic, a leader in its field, and where there’s clear opportunity for personal and company growth.

Todd Brous:

Jeff, how are we doing on time?

Jeff:

I should probably wrap up in a few minutes.

Todd Brous:

Okay—we’ve got time for three more quick ones: Ivon, Lilet, and Susan.

Ivon:

Hi Jeff—so, people often say that "humans with AI" will be the winners. But when you look at the output these tools are generating, it’s so good that I struggle to see how humans are expected to step in with judgment. Honestly, we’re kind of lazy, and we tend to defer to the tech when it’s this effective. Do you think the human element still plays a realistic role?

Jeff:

That’s a nuanced and very well-phrased question. I don’t know the full answer, but I do know that there's a lot of work happening around evaluation frameworks—ways to programmatically measure the quality of AI output with human judgment in mind.

Still, you’re right—humans are lazy. That’s one reason TikTok is as popular as it is. I’m not convinced we can fully count on human judgment alone.

Lilet:

Thanks, Jeff. I do some angel investing, and I’m curious about exits. It seems like we’re not seeing traditional exits anymore. Do you think the future will look more like a constant growth loop for AI-driven companies—or might we see model-level acquisitions rather than whole company acquisitions?

Jeff:

Great point. The last three years have seen really weak liquidity—very few IPOs, not much M&A. There’s hope that 2025 will look different, maybe thanks to a more business-friendly regulatory environment post-election.

The title of my latest investor letter is “The Return to Animal Spirits?” So maybe we’ll see big public companies stop being afraid of the FTC and get back to acquiring—especially to gain AI capabilities. It’s a pivotal moment.

Susan Shultz:

This has been so great—thank you again, Jeff. At the beginning of this session, we talked about customer discovery, which is essentially a form of experimentation.

A group of us is building an AI-forward app to make customer discovery easier—it’s currently super labor-intensive and still reliant on human judgment. I’ve been doing this for four years, and we pivoted completely based on discovery insights. But the biggest barrier I’ve seen is human hubris. People think they already know the answers and don’t want to experiment. Thoughts?

Jeff:

Yes, that’s a theme I address a lot in the book. One major piece of the answer is improving how we prompt AI.

We’ve all become great at Googling, but prompting is a totally different skill. I include an appendix in the book with tips and examples on how to get better at prompting.

These models respond best to rich, detailed input—give them context, documents, and go back-and-forth with them. It’s a new paradigm, and you’re right: a little effort goes a long way.

Todd Brous:

Thanks, Susan. Jeff, final question—we ask this of all our students. In the age of AI, what are the "forever jobs," and which ones should go away?

Jeff:

Jobs involving hands-on, human-to-human service—those are here to stay. Nurses, healthcare workers, elderly care—robots won’t replace that anytime soon. Same goes for physical trades: construction, plumbing, electrical work, landscaping.

That’s why I love Topline Pro—the value of a skilled plumber or welder has never been higher. Trevor’s comment in the chat about trying to get a plumbing apprenticeship? Smart move. Those are enduring vocations.

As for jobs that should go away… I want to be careful, but basically, high-salary roles that involve rote work. Think law firms, accounting, consulting, investment banking—lots of expensive professionals doing things that AI can automate.

Susan Shultz:

No government officials?

Jeff:

[Laughs] Well, my HBS colleague Mitch Weiss gives a whole speech on that. He talks about a 10x founder mindset applied to government—maybe City Hall becomes fully AI-run. Maybe we don’t even need a mayor someday! That’s Mitch’s talk, though—you’ll have to invite him next time.

Todd Brous:

Thank you so much, Jeff. This was fantastic. Everyone, thanks for joining! We’re off next week, but join us March 12 when Mike Salguero, CEO of ButcherBox, will talk about bootstrapping your business. Stick around if you want to hang out—we’re officially done recording.