1
00:00:00,000 --> 00:00:01,000
Zach Lloyd

2
00:00:01,000 --> 00:00:18,760
Okay. Hello, everyone. I'm excited to be here. My name is Zach Lloyd. Today I'm going to

3
00:00:18,760 --> 00:00:25,080
be talking about self-improving software factories, the new open source model, and basically what

4
00:00:25,080 --> 00:00:27,740
I think is happening to development.

5
00:00:27,740 --> 00:00:29,760
A little bit about me just to begin.

6
00:00:29,760 --> 00:00:33,900
So I am a former principal engineer from Google.

7
00:00:33,900 --> 00:00:36,900
I used to lead engineering on the Google Docs suite.

8
00:00:36,900 --> 00:00:41,200
I've been an engineer now for over 20 years, a long time.

9
00:00:41,200 --> 00:00:44,220
I am still shipping frequently, but I

10
00:00:44,220 --> 00:00:47,880
haven't written a line of code in the last six months.

11
00:00:47,880 --> 00:00:50,400
And I'm the founder of a company called Warp.

12
00:00:50,400 --> 00:00:54,520
Warp, if you're not familiar, is a open source agentic development

13
00:00:54,520 --> 00:00:55,840
environment.

14
00:00:55,840 --> 00:00:58,080
You may know us as a terminal.

15
00:00:58,080 --> 00:00:59,600
That is how the company started.

16
00:00:59,600 --> 00:01:03,480
We're basically a terminal that has agents built in.

17
00:01:03,480 --> 00:01:05,320
We open sourced it a couple months ago.

18
00:01:05,320 --> 00:01:08,040
And I'm going to talk a little bit about that experience

19
00:01:08,040 --> 00:01:09,640
and the motivation for it.

20
00:01:09,640 --> 00:01:12,340
It's a popular open source project, over 60,000 GitHub

21
00:01:12,340 --> 00:01:12,800
stars.

22
00:01:12,800 --> 00:01:15,580
We've had a couple hundred people contributing.

23
00:01:15,580 --> 00:01:19,760
We have over 800,000 active developers who are using warp.

24
00:01:19,760 --> 00:01:22,120
And increasingly, we are focused not just

25
00:01:22,120 --> 00:01:24,960
on the terminal aspect and the interactive aspect

26
00:01:24,960 --> 00:01:30,280
of development, but more so on how do you automate development.

27
00:01:30,280 --> 00:01:33,100
I'm going to talk mostly about that.

28
00:01:33,100 --> 00:01:37,880
So the thesis that I have is that the discipline

29
00:01:37,880 --> 00:01:41,860
of software engineering is going to become something more

30
00:01:41,860 --> 00:01:44,020
like factory engineering.

31
00:01:44,020 --> 00:01:46,200
And I'll explain what I mean by this in a minute,

32
00:01:46,200 --> 00:01:47,340
but just keep that in mind.

33
00:01:47,340 --> 00:01:50,360
That's what I think is going to happen.

34
00:01:50,360 --> 00:01:54,620
If you look at development over the past couple years,

35
00:01:54,620 --> 00:01:57,860
it's crazy how it's changed.

36
00:01:57,860 --> 00:02:02,060
We've gone from a world of chat and AI autocomplete,

37
00:02:02,060 --> 00:02:06,040
a cursor, co-pilot, to the phase that we're in now,

38
00:02:06,040 --> 00:02:09,140
which I consider to be mostly interactive agents.

39
00:02:09,140 --> 00:02:12,080
So you're sort of sitting at your computer

40
00:02:12,080 --> 00:02:14,560
and you are telling Cloud Code to do something,

41
00:02:14,560 --> 00:02:16,340
you're telling Warp to do something.

42
00:02:16,340 --> 00:02:17,800
And I believe what's gonna happen

43
00:02:17,800 --> 00:02:20,140
over the next six months, a year,

44
00:02:20,140 --> 00:02:22,060
hard to predict the pace, is that we're

45
00:02:22,060 --> 00:02:25,820
going to move much more towards a world of automation.

46
00:02:25,820 --> 00:02:27,940
But before I get into that, just a quick show of hands.

47
00:02:27,940 --> 00:02:32,080
How many folks in here are building with agents?

48
00:02:32,080 --> 00:02:34,300
100% makes sense.

49
00:02:34,300 --> 00:02:37,380
How many folks are building typically

50
00:02:37,380 --> 00:02:40,580
with multiple agents at one time?

51
00:02:40,580 --> 00:02:42,260
So again, almost everyone.

52
00:02:42,260 --> 00:02:44,180
How many people are running an agent right now?

53
00:02:44,180 --> 00:02:45,820
I'm not offended.

54
00:02:45,820 --> 00:02:47,280
OK, that's totally cool.

55
00:02:47,280 --> 00:02:48,820
I would be doing it too.

56
00:02:48,820 --> 00:02:53,360
How many folks are running agents in the cloud, out of curiosity?

57
00:02:53,360 --> 00:02:59,380
So that looks like less than half, but still significant.

58
00:02:59,380 --> 00:03:02,920
And how many folks have set up a system internally

59
00:03:02,920 --> 00:03:06,120
to automate the whole software development lifecycle?

60
00:03:06,120 --> 00:03:09,820
So everything from triaging, speccing, implementing,

61
00:03:09,820 --> 00:03:11,440
reviewing.

62
00:03:11,440 --> 00:03:14,500
So I see some hands.

63
00:03:14,500 --> 00:03:15,600
So some people are doing this.

64
00:03:15,600 --> 00:03:18,200
So this is what's going to happen.

65
00:03:18,200 --> 00:03:21,100
Every project of significant size, I believe,

66
00:03:21,100 --> 00:03:24,180
is going to have something like this.

67
00:03:24,180 --> 00:03:27,460
And it's going to look kind of like this big loop.

68
00:03:27,460 --> 00:03:29,140
And everyone is talking about loops.

69
00:03:29,140 --> 00:03:31,280
There's nothing that complicated about loops.

70
00:03:31,280 --> 00:03:33,200
This loop says the Cloud Software Factory.

71
00:03:33,200 --> 00:03:35,220
This loop could literally just say, like, the software

72
00:03:35,220 --> 00:03:36,160
development lifecycle.

73
00:03:36,160 --> 00:03:38,060
It's the same thing.

74
00:03:38,060 --> 00:03:41,440
But just to go through this loop, it's like ideas

75
00:03:41,440 --> 00:03:43,200
are going to come in at the top.

76
00:03:43,200 --> 00:03:45,240
Agents are going to do triage.

77
00:03:45,240 --> 00:03:48,180
If something is complicated, they will write a spec.

78
00:03:48,180 --> 00:03:50,160
These little blue boxes are where humans step in.

79
00:03:50,160 --> 00:03:52,540
Humans will review the spec.

80
00:03:52,540 --> 00:03:54,280
Agents will do the implementation.

81
00:03:54,280 --> 00:03:56,280
A human and agent will review the code.

82
00:03:56,280 --> 00:03:57,640
Agents will verify.

83
00:03:57,640 --> 00:03:59,500
Human will review the product.

84
00:03:59,500 --> 00:04:03,060
You ship and then you monitor and round and round you go.

85
00:04:03,060 --> 00:04:05,740
And this is what software development for better or worse

86
00:04:05,740 --> 00:04:07,940
I think is gonna end up looking like.

87
00:04:07,940 --> 00:04:10,040
So I repeat the thesis which is that

88
00:04:10,040 --> 00:04:13,000
if this is what software engineering is gonna look like,

89
00:04:13,000 --> 00:04:16,540
software engineers are going to be the ones

90
00:04:16,540 --> 00:04:20,040
who end up building and managing these factories.

91
00:04:20,040 --> 00:04:21,560
Now, I promised at the beginning,

92
00:04:21,560 --> 00:04:23,240
and I put in the title of the talk

93
00:04:23,240 --> 00:04:25,380
that I was gonna talk about open source,

94
00:04:25,380 --> 00:04:27,020
and so I wanna do that for a few minutes,

95
00:04:27,020 --> 00:04:28,680
I'm gonna take a quick digression.

96
00:04:30,040 --> 00:04:33,420
I bring up open source because one of the main reasons

97
00:04:33,420 --> 00:04:38,420
that Warp open source was to build a public factory.

98
00:04:38,420 --> 00:04:42,200
And so this is a picture of this website we've built

99
00:04:42,200 --> 00:04:43,580
called build.warp.dev,

100
00:04:43,580 --> 00:04:48,280
which shows all of the issues that are flowing through our system

101
00:04:48,280 --> 00:04:52,140
and what state they're in, what agents are working on them,

102
00:04:52,140 --> 00:04:54,180
what contributors are working on them.

103
00:04:54,180 --> 00:04:58,140
And it's kind of like a proto-factory done at scale.

104
00:04:58,140 --> 00:05:01,120
It's not working perfectly, but it is working.

105
00:05:01,120 --> 00:05:03,000
And one of the reasons we open sourced

106
00:05:03,000 --> 00:05:06,140
was to try to build this.

107
00:05:06,140 --> 00:05:08,100
Just in general, I think it's interesting

108
00:05:08,100 --> 00:05:12,640
to talk about open source in the time of agentic development.

109
00:05:12,640 --> 00:05:16,860
This is a really stupid graph, but it's like, you get it?

110
00:05:16,860 --> 00:05:21,240
It's becoming much cheaper to build software.

111
00:05:21,240 --> 00:05:25,520
A corollary of that is that it's becoming trivial

112
00:05:25,520 --> 00:05:27,160
to clone software.

113
00:05:27,160 --> 00:05:30,680
And so if you are in the software business,

114
00:05:30,680 --> 00:05:32,760
I don't know how many folks in this room

115
00:05:32,760 --> 00:05:34,960
are in the software business per se,

116
00:05:34,960 --> 00:05:37,380
but it's very hard to build a software business

117
00:05:37,380 --> 00:05:38,860
if it's free to build software.

118
00:05:38,860 --> 00:05:40,580
It's hard to capture the value,

119
00:05:40,580 --> 00:05:43,140
especially if a competitor can clone.

120
00:05:43,140 --> 00:05:46,560
And so my big takeaway or a big tip for everyone in here

121
00:05:46,560 --> 00:05:49,680
is that the first thing you should do is patent your code.

122
00:05:49,680 --> 00:05:53,160
I'm kidding, this is a complete joke.

123
00:05:53,160 --> 00:05:55,800
Don't do this.

124
00:05:55,800 --> 00:06:00,920
My first tip is obviously you need to have a great product.

125
00:06:00,920 --> 00:06:03,240
This has always been the case.

126
00:06:03,240 --> 00:06:06,840
But I would say a great product probably was never enough.

127
00:06:06,840 --> 00:06:09,500
But even now, more than ever, if you

128
00:06:09,500 --> 00:06:11,560
think that you're going to build a great software business just

129
00:06:11,560 --> 00:06:13,440
by building and shipping a great product,

130
00:06:13,440 --> 00:06:16,140
you're probably not going to succeed.

131
00:06:16,140 --> 00:06:18,780
You need advantages beyond the product.

132
00:06:18,780 --> 00:06:22,280
And so those advantages could look like distribution,

133
00:06:22,280 --> 00:06:25,100
ecosystem, it could be that you have a great brand

134
00:06:25,100 --> 00:06:28,480
or a data moat, you might have capital.

135
00:06:28,480 --> 00:06:30,200
But if you're a startup,

136
00:06:30,200 --> 00:06:32,420
again, I'm coming from the startup world here,

137
00:06:32,420 --> 00:06:34,620
you just don't have these advantages.

138
00:06:34,620 --> 00:06:37,980
And so you still want to break through.

139
00:06:37,980 --> 00:06:41,260
And one of the ways that I suggest doing this

140
00:06:41,260 --> 00:06:42,600
is by building in the open.

141
00:06:42,600 --> 00:06:45,440
And so to be clear, it took Warp five years

142
00:06:45,440 --> 00:06:48,700
of building closed to sort of make the leap

143
00:06:48,700 --> 00:06:51,260
into building in the open, and I'll explain why.

144
00:06:51,260 --> 00:06:55,380
But if you build in the open, it helps build your ecosystem.

145
00:06:55,380 --> 00:06:58,140
It can take you from being hated on Hacker News

146
00:06:58,140 --> 00:06:59,980
to tolerated.

147
00:06:59,980 --> 00:07:02,500
It can burnish your brand, it creates community,

148
00:07:02,500 --> 00:07:05,780
and so there's all these advantages to it.

149
00:07:05,780 --> 00:07:07,940
And some of the things that I think have traditionally

150
00:07:07,940 --> 00:07:11,700
been a pain can now be managed.

151
00:07:11,700 --> 00:07:15,400
And so the traditional pain of open source

152
00:07:15,400 --> 00:07:18,700
might be something like you get a lot of noisy issues.

153
00:07:18,700 --> 00:07:23,420
You get sloppy PRs, you can end up in code review hell,

154
00:07:23,420 --> 00:07:27,060
you can end up having to spend a lot of time verifying changes.

155
00:07:27,060 --> 00:07:29,900
And so the solution,

156
00:07:29,900 --> 00:07:33,400
it's a kind of a long-winded way of getting to this for open source,

157
00:07:33,400 --> 00:07:35,760
or at least for warp in the case of open source,

158
00:07:35,760 --> 00:07:40,080
the thing that made us finally decide to do this was that we built a whole set of

159
00:07:40,080 --> 00:07:43,500
automations, really a software factory,

160
00:07:43,500 --> 00:07:46,020
around managing the open source project.

161
00:07:46,020 --> 00:07:50,020
And so, like I said, this is what I think the future is going to look like.

162
00:07:50,020 --> 00:07:58,020
I'm going to drill into it a bit just to get a little bit more technical for folks who want to try to build something like this for their own projects.

163
00:07:58,020 --> 00:08:02,020
So what are the components of an effective software factory?

164
00:08:02,020 --> 00:08:06,020
It's really not that complicated to start or at a high level.

165
00:08:06,020 --> 00:08:08,020
You need a set of automations.

166
00:08:08,020 --> 00:08:12,020
You need a way of providing context and skills.

167
00:08:12,020 --> 00:08:16,460
you need a way of bringing humans in at the correct time,

168
00:08:16,460 --> 00:08:19,860
sort of like when things get stuck on the factory.

169
00:08:19,860 --> 00:08:21,940
And then a really important thing is you need some set

170
00:08:21,940 --> 00:08:23,800
of self-improvement capabilities.

171
00:08:23,800 --> 00:08:25,680
So think of this as loops.

172
00:08:26,640 --> 00:08:29,580
And if you do this right in the open source world,

173
00:08:29,580 --> 00:08:31,960
you can get a world where agents are helping

174
00:08:31,960 --> 00:08:33,840
contributors contribute,

175
00:08:33,840 --> 00:08:35,660
they're helping maintainers maintain.

176
00:08:37,060 --> 00:08:38,760
And I wanna emphasize, there's nothing special

177
00:08:38,760 --> 00:08:40,400
about open source here.

178
00:08:40,400 --> 00:08:43,940
I think every sizable project can benefit from this approach,

179
00:08:43,940 --> 00:08:49,400
and I predict that every company, every open source project,

180
00:08:49,400 --> 00:08:52,260
will have at its core a software factory,

181
00:08:52,260 --> 00:08:55,100
kind of like the way that CI CD became just like,

182
00:08:55,100 --> 00:08:57,560
oh, of course you have that.

183
00:08:57,560 --> 00:09:00,520
Maybe, I don't know when that happened, 10 years ago.

184
00:09:00,520 --> 00:09:03,020
Let's tour the factory floor for a second here.

185
00:09:03,020 --> 00:09:05,400
So you're not going to look at this.

186
00:09:05,400 --> 00:09:07,160
This is too much.

187
00:09:07,160 --> 00:09:10,240
The point of this slide is not to have you read the workflow.

188
00:09:10,240 --> 00:09:13,900
it's that the factory floor is basically a graph

189
00:09:14,820 --> 00:09:17,980
of steps where you are defining like,

190
00:09:17,980 --> 00:09:21,160
okay, how does software get built for my product?

191
00:09:21,160 --> 00:09:23,360
And it looks pretty similar for every product.

192
00:09:23,360 --> 00:09:25,220
Things come in, they flow through,

193
00:09:26,100 --> 00:09:27,820
they get stuck at certain points.

194
00:09:29,240 --> 00:09:31,220
And, you know, broadly speaking,

195
00:09:31,220 --> 00:09:34,220
just to back out a second, so there's the inputs.

196
00:09:34,220 --> 00:09:36,760
The inputs are really ideas.

197
00:09:36,760 --> 00:09:38,760
The inputs could be coming from your team,

198
00:09:38,760 --> 00:09:42,540
They could be coming from your users.

199
00:09:42,540 --> 00:09:44,400
The inputs themselves tend to come in

200
00:09:44,400 --> 00:09:46,900
through certain channels that you should think about

201
00:09:46,900 --> 00:09:49,880
as like your task tracker is an obvious one,

202
00:09:49,880 --> 00:09:54,020
or Slack, your Teams, like your communication channels.

203
00:09:54,020 --> 00:09:57,400
It could come directly from like your terminal or IDE.

204
00:09:57,400 --> 00:09:59,600
They could come from your monitoring systems,

205
00:09:59,600 --> 00:10:00,780
but there's some set of inputs

206
00:10:00,780 --> 00:10:02,540
that bring work into the factory.

207
00:10:03,660 --> 00:10:04,560
There's triage.

208
00:10:04,560 --> 00:10:06,040
This is a really important step.

209
00:10:06,040 --> 00:10:09,560
So again, I boil it down to something very simple,

210
00:10:09,560 --> 00:10:12,820
but like you want an agent that is looking at issues

211
00:10:12,820 --> 00:10:15,240
as they come in and just saying, you know,

212
00:10:15,240 --> 00:10:18,380
if this is easy and this is unambiguous,

213
00:10:18,380 --> 00:10:20,760
just implement it, and this is how you can actually

214
00:10:20,760 --> 00:10:22,380
get going with a factory.

215
00:10:22,380 --> 00:10:29,000
If an issue is hard, I recommend having an agent that produces specs.

216
00:10:29,000 --> 00:10:33,580
Folks in here using spec-driven development, show of hands.

217
00:10:33,580 --> 00:10:36,740
You can do this many different ways, but I think it's very effective.

218
00:10:36,740 --> 00:10:42,360
The way that we do it at Warp that I recommend is having an agent write what we call a product

219
00:10:42,360 --> 00:10:44,960
spec and a tech spec.

220
00:10:44,960 --> 00:10:48,820
Product spec describes the product invariance that you're building towards.

221
00:10:48,820 --> 00:10:53,320
spec describes the architecture and the shape of the code.

222
00:10:53,320 --> 00:10:55,700
Then you have an implementation agent.

223
00:10:55,700 --> 00:10:59,640
This is basically a coding agent that runs somewhere in the Cloud.

224
00:10:59,640 --> 00:11:03,220
It makes a diff. You can use all sorts of coding agents for this.

225
00:11:03,220 --> 00:11:08,280
You have review. This is in many ways the most painful part.

226
00:11:08,280 --> 00:11:12,180
Like I expect that people are a little bit tired of reviewing

227
00:11:12,180 --> 00:11:16,820
agentic slop. I would have an agent to code review first,

228
00:11:16,820 --> 00:11:21,820
And then it becomes over time like a risk management exercise

229
00:11:21,820 --> 00:11:24,820
of like when do you bring in humans to do code review,

230
00:11:24,820 --> 00:11:27,820
but you want to have a step in here where humans can do it.

231
00:11:27,820 --> 00:11:31,820
This is a very important step for certain types of apps,

232
00:11:31,820 --> 00:11:32,820
the verification step.

233
00:11:32,820 --> 00:11:35,820
So this would be things like computer use.

234
00:11:35,820 --> 00:11:37,820
If you're building a sort of UI,

235
00:11:37,820 --> 00:11:40,820
having the computer actually use the code that the agent produced

236
00:11:40,820 --> 00:11:42,820
and producing videos and screenshots.

237
00:11:42,820 --> 00:11:45,820
CI, CD still use it, obviously.

238
00:11:45,820 --> 00:11:51,820
And then monitoring, so agents don't stop in your factory when code is shipped.

239
00:11:51,820 --> 00:11:54,820
They should observe what's been shipped.

240
00:11:54,820 --> 00:11:55,820
Is it crashing?

241
00:11:55,820 --> 00:11:56,820
Is it being used?

242
00:11:56,820 --> 00:12:00,820
And round and round you go because you take the output of this monitoring step

243
00:12:00,820 --> 00:12:02,820
and you feed it back into the top of the factory.

244
00:12:02,820 --> 00:12:05,820
Now, you could try and build this.

245
00:12:05,820 --> 00:12:10,820
And I went to a talk earlier that my friend Adam gave where Uber has built

246
00:12:10,820 --> 00:12:14,820
an internal version of this, and it's pretty cool.

247
00:12:14,820 --> 00:12:19,140
I would say for most organizations,

248
00:12:19,140 --> 00:12:20,540
it really depends where you are,

249
00:12:20,540 --> 00:12:22,760
you'll be able to build a simple version of this easily,

250
00:12:22,760 --> 00:12:26,120
but to build a thing that actually scales,

251
00:12:26,120 --> 00:12:28,020
it's probably, like you should probably be focusing

252
00:12:28,020 --> 00:12:31,460
on your own product, not building this infrastructure,

253
00:12:31,460 --> 00:12:33,400
because there's a lot of stuff that you end up wanting.

254
00:12:33,400 --> 00:12:35,260
You're not supposed to read this.

255
00:12:35,260 --> 00:12:36,600
It's just a lot of stuff.

256
00:12:36,600 --> 00:12:40,100
If you do build it, or if you buy it,

257
00:12:40,100 --> 00:12:43,300
you'll end up with something that looks kind of like this,

258
00:12:43,300 --> 00:12:46,600
which is you're going to have a bunch of ways

259
00:12:46,600 --> 00:12:48,320
of getting work into your factory.

260
00:12:48,320 --> 00:12:50,000
That's what's at the top here.

261
00:12:50,000 --> 00:12:51,600
You're going to have a sort of control plane

262
00:12:51,600 --> 00:12:54,440
for figuring out how work gets distributed

263
00:12:54,440 --> 00:12:56,600
across your factory floor.

264
00:12:56,600 --> 00:12:57,800
You're going to have the actual place

265
00:12:57,800 --> 00:12:59,440
where the work happens.

266
00:12:59,440 --> 00:13:02,240
And so that's going to be cloud sandboxes.

267
00:13:02,240 --> 00:13:05,320
It's going to be figuring out what agent to run.

268
00:13:05,320 --> 00:13:07,100
So what's the harness?

269
00:13:07,100 --> 00:13:08,680
What's the model?

270
00:13:08,680 --> 00:13:10,920
And then finally, I think this is a really important thing.

271
00:13:10,920 --> 00:13:15,680
you're going to want to set up some kind of data plane that sits below your factory.

272
00:13:15,680 --> 00:13:24,700
And so that's something that lets agents remember what they've done, learn, improve over time.

273
00:13:24,700 --> 00:13:28,640
The factory is not just like a product, it's also a mindset.

274
00:13:28,640 --> 00:13:33,900
And this brings me back to the thesis I had at the beginning.

275
00:13:33,900 --> 00:13:35,960
You need to measure and improve.

276
00:13:35,960 --> 00:13:40,920
So factory, this is where the, I don't know, you can stretch this metaphor as far as you

277
00:13:40,920 --> 00:13:44,480
want, but like you should be thinking of efficiency.

278
00:13:44,480 --> 00:13:50,480
And so that means like how much software did you ship, how much did it cost in terms of

279
00:13:50,480 --> 00:13:55,200
human time and token time, and you're going to want to measure this and try to improve

280
00:13:55,200 --> 00:13:57,900
it over time.

281
00:13:57,900 --> 00:14:00,660
A key part of this is creating loops.

282
00:14:00,660 --> 00:14:04,560
So loops are, again, they sound complicated.

283
00:14:04,560 --> 00:14:05,920
They're not that complicated.

284
00:14:05,920 --> 00:14:09,760
are basically ways of having agents improve

285
00:14:09,760 --> 00:14:12,980
by like observing what they're doing,

286
00:14:12,980 --> 00:14:14,200
where they're failing.

287
00:14:14,200 --> 00:14:17,440
And so a common kind of loop that you're gonna wanna

288
00:14:17,440 --> 00:14:19,800
put in your factory is like a skill loop.

289
00:14:19,800 --> 00:14:21,380
That means you're gonna have your factory agents

290
00:14:21,380 --> 00:14:22,780
that are running skills,

291
00:14:22,780 --> 00:14:24,400
and then you'll have observer agents

292
00:14:24,400 --> 00:14:27,040
that are seeing how those skills are being applied,

293
00:14:27,040 --> 00:14:29,740
looking for issues and trying to improve the skills.

294
00:14:29,740 --> 00:14:31,700
So for instance, if you had a code review agent

295
00:14:31,700 --> 00:14:33,720
and it was leaving comments

296
00:14:33,720 --> 00:14:35,900
and a senior engineer on your team

297
00:14:35,920 --> 00:14:37,900
going and correcting those comments,

298
00:14:37,900 --> 00:14:41,200
you'd want an observer agent that would look at that

299
00:14:41,200 --> 00:14:44,700
and basically improve the code review agent for the next run.

300
00:14:48,100 --> 00:14:49,900
This is one thought just to leave folks with.

301
00:14:49,900 --> 00:14:53,900
Like, where does this leave engineers?

302
00:14:53,900 --> 00:14:57,460
I think you're going to have to get into this mindset.

303
00:14:57,460 --> 00:14:59,820
And I'm trying very hard to get our team into this mindset.

304
00:14:59,940 --> 00:15:05,620
not always easy, that you're not just building the product, but you're building the thing

305
00:15:05,620 --> 00:15:12,240
that builds the product, and that's like, it's just different. It's more like process

306
00:15:12,240 --> 00:15:17,600
engineering or manufacturing or something like that. And you could think like, okay,

307
00:15:17,600 --> 00:15:23,820
maybe that's a bummer. Like, is that a bummer? Is that, you know, and it depends. Like, it

308
00:15:23,820 --> 00:15:26,500
it depends what joy you get out of software engineering.

309
00:15:26,500 --> 00:15:28,760
If your joy is in writing the code,

310
00:15:28,760 --> 00:15:31,440
I think everyone in here who is a software engineer

311
00:15:31,440 --> 00:15:33,380
is going to be writing less code.

312
00:15:33,380 --> 00:15:36,780
But if your joy is in shipping product,

313
00:15:36,780 --> 00:15:38,380
like it's never been a better time,

314
00:15:38,380 --> 00:15:40,120
and this is actually where I find my joy.

315
00:15:40,120 --> 00:15:43,160
It's like I like building and shipping the thing.

316
00:15:43,160 --> 00:15:45,620
So everyone in here is gonna code less,

317
00:15:45,620 --> 00:15:47,000
but they're gonna ship more,

318
00:15:47,000 --> 00:15:49,320
and that's gonna be a trade-off.

319
00:15:49,320 --> 00:15:53,320
But if you approach it like you're a factory engineer,

320
00:15:53,820 --> 00:15:57,880
I think you can see that there's still a really cool set

321
00:15:57,880 --> 00:15:59,860
of engineering challenges.

322
00:16:00,820 --> 00:16:02,580
You can almost think of it as meta engineering,

323
00:16:02,580 --> 00:16:05,240
like how do you engineer your system of agents

324
00:16:05,240 --> 00:16:07,280
to be the best possible of engineering?

325
00:16:07,280 --> 00:16:08,880
That I think is a very compelling

326
00:16:08,880 --> 00:16:10,940
and interesting set of challenges to solve.

327
00:16:12,560 --> 00:16:13,400
So that's it.

328
00:16:14,580 --> 00:16:16,260
For folks who are interested,

329
00:16:17,440 --> 00:16:19,340
if you follow the link on this QR code,

330
00:16:19,340 --> 00:16:22,940
I've set up a open source GitHub repo

331
00:16:22,940 --> 00:16:26,760
where anyone who wants to try building their own factory

332
00:16:26,760 --> 00:16:28,160
agents can do it.

333
00:16:28,160 --> 00:16:32,200
This uses Warp's agent platform as part of it.

334
00:16:32,200 --> 00:16:33,820
But you honestly, you don't have to use it.

335
00:16:33,820 --> 00:16:35,720
I'm not trying to push into our product.

336
00:16:35,720 --> 00:16:37,700
But this should give you a good sense of, OK,

337
00:16:37,700 --> 00:16:41,800
if you want to set up an agent that does triage

338
00:16:41,800 --> 00:16:45,060
or an agent that does spec writing,

339
00:16:45,060 --> 00:16:46,440
how do you actually do that?

340
00:16:46,440 --> 00:16:50,380
How do you get from the theory of working with a factory

341
00:16:50,380 --> 00:16:52,880
to actually putting it into practice.

342
00:16:54,240 --> 00:16:55,820
I don't know if we have the capability

343
00:16:55,820 --> 00:16:56,740
to do questions in here.

344
00:16:56,740 --> 00:16:58,360
I saved a few minutes for questions,

345
00:16:58,360 --> 00:17:01,800
if anyone has questions, otherwise I will wrap up.

346
00:17:01,800 --> 00:17:02,640
Yes?

347
00:17:02,640 --> 00:17:03,480
Yeah, I have a question.

348
00:17:03,480 --> 00:17:05,380
There's a bit of a tension in what you said,

349
00:17:05,380 --> 00:17:06,760
where it's like you don't want to build it,

350
00:17:06,760 --> 00:17:07,680
it's a lot of work.

351
00:17:07,680 --> 00:17:08,520
Yes.

352
00:17:08,520 --> 00:17:09,520
But you are also a factory builder,

353
00:17:09,520 --> 00:17:10,720
like where do you get down to it?

354
00:17:10,720 --> 00:17:11,740
It's a great question.

355
00:17:11,740 --> 00:17:14,300
So I said something that's almost contradictory.

356
00:17:14,300 --> 00:17:17,080
I think you should, the way you should think of it

357
00:17:17,080 --> 00:17:20,920
is everyone's going to deploy some sort of factory,

358
00:17:20,920 --> 00:17:23,360
but then the tuning of the factory,

359
00:17:23,360 --> 00:17:27,000
the like, are these the right skills for my domain?

360
00:17:27,000 --> 00:17:30,540
Is this factory building my product in the right way?

361
00:17:30,540 --> 00:17:31,700
I still think there's a bunch

362
00:17:31,700 --> 00:17:33,520
of interesting engineering challenges.

363
00:17:33,520 --> 00:17:35,620
And for some places, you can build this.

364
00:17:35,620 --> 00:17:39,680
But again, I think that you should probably be focusing

365
00:17:39,680 --> 00:17:41,700
on building the core product for your company

366
00:17:41,700 --> 00:17:44,680
for the most part, but there's a bunch of tuning

367
00:17:44,680 --> 00:17:48,840
and figuring out how to make the factory work for your product that matters.

368
00:17:48,840 --> 00:17:49,840
That's a great question.

369
00:17:49,840 --> 00:17:50,840
Yes.

370
00:17:50,840 --> 00:17:51,840
Hey, Tom.

371
00:17:51,840 --> 00:17:52,840
Thank you so much.

372
00:17:52,840 --> 00:17:53,840
I have a question.

373
00:17:53,840 --> 00:17:58,680
So if you were an college student right now, and graduated and entered the workforce, where

374
00:17:58,680 --> 00:18:00,180
would you expect that?

375
00:18:00,180 --> 00:18:01,180
Yeah.

376
00:18:01,180 --> 00:18:05,080
So the question in case people couldn't hear was if I was a college student graduating

377
00:18:05,080 --> 00:18:07,180
and entering the workforce right now.

378
00:18:07,180 --> 00:18:12,780
So I think that the most important skills in this new world are adaptability.

379
00:18:12,780 --> 00:18:17,980
I think that's critical thinking, it's like the speed at which you can learn.

380
00:18:17,980 --> 00:18:21,180
I do think, I don't know if you're a computer science student, but I still think there's

381
00:18:21,180 --> 00:18:27,120
a ton of value in understanding the underlying systems and architecture and being able to

382
00:18:27,120 --> 00:18:34,360
reason and understand the code, understand the specs that agents are writing.

383
00:18:34,360 --> 00:18:38,940
So I would focus on those skills.

384
00:18:38,940 --> 00:18:42,940
And like, I don't know, we're hiring more people than we've ever hired.

385
00:18:42,940 --> 00:18:47,780
There's a lot of kind of like misdirection around like, you know, people not being hired

386
00:18:47,780 --> 00:18:48,780
because of AI.

387
00:18:48,780 --> 00:18:50,560
That's not the experience we've had so far.

388
00:18:50,560 --> 00:18:57,380
And what I'm looking for are like really adaptable product focused thinkers who can like basically

389
00:18:57,380 --> 00:19:02,660
be great problem solvers even as the underlying technology changes.

390
00:19:02,660 --> 00:19:05,620
Yeah, I think I have time for one more question.

391
00:19:05,620 --> 00:19:06,620
Yes?

392
00:19:06,620 --> 00:19:18,100
What about the, like, so the question was what about the product taste in like the,

393
00:19:18,100 --> 00:19:21,200
how do you actually build something useful I think is probably the right, like maybe

394
00:19:21,200 --> 00:19:24,440
the framing and like where do the ideas come from?

395
00:19:24,440 --> 00:19:29,860
And so I think that the problem with the factory metaphor, even though I'm like leaning into

396
00:19:29,860 --> 00:19:34,940
it because I think that's like, there's something to it, is that it can kind of sound like mechanizing

397
00:19:34,940 --> 00:19:41,060
or dehumanizing. And I still think underlying all of this, the only thing that matters is

398
00:19:41,060 --> 00:19:47,520
are you building something useful? And if you have a factory that is churning out shit

399
00:19:47,520 --> 00:19:52,460
that no one cares about, it's like what's the point? And I think that human taste, human

400
00:19:52,460 --> 00:19:58,960
input, human product sense, humans guiding at those touch points where you can't automate

401
00:20:00,120 --> 00:20:04,680
absolutely like essential and like that's like what I do I'm like I'm trying to

402
00:20:04,680 --> 00:20:07,780
figure out what do customers want what do people want what's gonna be valuable to

403
00:20:07,780 --> 00:20:13,220
them so I think that's an absolutely key point to it I think I'm at time so I have

404
00:20:13,220 --> 00:20:18,040
to go I hope folks enjoyed this chat I'm really grateful for being invited to

405
00:20:18,040 --> 00:20:21,240
speak so thank you all very much