Multipart AI

I made another realization over the past couple of days, and it actually dovetails nicely with the stuff I’ve been writing about AI.

The project I’m working on is writing a system to download and generate stats from a set of GitHub repos.

Now, the simple way to attack this with an AI helping out is to simply ask one of the Chats to “write a program to query GitHub and produce the stats I want.” This, of course, works. But the cycle time for iterating on the solution is slow. The script isn’t the bottleneck, it’s calling GitHub.

This is where being a programmer pays off — this is where you need to be to be successful. I asked to write something that queries GitHub and writes the intermediate results to a JSON. I also had it write documentation for the JSON format.

With this in hand, I could say “given a file in this format, write a program that does foo.” This part is all local, so you can iterate on this really quickly. It also separates out the downloading from the processing, making it far easier to verify each part is correct.

That last bit is crucial. AI is a tool. But it’s up to us humans to be able to verify that it’s doing what we want it to do. Not knowing how to test makes you far less valuable as a developer.

So the punchline is program like you’ve always done so: break a problem down into more manageable chunks and have a clear handoff between systems or parts of systems.