Extensible, personal software is the sweet spot for LLMs

I’ve been thinking about how awesome it is to use software that is easy to extend with LLMs.

Two great examples I use daily are Pi and Herdr.

Pi has a very simple foundation and is extensible by design. But extensible software has existed forever. The true beauty of Pi is that you just prompt your agent to extend it: tell Pi what you want, it does its thing, you type /reload, and it works. You do not have to care about the code it created at all!

Likewise, Herdr lets you build plugins that can run inside Herdr. But the other thing I love about Herdr is that it comes with its own agent automation tooling that tells Herdr how to modify itself. Want to create a new workflow that creates a worktree and launches two different agents in different panes? Just ask your clanker to do it!

There’s something special about how I work with those tools that makes me feel okay just prompting the LLMs and not caring about the code at all. What is that?

Properties that make them sweet spots

The first thing to acknowledge is that Pi and Herdr provide great foundations and hooks upon which to build those extensions and automations. So, that’s a prerequisite to hit that sweet spot.

But I’m also curious about the properties of the extensions themselves, what we’re creating purely by prompting the LLMs:

• Their behavior is easy to verify
• They’re limited in scope
• They don’t need to scale (adapt to more usage) or adapt to changing requirements

Easy to verify

Because we’re just extending a piece of software for our own use, we can typically verify it very quickly and cheaply.

In Pi, we just type /reload and see if it works. If we build a Herdr extension in Rust, we run just link, reload Herdr, and confirm it’s doing what we want. And if we create a new custom workflow with Herdr’s automation, we can see it in action as it opens worktrees and panes and launches other agents.

We are the sole judges of whether or not it works, and we can make that judgment in an instant.

Limited in scope

And because it’s personalized software, it usually has an end goal. It doesn’t have to grow into a monstrosity (though, if you keep iterating on some plugins long enough, they might 😆). Typically, there’s a point at which we call it “done”. The software is just for us, so it only needs to satisfy our narrow use case.

No need to scale or adapt to changing requirements

And because each extension is limited in scope, and we’re typically the only users, change is limited.

Our plugin doesn’t have hundreds of customers with ever-changing needs that our software needs to satisfy. No, it’s closer to a Unix tool that does one thing well enough, and in this case, just for us. If we need to do something else, we’ll likely create a different plugin.

Compare that to how most of our production applications are today: they are hard to verify (at least some parts of them), frequently grow in scope, and need to scale and adapt to ever-changing requirements.

It doesn’t mean LLMs can’t help there. I just think that’s one reason why we use LLMs differently (and why it feels less magical) when we’re building software that needs to scale, grow, and change over time.