Ephemeral testing – Daniel Lemire’s weblog

We have some ways to make sure software program high quality. Unit testing. Fuzz testing. Integration testing. And so forth.

I’d prefer to suggest a technique that was unthinkable earlier than: ephemeral testing. (Ephemeral is a elaborate phrase for ‘throw away’ or ‘momentary’.)

You write your code. You construct your software program element. Or the AI agent does it for you, it doesn’t matter.

Then you ask an AI agent to construct on it: an software, one other layer, perhaps a number of. You have it take a look at what it constructed. You don’t assess the unique work instantly. You assess how good the software program constructed on high of it’s.

It is a type of integration testing. The distinction is that the software program on high is totally ephemeral. You throw it away when you find yourself finished.

A library with a clear API, steady invariants, and helpful errors lets the agent produce one thing that works shortly. A library with hidden state, shocking defaults, or incomplete docs produces a pile of patches and failures. The failures are proof about your code, not in regards to the agent.

You can repeat it. Different brokers, totally different duties, identical basis.

In impact, as a substitute of constructing the core whereas attempting to anticipate what is perhaps wanted on the different layers, you simply simulate the opposite layers by really constructing them.

Of course, you can argue that with AI, you may rebuild all the pieces every time you want to. But that’s not sensible. You want some type of stability.

I’ve been making use of this trick to numerous initiatives. As I contemplate a brand new characteristic, I ask my AI to prototype shortly what I would later construct based mostly on what I’m doing it. Ephemeral testing works for me so far.