Cana: An Agentic IDE and Multi-Model Runtime

At Colate I've been building Cana, an agentic IDE backed by a secure multi-model runtime. The interesting part of this project was never a single clever prompt — it was the infrastructure required to run agents reliably, safely, and across different model providers, in production, with real users.

This post is a reflection on what building agent infrastructure actually looks like once it leaves the demo stage.

Why the runtime mattered more than the agent

It's easy to make an agent that works once in a notebook. It's much harder to make one that behaves the same way on the hundredth run, under load, for people who didn't write it. Cana forced me to treat the runtime as the real product:

• Agents run inside a hardened AWS Kubernetes environment, so untrusted execution is isolated by default.
• The model layer is swappable — OpenAI, Anthropic, Gemini, and Kimi backends can be substituted without rewriting agent logic.
• Everything is measured, so "it works" becomes a number instead of a vibe.

That last point ended up being the discipline that kept the project honest.

Swappable backends and a codegen proxy

Different providers return different shapes of output, and if you let that leak into your agents, you end up with provider-specific glue everywhere. To avoid that, Cana ships a compiler-style multi-model codegen proxy: it takes a prompt, lowers it to an intermediate representation, and then emits code.

• The prompt to IR to code path gives a stable middle layer to reason about.
• The proxy normalizes output formats across providers, so a switch from one backend to another doesn't ripple through the whole system.
• This proxy has handled 900M+ production requests, which is where a lot of the sharp edges around formatting and reliability actually showed up.

Thinking of codegen as a compiler pass — rather than a single LLM call — made the whole thing far easier to test and swap.

Shipping it to real users

Cana also ships a drop-in embeddable web agent, which meant the runtime had to be something other teams could adopt without understanding its internals. Across 100+ internal users, we tracked a 98.2% agent task pass rate.

A few things I took away from running this in production:

• Isolation is a feature, not overhead. Hardened, sandboxed execution is what makes agent code safe to run at all.
• A normalization layer buys you optionality. Because output formats were normalized, changing model backends became a configuration decision, not a rewrite.
• Measure the pass rate continuously. A single reliability number aligned everyone on whether changes were actually helping.

What building agent infra taught me

The recurring lesson was that agent quality in production is mostly an infrastructure problem. The model matters, but the runtime around it — isolation, normalization, measurement, embeddability — is what determines whether agents can be trusted by people other than their author.

Cana pushed me to design for that from the start rather than bolt it on later, and that shift in framing is the part I'll carry into whatever I build next.

If you want to see it, here it is: Cana.