Putting our product analytics on autopilot with a persistent Goal

A few weeks into the Telos Cloud alpha, we wanted to understand how our early users were actually using the product.

We could see signups, but not what happened next. Did their first deploy succeed? Where did the ones who left get stuck? What did people come back to? And with users across time zones, we couldn't be on call 24/7 to onboard and debug live.

These are ordinary questions, but the flows behind them are ours. We wanted analytics shaped around our product that told us when something moved, without one of us babysitting it.

So, about a month ago, we wrote that into a Goal and let Telos handle it for us. (If you're new here: Telos is a platform for declarative, goal-oriented software engineering. Read Introducing Telos Cloud for an overview.)

In our last post, we showed that a Goal could build and evolve a multiplayer FPS. That was a demonstration of what the platform could do (we are not, sadly, a game studio). This is something we built on Telos to solve one of our real business problems.

Could we go bespoke?

Of the SaaS offerings, the obvious answer was PostHog. It's excellent for what it does, and it runs on ClickHouse too. But we'd mostly read the data through a coding agent anyway, and the agent only answers when we ask it something.

Every option assumed someone was sitting on the other side of the software, building the funnels, tuning the thresholds, and keeping everything up-to-date as the product changed. On a small team, that someone was us, usually late at night.

With all the chatter lately about bespoke software services, we kept coming back to the same question: could we build something tailored to us, without the maintenance overhead?

That's exactly the problem we built Telos for. You write a Goal and Telos builds the service, runs it, and keeps checking it against the Goal for as long as it lives. So, halfway through setting up an account on PostHog, we decided to dogfood our own platform.

We boiled it down to four questions:

1. What are people doing?
2. What's breaking?
3. How slow is the API?
4. Who is behind these events?

If those four could be a Goal, there was no reason to stop at the software. Telos could own the whole analytics function, intelligence included.

Turning a Goal into a live production service

We worked with our coding agents to write out the goal spec against our high-level questions. We made sure we:

• Wrote outcomes, not implementation. "Meaningful change surfaces before anyone asks."
• Put the SLA in the contract. "At one million events, every default view responds within 3 seconds."
• Attached our standards as rubrics. Our design system and components, so it would feel Telos-native.
• Put it on a schedule. interval: 6h, so Telos re-checks the running service four times a day.

A tailored fit

Half a day after that first apply, we had a running service built for us: event ingestion, identity resolution, ClickHouse storage, a generated interface, and a production deployment with copy-paste client snippets.

All of that from one telos apply.

We wired the schemaless API into our frontend and backend and started shipping the events we cared about.

Here's what the result looked like:

The service composes its own pages from the evidence, and adds, removes, and reorganizes them as the data changes. package_pulled is an event only our product emits, and it got its own panel because the evidence warranted one.

It's not a one-size-fits-all SaaS dashboard with our events poured in. The pages exist because our events do. This was one of the outcomes we were most excited about — we didn't have to learn a new product. Instead, it learned ours.

Every five minutes, a background analysis loop studied the event stream, refreshed its findings, and recomposed the product when the evidence warranted a change. In the four weeks since, it has recomposed the interface 40 times without a prompt from us.

Front and center on the home page, it surfaced three things we hadn't asked about, one of them a real bug:

• At the top, API p95 had climbed to 1,033 ms, 691% above its seven-day daily mean, with the query and where to click.
• Beside it, 48 database is locked exceptions against a baseline of 35, a real bug in our control plane we hadn't noticed (...and probably wouldn't have).
• Package pulls up ~95% on the same window, which is just our open beta launch showing up.

The system healed itself

Getting the service built took half a day. The real test of our system was whether it could keep the software alive continuously.

For four weeks, the service just ran. Then, the day after our open beta, the weather changed. The database crossed a million live production events, and the night after launch, the three-second view SLA we had written into the Goal failed.

We didn't find out until the next morning. By then Telos had spent the whole night on it, finding and fixing real regressions. Here are the highlights:

The culprit was the analysis loop that builds our home screen. Each cycle ran 178 queries over the store, reading 19 million rows, on the same compute serving user-facing queries.

Importantly — nothing had suddenly broken. The store had grown organically with real usage, and queries that were fine in week one weren't fine in week four.

Telos folded two scans into one, generated time series only for findings that survived ranking, and halved the loop's share of ClickHouse.

All without human intervention.

Hands-free mode

Software is a living thing. Even code that was right the day you shipped it drifts, because the world it runs in keeps moving: data grows, usage patterns shift, requirements change underneath it. Continuously reconciling that drift is most of what software costs. Robert Glass put maintenance at 60 percent of the lifetime cost of software two decades ago, and nothing since has made it cheaper.

Normally the wear keeps accumulating until someone notices. We would have had to jump in ourselves: open an agent thread, point it at the code, and ask it to investigate. With Telos, the system ran on autopilot, like a self-driving car correcting a drift before the driver feels it.

We've both been on the human side of nights like this one: paged at 21:38, reading query logs, finding a deploy defect at 9 in the morning. This time it took under eleven hours and we slept through all of it. We'd like a lot more mornings like that.

Try it out

With this new way of working, writing the Goal was still the hard part, and we think deciding what we actually want should stay with human teams. For everything else: we were happy to take it off our plate.

Telos Cloud is in open beta. Try it in the cloud, or run the harness locally (source code).

If you have an internal service you've been putting off building, or an off-the-shelf SaaS product you want more out of, give this to your coding agent to get started:

Questions, bug reports, and strange Goals are welcome at [email protected].