Starting again would not mean choosing a different model first. It would mean writing down a different set of questions.

How does a child begin? How do they continue? What happens when they turn a page, interrupt, go quiet or lose Wi-Fi? When is the camera active? Which behaviours must survive every refactor? How will a device already outside the building receive a safe fix?

We eventually answered many of those questions. We would have benefited from asking them before the service diagram became the product flow.

1. Design the natural interaction first

Our first architecture began with capabilities: transcription, vision, generation and text-to-speech. A better starting point would have been a storyboard of the complete reading session.

The child starts Ted, positions a book, hears the first page, turns it, speaks, waits, recovers from an unclear image and stops. The caregiver understands when the device is active and what happens offline.

Service boundaries should follow that loop. The user should never have to follow the boundaries of the APIs.

This would have led us towards a persistent session earlier and made the page-turn problem visible before it became a firmware and protocol retrofit.

2. Write down the non-negotiable invariants

The lifecycle lesson was our sharpest example. Simplifying the main device path was the right direction, but we had not made maintenance and recovery behaviour explicit enough in the task or tests.

If we started again, the repository would contain its invariant checks from the beginning:

  • preserve authenticated maintenance and recovery;
  • avoid blocking the core experience when a supporting service is unavailable;
  • keep service credentials away from the consumer device;
  • minimise and bound child data;
  • use sensors only when needed;
  • keep resource headroom; and
  • return to a clear safe state after failure.

Once an invariant matters, it belongs in CI and tests, not only in a person’s memory.

3. Test AI-generated changes on hardware earlier

Cloud tests and emulators gave us fast feedback and remain essential. They could not show enclosure effects, resource contention or child-turn behaviour in a normal room.

Coding agents can produce coherent cross-repository changes quickly. That makes early physical testing more important, not less. The cost of a large untested diff has fallen; the cost of believing it has not.

We would define a representative-device gate for every material firmware change and record the evidence beside the software tests.

4. Treat latency as a feature

We initially measured whether each stage worked. We should have measured the silence the child experienced.

Time to first useful audio, interruption behaviour and page-change success are product metrics. A service-level response time is supporting evidence.

The move to realtime came from that realisation. Starting there would have reduced the number of orchestration paths we later removed.

A continuous session is not a cure-all. It adds authentication, buffering and cleanup work. But it better matches the unit of experience: the reading session.

5. Label prototype shortcuts

Prototype decisions have a habit of becoming architecture.

A fixed capture window, a hard-coded session assumption or a broad log may be acceptable for proving a concept. Each shortcut should have an owner, a replacement condition and a deadline. Otherwise a successful demo can conceal operational risk.

We would maintain a visible list: “valid for prototype only”, “required before supervised testing” and “required before production”. That is more useful than a single percentage-complete number.

6. Establish privacy principles before collecting data

The current direction is designed to minimise persistence, keep sensor use on demand and prevent child content from becoming routine operational telemetry.

Those principles should have existed before any logging or history feature was discussed. Necessity, retention, access and deletion are architecture inputs.

The question is not “could this data improve the product?” Almost any data could. The question is “is the benefit necessary and proportionate for a child-facing reading companion?”

7. Avoid overbuilding orchestration

Our first system had an endpoint for each capability and a firmware path for each experience. It made sense during feasibility work and became harder to maintain as the product focus sharpened.

We would begin with fewer state owners, fewer hand-offs and one primary interaction. Other modes would have to earn their complexity with evidence.

This is a product lesson as much as an architecture lesson. A broad list of features can make a pitch sound complete while making the core experience worse.

8. Track embedded resources continuously

A build warning once showed that application headroom had become negligible. That should not have been a surprising moment.

Firmware size, memory at key transitions and large allocations should be continuous signals. Maintenance and recovery capacity should be release gates.

The device needs room for the next fix, not only the current feature.

9. Invest in acceptance tests earlier

Unit tests proved individual components. We needed more full-flow scenarios sooner:

  • start and complete a reading session;
  • turn pages verbally and with the button;
  • handle duplicate events;
  • recover from unclear images and network loss;
  • interrupt or remain silent;
  • exercise maintenance and recovery; and
  • exercise privacy and content-safety failures.

The expected result should describe what the child and caregiver observe, not only a successful status code.

What we would keep

We would keep the physical book at the centre.

We would keep a tangible companion rather than defaulting to another screen, while remaining honest about the manufacturing and privacy work that choice creates.

We would keep using ChatGPT and Codex aggressively. They expanded what a small team could explore and build. We would pair that speed with stronger repository invariants, staged tasks and independent review from the first day.

We would keep the narrow product principle: support the reading moment rather than build a general-purpose AI friend.

And we would keep documenting the mistakes. A polished retrospective can make good decisions look inevitable. They were not.

Where the project stands

As of this draft, early Tell Me Ted prototypes support the core technical-feasibility case. The current product direction uses a protected, session-based interaction with bounded sensor use and an explicit maintenance path. Natural page-turn and child-turn behaviour is still being refined.

The company still has important work ahead: structured parent research and supervised testing, product and acoustic refinement, formal safety and privacy review, compliance, manufacturing validation and go-to-market decisions.

That is not a triumphant ending or a failure. It is an honest point in a physical-product journey.

The biggest lesson is that faster tools do not remove the hard parts. They bring the hard parts forward. When code, plans and prototypes arrive more quickly, teams must become better at choosing the interaction, protecting the constraints and deciding what evidence is enough.

I will continue writing about those questions in Tell Me Ted and in the broader work of building reliable AI products.

Tell Me Ted is an independent, pre-launch project that I work on outside my role at HubSpot. The views expressed here are my own and do not represent HubSpot.