Most AI products begin with a text box. Ours begins with a child opening a book beside a teddy bear.
That difference sounds small, but it shaped almost every decision we have made while building Tell Me Ted.
The original idea came from a familiar question: how could a comforting physical object help a child through stories, routines and new experiences without making another screen the centre of attention? Early concepts explored milestone stories and personalised recordings. As we built and tested the idea, the product became more focused. The clearest version was not a teddy containing a library of digital stories. It was a teddy that could engage with the physical book already in front of the child.
That is the idea we are now pursuing: a conversational reading companion that can see a page, hear a child and respond through a speaker. The book remains the main object. The technology is there to support the moment around it.
This is also an important boundary. Tell Me Ted is intended to augment the adults already involved in a child’s reading life, not replace parents, carers or teachers. A good outcome is not a child spending more time talking to a machine. It is a child spending more time with books, stories and people.
Why a teddy rather than an app?
Children already know what to do with a teddy. They sit with it, talk to it and take it into familiar routines. There is no menu to learn and no blank prompt box asking them to invent the correct command.
An app would have been easier to build. It would also have changed the experience. The screen would compete with the page, notifications would sit one gesture away, and the device would still feel like a phone or tablet borrowed from an adult.
A physical companion creates a different kind of interaction. The child can look at the book while listening. The interface can be a voice, a clearly understood button and a small number of visible or audible states. The product can fit into an existing reading routine rather than demanding a new one.
Of course, choosing a teddy creates a formidable list of other problems. A plush enclosure is a terrible place for a camera, microphones, a speaker, batteries and heat-producing electronics. It turns millimetres of camera alignment, a fold of fur and the direction of a speaker into software-quality issues.
That is one of the recurring themes in this project: the product decision simplifies the child’s experience by moving complexity onto the team building it.
What Ted has to do
At a high level, the interaction appears simple. A child positions a book, starts a reading session and asks Ted to read. Ted captures the page when needed, sends it for text recognition, maintains the reading context and plays speech back through the speaker. The child can interrupt, say that they have turned the page, or use a physical button as a predictable fallback.
Behind that experience are several systems working together:
- an ESP32-S3-based embedded device coordinating the camera, audio, Wi-Fi and local controls;
- cloud services that create an authenticated reading session;
- a realtime relay that connects the device to the AI service without putting a long-lived model key in the bear;
- on-demand page capture and OCR;
- streaming audio back to the device; and
- update, rollback and recovery paths for firmware already installed on physical hardware.
The current work is still prototype work. Our feasibility phase produced three proof-of-concept units, including two functional alpha prototypes. They demonstrate that the core hardware and software idea is achievable within a plush product. They do not mean that certification, structured family testing, compliance work or production readiness is complete.
That distinction matters. A prototype proves that an interaction can happen. A consumer product must prove that it can happen repeatedly, safely and predictably in a family home.
The first architecture worked — and felt wrong
Our original software followed a logical chain. Capture audio. Transcribe it. Interpret the request and page. Generate a response. Convert the text to speech. Download the audio. Play it.
Each stage could be tested independently, and each API did the job it was given. But a child does not experience APIs independently. They experience the total silence between speaking and hearing an answer.
Latency accumulated at every boundary. State had to be passed across separate requests. Interruptions were awkward. The result could feel more like issuing a command than reading with a companion.
We have since moved the default MVP towards a persistent realtime reading session. Audio can be streamed, context can remain with one session owner, and the camera is requested only when the system needs a page. The current design is deliberately half-duplex: Ted does not record a child while playing audio. After speaking, it opens a bounded listening window for the next child turn.
This does not eliminate latency or solve every page-turn problem. It is a better architectural match for the interaction we want to test.
The page-turn problem
One tiny product detail has consumed a surprising amount of thought: what happens when the child turns the page?
Requiring “Hey Ted” on every page is technically tidy. It is also unnatural. Once a reading session has started, the child should be able to say “next page” or an equivalent phrase. If speech is unclear, a physical button should offer a reliable fallback.
That fallback is not an admission that the AI failed. It is good product design. A young child should not need to understand why the model missed an utterance, why the network paused or why the system lost context. They need a simple way to continue.
Our current prototype work treats both verbal and button-based page turns as events inside the active reading session. Camera capture remains on demand rather than continuous. This reduces unnecessary data collection, power use and ambiguity about when the device is looking at a page.
What building it has taught us
First, the desired interaction has to lead the architecture. A collection of individually sensible services can still create the wrong experience.
Second, hardware makes abstractions physical. Camera focus is affected by fabric. Speaker clarity is affected by stuffing and enclosure geometry. Audio buffering competes with camera frames and networking for memory. Firmware size affects whether a safe update and rollback path remains available.
Third, AI tools change how quickly a small team can explore the problem, but not who is accountable for the result. We use ChatGPT and coding agents extensively for planning, implementation and review. That has increased our output. It has also increased the amount of judgement, testing and system ownership required.
Finally, building for children changes the meaning of “good enough”. Privacy, visible device state, failure behaviour, parental control and age-appropriate responses must shape the system before the product is polished.
Building in public
This series will document the work as it actually happened: the useful decisions, the over-complicated architecture, the firmware warning that left almost no partition headroom, the AI-assisted refactor that made us strengthen our OTA safeguards, and the many ways a physical product exposes software assumptions.
There will be posts on hardware, realtime voice, privacy, product design, coding agents and manufacturing. Some parts are implemented. Some are experiments. Some are still unresolved. I will label them accordingly.
The goal is not to present a finished success story. It is to show what happens when experienced software builders use modern AI tools to create a physical consumer product from scratch — and discover that faster building only makes clear thinking more important.
Next in the series: how ChatGPT changed what our small founding team believed it could attempt.
I am a co-founder of Tell Me Ted. The views here are my own.