An AI teddy bear is not a model in a soft enclosure. It is a small networked computer, camera, audio system and battery product competing for the same limited space.
When people see Tell Me Ted respond to a book, it is natural to focus on the AI service. In practice, the quality of the experience depends just as much on camera alignment, microphone placement, speaker acoustics, power rails, firmware timing and the shape of the bear’s face.
This is a high-level tour of the current prototype architecture. It is deliberately not a production schematic. Component choices and board designs can still change as the product moves through testing, compliance and design-for-manufacture work.
The system at a glance
The simplest way to think about Ted is as five layers:
- Sensing: a camera sees a physical page and microphones capture speech.
- Local control: an embedded controller coordinates peripherals, buttons, state, power and networking.
- Connectivity: Wi-Fi connects the device to authenticated cloud services.
- Cloud intelligence: a reading session handles OCR, conversation and speech generation.
- Playback: decoded audio travels through the codec and amplifier to the speaker.
The device is intentionally not trying to run a large language model locally. The embedded processor handles sensing, transport and actuation. The cloud handles the computationally heavy AI work.
That division lowers the processing requirement inside the bear, but it introduces network, privacy and session-management responsibilities. There is no free architecture; there is only a different distribution of constraints.
It also defines what must keep working when the AI service is unavailable. The device still owns safe boot, battery handling, local controls, clear failure states and shutdown. Cloud intelligence should never become an excuse to make the physical product unable to look after itself.
The local controller
An embedded controller coordinates device state, camera and audio interfaces, connectivity and communication with protected cloud services.
It also has to manage local concerns that the model should never own: boot, update checks, battery state, button events, timeouts, error recovery and safe shutdown.
Compared with a phone or laptop, memory and flash are limited. Camera frames, audio buffers, network stacks and task stacks compete for the same resources. A feature that looks small in a product document may require a new buffer, a larger binary or more time with the radio active.
This is why embedded architecture rewards explicit budgets. “It compiles” is a necessary checkpoint, not evidence that the product is healthy.
The eye: a camera for the page
The camera exists for one reason: to understand the physical book in front of Ted.
Putting it in the eye is natural from a product perspective, but it creates engineering trade-offs. The angle must work when the bear is sitting. Focus and field of view must tolerate how families actually place books. The image must be clear enough for OCR without making every transfer unnecessarily large.
Then the plush construction joins the problem. Fur can obscure the lens. Stuffing can shift the alignment. A mounting point that works on an exposed board may move after the product is handled.
Our prototype feedback made this very concrete: an apparently cosmetic construction detail could partially block the camera and degrade the AI system. The fix is not a better prompt. It is stable mechanical placement, controlled assembly and a clear field of view.
The current realtime design keeps capture on demand. The camera is used for the first page, an explicit page-change event or a model-requested capture. It is not continuously polling the room.
The ears: audio capture in the real world
Voice demos are usually recorded on a desk in a quiet room. A family home includes televisions, conversations, movement, hard surfaces and children who may speak softly or unpredictably.
Microphone quality is only part of the answer. Placement, fabric, electrical noise and speaker feedback all matter. If the microphone is too close to the speaker, Ted hears itself. If fabric covers an acoustic path, the frequency response changes. If the firmware and codec disagree about sample rate, a cloud model cannot repair the missing information.
The precise microphone, audio and acoustic arrangement remains subject to hardware and enclosure testing. Echo control and noise handling are system properties, not a checkbox on one component.
The voice: codec, amplifier and speaker
Generated speech must travel back through the device, into the audio codec, through an amplifier and out of a speaker hidden inside a plush body.
The speaker has to be audible in a normal room without becoming distorted or uncomfortably loud. Orientation and acoustic channels can matter as much as nominal power. Stuffing absorbs sound. The enclosure changes resonance. Higher volume consumes more battery and can increase heat.
One prototype issue was simple to describe and difficult to solve well: Ted sounded acceptable in a quiet test but was not clear enough with ordinary household noise nearby. That is why representative testing matters. A product must work where it will live.
Bounded local interaction
Some simple activation and device-control functions can be handled locally rather than by the cloud conversational model.
That can reduce unnecessary network activity and help the device remain in a lower-power state. The exact boundary is still being refined, which is a recurring rule in prototype documentation: distinguish what the hardware could support from what the product currently uses.
Storage and power
The board includes local storage for essential device needs. The software design also reserves a supported path for maintenance and recovery after a device leaves the development bench.
Power is more complicated than choosing a battery capacity. The device must monitor battery state, manage charging, convert voltage for different rails and shut down predictably. Wi-Fi, camera use and speaker volume all affect runtime.
A plush enclosure makes charging and heat especially important. Production decisions require specialist electrical, battery and toy-safety review. A functional alpha is not evidence that those reviews are complete.
From prototype to product
Early prototypes indicate that sensing, audio, processing, connectivity and power can be integrated within the intended form factor and connected into a reading flow.
A production product must go much further. It needs repeatable assembly, tolerance control, durability, serviceability, safe access to charging, verified materials, electromagnetic and electrical testing, secure manufacturing and an update path that remains dependable after shipment.
That is why the “intelligence” of an AI teddy bear is not located in one model. It lives in the relationship between the page, the child, the hardware, the network and the cloud and in how the whole system behaves when one part is imperfect.
Next in the series: why we replaced a chain of AI calls with a persistent realtime session.
Tell Me Ted is an independent, pre-launch project that I work on outside my role at HubSpot. The views expressed here are my own and do not represent HubSpot.