Lab: Home Assistant, Frigate, and a Coral TPU that never touch a cloud

Most smart homes carry a dependency their owners never chose. The internet drops, the camera app shows a spinner, and the lights wait on a server somewhere else. For our family homestead, we're building the opposite, the way we'd build it for a client with unlimited paranoia: everything local.

A local-first smart home runs Home Assistant OS on hardware in the house, Frigate as the network video recorder, and a Google Coral edge TPU for on-device object detection. Cameras, presence, lighting, and property automations keep working with the internet unplugged, and no footage leaves the building.

We call the approach local-first by architecture, and it has five rules: no cloud dependency, no subscriptions, graceful degradation, privacy by architecture, and automations by meaning rather than motion. "By architecture" is the part that matters. A privacy setting is a promise someone can change later; footage that never leaves the building needs no promise. This is part one of an ongoing build log: what we're building and why.

Why local-first

Because a smart home that depends on the internet works on the internet's schedule, and because cameras record the most sensitive data a property produces. Local-first trades some convenience for four things we value more: outage resilience, privacy, no subscriptions, and longevity. Each is a design decision made up front, not a setting toggled later.

Outages. The internet is the component you control least. When a device's logic lives in a vendor's cloud, every cut cable, provider outage, or badly timed router update turns a light switch into a suggestion.

Privacy. Cameras record who arrives, when, and how often. That data belongs in exactly one place: a disk you can point at.

Subscriptions. Cloud cameras commonly put recorded history and smart alerts behind a monthly plan, and the features can change whenever the pricing does.

Longevity. A device whose brain lives on a vendor's server works exactly as long as that server does. Open-source software on hardware you own doesn't get switched off by someone else's roadmap.

The standard is the one we'd use for a client with unlimited paranoia: assume the internet fails at the worst moment, uploads leak, and vendors change terms. Then design so none of that matters. In practice it's ordinary engineering with the optimism removed.

The stack: Home Assistant OS, Frigate, and a Coral edge TPU

Five layers, all inside the property line: Home Assistant OS as the brain, Frigate as the network video recorder, a Google Coral edge TPU as the detection accelerator, IP cameras as the eyes, and a local network that carries everything. None of them needs the internet to do its job, and each can be swapped without redesigning the rest.

Home Assistant OS runs presence, lighting, climate, and property automations on hardware in the house. Its pitch is local control and privacy first, and every integration is labeled by how it talks to its device: local push, local polling, cloud polling, or cloud push. That label tells you before you buy whether a device survives an outage.

Frigate records the camera streams, runs object detection, and turns what it finds into events, with zones and masks marking which parts of each view matter. It hands detections to Home Assistant over MQTT and through its integration, so an automation can act on them.

The Coral edge TPU, a USB stick or an M.2 or Mini PCIe card, runs the detection model so the main processor doesn't have to.

The cameras serve video over the local network, usually as an RTSP stream, ideally with a low-resolution substream for detection and the main stream for recording. Wired Power over Ethernet is the sensible default: one cable, no Wi-Fi dependency.

The local network carries all of it; the internet uplink is one port on the router, not the foundation. The paranoid version, the one we'd specify for a client, gives cameras no route to the internet at all.

Software updates are the deliberate exception; nothing running waits on them. The last row is the honest one.

Recognition on a chip the size of a stamp

An edge TPU is a small accelerator that runs compact, quantized neural networks fast and at low power. Here it has one job: Frigate hands it regions of camera frames, and it returns what it sees (person, vehicle, animal) with a location and a confidence score. The main processor stays free for everything else.

Motion detection runs first as a cheap pixel check, and only regions where something changed are cropped and sent to the detector. That load scales with activity across every camera, which is why Frigate is designed to run detection on an accelerator or GPU rather than the CPU. One camera on a CPU is a demo. Several busy cameras make a space heater with opinions.

Three honest limits:

It's a detector, not a general model. It returns boxes and labels from a fixed list, and it doesn't describe scenes or run language models. Frigate's default model covers people, common vehicles, and several animals, though Frigate tracks only people until you add more.

It only runs models built for it. Models must be quantized to 8-bit integers and compiled for the Edge TPU; Coral's guide to running TensorFlow models on the Edge TPU explains the constraints. You pick from models that fit the chip, not the newest one you read about.

Availability has been uneven. People ask whether the Coral is discontinued; we don't make claims about Google's plans or current stock. Uneven availability is reason enough not to depend on one accelerator, and Frigate supports a range of detector hardware, including Intel graphics through OpenVINO and NVIDIA and AMD GPUs, plus CPU detection at much lower throughput.

Rule of thumb: Choose the detector last and keep it swappable. Zones, rules, and automations should never care which chip produced the label.

Automations by meaning, not motion

Motion is a pixel statistic; meaning is what moved, where, and when. A vehicle at the gate is not the same event as a deer in the orchard, and a branch moving in the wind is not an event at all. So automations trigger on labeled detections in zones, plus what Home Assistant knows: time of day and presence.

Plain motion alerts fail predictably: shadows, headlights, rain, and insects circling an infrared lamp all count. Alerts pile up, someone mutes them, and the cameras become a recording nobody watches. Motion answers the wrong question.

Frigate gives an automation three facts instead of one. What: the label, such as person, car, or dog. Where: the zone it entered, limited to the object types that matter there. How long: passing through or staying, via a zone's loitering time and stationary-object tracking, so a parked car doesn't raise an alarm every few minutes. Home Assistant adds the fourth, context: presence, time of day, and whatever state the house tracks.

Some illustrative rules:

• A vehicle enters the gate zone after dark: approach lights on, announce it inside.
• A person near the house at night while presence says it's empty: record, lights on, alert.
• An animal in the orchard zone: a snapshot for the morning, no alert. A deer is a note, not a 2 a.m. siren.
• Motion with no detected object: nothing at all.

The model only says what's in the frame; everything after that is deterministic automation, testable and boring, the shape we argue for in when plain software beats AI.

Rule of thumb: Motion is a reason to look, never a reason to act. The label, the zone, and the clock decide what happens.

Graceful degradation

Graceful degradation means deciding in advance what an outage is allowed to break. With the internet down, everything local keeps running: recording, detection, lights, climate, and property automations. What stops is anything that has to reach someone off the property, such as remote access and phone notifications, unless a separate path exists.

The first half is easy. Frigate's documentation says that after initial setup it can run fully offline: default detection models ship with it, and recording, playback, and live view stay local. Home Assistant keeps running every automation built on local integrations. The house doesn't notice the internet is gone.

The second half needs honesty. A push notification to a phone away from home is routed through the phone platform's push service, and remote viewing needs a way in from outside; both need the internet. If off-property alerts matter, the local-first answer is a second path that doesn't share the failure, such as a cellular backup connection. Without one, write down what happens instead: sirens and lights on the property still fire, events are recorded locally, and everything is waiting when the connection returns.

The same discipline applies to business workflows, and why automations break covers the properties that let a system degrade gracefully instead of silently.

Rule of thumb: Unplug the internet and walk the property. Everything that stops working is a dependency somebody else chose for you.

What this teaches about client work

The homestead is a scaled-down rehearsal for engagements where data can never leave the building. The habits carry over directly: run inference where the data already lives, keep other people's clouds off the critical path, and make privacy a property of the architecture rather than a paragraph in a policy. Local-first by architecture is how we'd describe both.

"The building" isn't always a building. For a client it might be a cloud region or a tenancy, but the question is the same: where does each piece of data go, and what enforces that? On the homestead, it's a detector on a chip and cameras that stream nowhere but the recorder. For a client, it might be a small open-weight model inside their environment, models consumed through their own cloud account, or, at the strict end, local models on their own hardware, up to air-gapped setups.

Field note: A professional-services firm in Ontario needed a document-intake assistant for files carrying personal client information under Canadian privacy law. Processing runs in a Canadian cloud region, and a small open-weight model inside it masks personal identifiers before any external model call. Same instinct as the cameras: decide the boundary, then make the architecture enforce it.

The intake side of that work has its own note: forms, PDFs, and voicemail full of data that can't leave the building. The homestead keeps us fluent in edge AI: quantized models, detector choice, latency, failure modes. It's why the build sits in the lab section of our work page, and why the same engineering runs through our AI and LLM engineering work.

When not to do this

If you don't want to maintain a server, don't build this. Cloud cameras are simpler: they install in minutes, update themselves, and someone else is responsible for the uptime. For plenty of households that's the right trade. Buy the subscription and be at peace; it's a reasonable engineering decision, not a moral failing.

What it costs:

• You are the operations team. Updates, backups, disk space, and release notes worth reading first; Home Assistant releases often and lists breaking changes each time.
• The cost is up front. A machine, storage, an accelerator, wired cameras, and the cable runs to reach them.
• Remote access is your job. Build a secure way in yourself, or pay for a service that does. A fine compromise, but a compromise.
• When it breaks, nobody else fixes it. That's the whole deal.

If "check the release notes before updating" sounds like a chore rather than a pastime, the subscription is the better product. We give clients the same advice about private AI: a system nobody will maintain is worse than a service somebody else maintains well.

Next in the series

The next installment goes deep on the vision pipeline: camera streams, detection settings, zones, and how a raw label becomes an event worth acting on. After that come automations that understand context, combining what was seen with where, when, and what the house already knows. Both are upcoming, and both will cover what failed as well as what worked.

Until then, run the first part yourself this week:

1. Do the unplug test and write down everything that stopped working.
2. For each item, look up the integration's IoT class in Home Assistant. Local push and local polling survive an outage; cloud polling and cloud push don't.
3. Point Frigate at one camera, draw two zones, and watch a week of detections before writing any automation.
4. Fill in the stack table above for your own house. The right-hand column is your real design document.

Local-first by architecture sounds like a slogan, but it works more like a checklist: keep the data where it's made, know in advance what breaks, and make the rules about meaning. The hardware will change. That habit is the part worth building.