LocalKeepGuide · 4-minute read
What is on-device AI and why it matters
For years, "using AI" meant sending your data to a data center. That is changing: today's chips can run language models inside the device itself. Here's how it works, and what it changes.
What "on-device" means
On-device (or local) AI means the language model — the "brain" that summarizes, rewrites, or extracts information — is installed inside your device and runs on its processor. Your text enters the chip, the result leaves the chip, and nothing crosses the internet. No API, no server, no middlemen: inference happens inches from your screen.
The difference from cloud AI isn't one of degree but of architecture: it's not that your data is "handled carefully" while traveling — it's that it doesn't travel.
What makes it possible now
Two things matured at once. On one side, the chips: Apple Silicon (A17 Pro onwards on iPhone, M series on iPad and Mac) includes a Neural Engine capable of running large neural networks at reasonable power. On the other, compact models: models of a few billion parameters, optimized and compressed, that fit in a phone's memory and handle focused text tasks very well.
Apple integrated both pieces into the system with Apple Intelligence: iOS, iPadOS, and macOS ship with a language model that apps can talk to directly. That's the one LocalKeep uses — inference costs nothing, for us or for you, and requires no account of any kind.
What it does well (and what it doesn't)
Honesty first: a local model of around 3 billion parameters is not a giant data-center model, and doesn't pretend to be.
The key is choosing the tool based on the data: for processing your own — often sensitive — text such as emails, minutes, and notes, the local model does exactly what's needed without the text ever leaving your machine.
Privacy by architecture, not by promise
This is the deeper change. Cloud AI privacy depends on policies: documents that can change and that you cannot audit. On-device AI privacy depends on the physics of the design: if the app makes no connections, there is nothing to leak, sell, or intercept in transit. And that, you can verify yourself — with airplane mode, a network monitor, or the iOS App Privacy Report. We show all three ways in "See for yourself".
For the full contrast with the cloud, continue with AI and privacy: what happens to your texts.