---
title: Build an Offline AI Chatbot with Capacitor
description: Build an offline AI chatbot in a Capacitor app with the Capacitor LLM plugin — stream tokens into a chat UI, add a stop button and handle the 4k context window.
date:
created: 2026-11-02
updated: 2026-11-02
authors:
- robingenz
categories:
- Capacitor
- Guides
- SDKs
links:
- Capacitor LLM: sdks/capacitor/llm.md
faq: true
---
# Build an Offline AI Chatbot with Capacitor
An offline AI chatbot in a Capacitor app answers from the language model that ships with the phone, so it keeps working in airplane mode and sends nothing to a server. This guide builds the chat loop on top of the [Capacitor LLM plugin](../../sdks/capacitor/llm.md): a chat session with instructions, tokens streamed into a message list while the model writes, a stop button, and a rollover for the moment the conversation outgrows the model's context window of roughly 4,000 tokens. The plugin runs Apple Intelligence on iOS and Gemini Nano on Android, and it is part of [Capawesome Insiders](../../insiders/index.md).
**Key takeaways:**
- The Capacitor LLM plugin generates text on the device with Apple Intelligence on iOS 26 (iPhone 15 Pro or later) and Gemini Nano on Android devices with AICore, so the chatbot needs no network and no API key.
- `streamText(...)` emits one `textChunk` event per piece of the answer; append the chunks in arrival order to the last assistant message to render the reply while it is written.
- One generation runs per chat at a time and a second call rejects with `GENERATION_IN_PROGRESS`, so the send button stays disabled until the promise settles.
- `cancelGeneration(...)` rejects the pending promise with `GENERATION_CANCELED`; on Android a few chunks may still arrive afterwards, so drop chunks once the user pressed stop.
- Both models work with a context window of about 4,000 tokens; when a conversation gets long, summarize it and continue in a new chat seeded with that summary.
## What you will build
The chatbot in this guide is one screen: a scrolling list of user and assistant messages, a text field with a send button that turns into a stop button while an answer is being generated, and a status line that tells the user why the model is not available yet. All plugin calls live in one `ChatController` class written in plain TypeScript with an `onChange` callback, so the same class drives an Ionic Angular page, a React component or a Vue view. The code below shows that class piece by piece; each block adds members to the same class.
The guide starts after installation. To install the Capacitor LLM plugin, please refer to the [Installation](../../sdks/capacitor/llm.md/#installation) section in the plugin documentation. [Announcing the Capacitor LLM Plugin](./announcing-the-capacitor-llm-plugin.md) explains the seven availability statuses, the Android model download and the platform limits in detail, so this guide covers them only as far as the chat screen needs them.
Two types describe the state the UI renders:
```typescript
export interface ChatMessage {
role: 'user' | 'assistant';
text: string;
}
export interface ChatState {
messages: ChatMessage[];
generating: boolean;
}
```
## Check the model
The chat screen must not show an input field before the model can answer, because on most phones in use today it cannot. [`getAvailability()`](../../sdks/capacitor/llm.md#getavailability) never rejects and returns one of seven statuses. For the chat screen, three of them need their own UI: `available` enables the input, `downloadable` on Android shows a download button that calls [`downloadModel()`](../../sdks/capacitor/llm.md#downloadmodel) and renders the `downloadProgress` event, and `not-enabled` on iOS tells the user to turn on Apple Intelligence in Settings. Every other status hides the chat or switches to a fallback:
```typescript
import { Llm } from '@capawesome-team/capacitor-llm';
const prepareModel = async (onStatus: (text: string) => void): Promise => {
const { status } = await Llm.getAvailability();
if (status === 'downloadable') {
await Llm.addListener('downloadProgress', event => {
onStatus(`Downloading model: ${Math.round(event.progress * 100)}%`);
});
await Llm.downloadModel();
return true;
}
if (status === 'not-enabled') {
onStatus('Turn on Apple Intelligence in Settings to chat offline.');
return false;
}
if (status === 'not-ready') {
onStatus('The model is being prepared. Try again in a moment.');
return false;
}
return status === 'available';
};
```
The status changes while the screen is open, for example when the user comes back from Settings or when a download finishes. Register an `availabilityChange` listener next to this call and run `prepareModel(...)` again when it fires; the plugin only watches the status while a listener is attached.
## Create the chat
A chat is the unit of conversation context in the Capacitor LLM plugin. [`createChat(...)`](../../sdks/capacitor/llm.md#createchat) takes the `instructions` that set the assistant's role and tone for every turn, plus default generation parameters. The controller creates one chat per conversation and deletes it when the screen closes:
```typescript
import { Llm } from '@capawesome-team/capacitor-llm';
export class ChatController {
private chatId = '';
private instructions = '';
private state: ChatState = { messages: [], generating: false };
constructor(
private readonly options: {
instructions: string;
onChange: (state: ChatState) => void;
},
) {}
async start(): Promise {
this.instructions = this.options.instructions;
const { id } = await Llm.createChat({
instructions: this.instructions,
temperature: 0.5,
maxOutputTokens: 512,
});
this.chatId = id;
}
async dispose(): Promise {
await Llm.deleteChat({ id: this.chatId });
}
}
```
A `temperature` of `0.5` keeps answers on topic without making them repetitive, and it stays inside the Android range of `0.0` to `1.0`. The `maxOutputTokens` value caps each answer at 512 tokens, which is a few paragraphs; Android allows at most `4096`. Both values can be overridden per request later.
The chat holds native resources, and its history lives in memory only. On iOS, the plugin backs each chat with a native language model session. On Android, the system API has no multi-turn sessions, so the plugin keeps the history itself and includes it in every prompt. [`deleteChat(...)`](../../sdks/capacitor/llm.md#deletechat) frees the session and cancels a generation that is still running, so `dispose()` is safe to call while the model is mid-sentence.
## Stream the answer
Streaming is what makes an on-device chatbot feel responsive, because the model is slow by server standards. [How to Use Gemini Nano in a Capacitor App](./how-to-use-gemini-nano-in-a-capacitor-app.md) quotes Google's measurement of 11 tokens per second on a Pixel 9 Pro, so a three-paragraph answer takes well over ten seconds to finish. [`streamText(...)`](../../sdks/capacitor/llm.md#streamtext) emits a `textChunk` event for every piece of the answer and resolves with the complete text at the end. Each [`textChunk`](../../sdks/capacitor/llm.md#addlistenertextchunk-) event carries the `chatId` it belongs to and a `text` fragment.
The controller's `send(...)` method pushes the user's message and an empty assistant message, then appends each chunk to that last message:
```typescript
import { Llm } from '@capawesome-team/capacitor-llm';
export class ChatController {
private stopped = false;
async send(prompt: string): Promise {
if (this.state.generating) {
return;
}
this.stopped = false;
this.update({
generating: true,
messages: [
...this.state.messages,
{ role: 'user', text: prompt },
{ role: 'assistant', text: '' },
],
});
const listener = await Llm.addListener('textChunk', event => {
if (event.chatId === this.chatId && !this.stopped) {
this.appendToAnswer(event.text);
}
});
try {
await Llm.streamText({ chatId: this.chatId, prompt });
} catch (error) {
if ((error as { code?: string }).code !== 'GENERATION_CANCELED') {
throw error;
}
} finally {
await listener.remove();
this.update({ generating: false });
}
}
private appendToAnswer(text: string): void {
const messages = [...this.state.messages];
const last = messages[messages.length - 1];
messages[messages.length - 1] = { ...last, text: last.text + text };
this.update({ messages });
}
private update(patch: Partial): void {
this.state = { ...this.state, ...patch };
this.options.onChange(this.state);
}
}
```
Three details in this method come from the plugin's rules. The listener filters by `event.chatId`, because every chat's chunks arrive through the same event. The `generating` guard at the top exists because only one generation may run per chat; a second `streamText(...)` call rejects with `GENERATION_IN_PROGRESS` instead of queueing, so the UI disables the send button while `generating` is `true`. And the chunks are appended in arrival order, which the plugin documents as the way to reconstruct the response; the text in the resolved promise equals the concatenation, so the controller does not need it.
On the rendering side, write each message's text with `textContent`, never `innerHTML`, since the model's output is untrusted text. Re-render the last message on every change and scroll to the bottom only while the user has not scrolled up; an update every few hundred milliseconds that yanks the list down makes the answer unreadable.
## Add a stop button
A stop button is the second half of streaming. While `generating` is `true`, the send button becomes a stop button that calls [`cancelGeneration(...)`](../../sdks/capacitor/llm.md#cancelgeneration):
```typescript
import { Llm } from '@capawesome-team/capacitor-llm';
export class ChatController {
async stop(): Promise {
this.stopped = true;
await Llm.cancelGeneration({ chatId: this.chatId });
}
}
```
The pending `streamText(...)` promise rejects with `GENERATION_CANCELED`, which `send(...)` already swallows, so the partial answer stays in the message list and the input becomes usable again. The `stopped` flag covers the platform difference: on iOS the generation stops immediately, on Android cancellation is best-effort and a few more `textChunk` events can arrive after the call. With the flag set, those chunks are dropped instead of appearing after the user asked the model to stop.
## Respect the context window
Both system models work with a context window of about 4,000 tokens, and a chatbot is the one app shape that walks into that limit by design. The plugin's documentation puts the Android input limit under about 4,000 tokens and the iOS context window at about 4,096 tokens per session. Everything in a chat counts: the instructions, every user message, every answer, and the answer being generated. When a chat exceeds the limit, the generation rejects with `GENERATION_FAILED`, and the fix the documentation gives is to create a new chat.
The plugin exposes no token counter, so the controller budgets in characters. A common rule of thumb for English text is about four characters per token, which makes 12,000 characters a conservative stand-in for 3,000 tokens and leaves room for the next answer. Before each `send(...)`, the controller checks the budget and, if it is used up, asks the current chat for a summary, replaces the chat, and seeds the new one with that summary:
```typescript
import { Llm } from '@capawesome-team/capacitor-llm';
const MAX_CONTEXT_CHARACTERS = 12_000;
const MAX_PROMPT_CHARACTERS = 4_000;
export class ChatController {
private contextStart = 0;
private contextCharacters(prompt: string): number {
const messages = this.state.messages.slice(this.contextStart);
return (
this.instructions.length +
messages.reduce((sum, message) => sum + message.text.length, 0) +
prompt.length
);
}
private async rollOverIfNeeded(prompt: string): Promise {
if (prompt.length > MAX_PROMPT_CHARACTERS) {
throw new Error('The message is too long for the on-device model.');
}
if (this.contextCharacters(prompt) < MAX_CONTEXT_CHARACTERS) {
return;
}
const { text: summary } = await Llm.generateText({
chatId: this.chatId,
prompt: 'Summarize our conversation so far in under 150 words. Keep names, numbers and open questions.',
maxOutputTokens: 300,
});
await Llm.deleteChat({ id: this.chatId });
this.instructions = `${this.options.instructions}\n\nSummary of the conversation so far:\n${summary}`;
this.contextStart = this.state.messages.length;
const { id } = await Llm.createChat({
instructions: this.instructions,
temperature: 0.5,
maxOutputTokens: 512,
});
this.chatId = id;
}
}
```
Call `rollOverIfNeeded(prompt)` at the start of `send(...)`, before the user's message is pushed, and pass the pending prompt, because the budget has to hold the question as well as the history. The summary request uses [`generateText(...)`](../../sdks/capacitor/llm.md#generatetext) rather than streaming, because the user never sees it. The message list in the UI is untouched by the rollover; only the model's memory is compressed. `contextStart` remembers where the new chat's history begins, so the next check counts the summary-seeded instructions plus the messages since the rollover rather than the whole transcript, and the chat is not recreated on every send. A single prompt longer than `MAX_PROMPT_CHARACTERS` cannot fit even in a fresh chat, so the method rejects it before anything is sent; shorten it or ask the user to.
The rollover is proactive on purpose. Once a generation has already failed with `GENERATION_FAILED`, there is no room left to ask for a summary, and the error message is the only hint whether the cause was the context window, a safety filter or, on Android, the AICore quota. Catch that error in `send(...)`, show the message, and offer a "New conversation" action that calls `dispose()` and `start()`.
## Keep the transcript
The model forgets everything when the app restarts, because chats live in memory and are not restored. If the chatbot should pick up where it left off, your app has to store the transcript and rebuild the context. Store `messages` in the free `@capacitor/preferences` plugin for a single short conversation, or in the [Capacitor Secure Preferences plugin](../../sdks/capacitor/secure-preferences.md) when the content deserves encryption at rest, which the plugin's documentation lists as its purpose for chat data. On launch, create the chat with the stored summary or the last few turns in the instructions, the same way `rollOverIfNeeded(...)` does.
Voice fits the same loop. The [Capacitor Speech Recognition plugin](../../sdks/capacitor/speech-recognition.md) turns speech into the `prompt` for `send(...)`. The [Capacitor Speech Synthesis plugin](../../sdks/capacitor/speech-synthesis.md) reads the answer aloud once `streamText(...)` resolves. Read the complete text rather than the chunks, since chunks split inside words and sentences.
## FAQ
### How do you stream tokens from an on-device model into a chat UI?
Call `streamText(...)` of the [Capacitor LLM plugin](../../sdks/capacitor/llm.md) and register a `textChunk` listener before the call. Each event carries the `chatId` and a `text` fragment; filter by the chat, append the fragments in arrival order to the last assistant message, and re-render. Remove the listener when the promise settles. The promise resolves with the complete text, rejects with `GENERATION_CANCELED` after a stop, and only one generation may run per chat at a time.
### Does the chatbot work without an internet connection?
Yes. Apple Intelligence and Gemini Nano run on the device, so prompts and answers never leave the phone. The one step that needs connectivity is the initial Gemini Nano download on Android, which `downloadModel()` triggers when `getAvailability()` reports `downloadable`.
### Which devices can run the offline chatbot?
On iOS, iPhone 15 Pro or later with iOS 26 and Apple Intelligence turned on. On Android, [Gemini Nano-capable devices](https://developers.google.com/ml-kit/genai#prompt-device){:target="_blank"} with AICore, such as the Pixel 9 series. Other devices report `device-not-eligible` or `unavailable`, so ship a fallback or hide the feature.
### What happens when the conversation gets too long?
The generation rejects with `GENERATION_FAILED` once the chat exceeds the context window of about 4,000 tokens. Avoid that by budgeting characters as shown above: summarize the conversation in the current chat, delete it, and create a new chat whose instructions include the summary.
### Can I fall back to a cloud model on unsupported devices?
Yes. Keep the `ChatController` interface and swap the implementation to your own API when `getAvailability()` does not resolve with `available`. Tell the user which mode is active, since the privacy promise of an on-device chatbot does not hold for the cloud path.
### Is the Capacitor LLM plugin free?
No. It is part of the [Capawesome Insiders](../../insiders/index.md) subscription and installs from the Capawesome npm registry with your license key. If you only ship Android and only need Gemini Nano, the free [Capacitor ML Kit GenAI Prompt plugin](../../sdks/capacitor/mlkit/genai-prompt.md) wraps the same Google API.
## Related posts
- [Announcing the Capacitor LLM Plugin](./announcing-the-capacitor-llm-plugin.md)
- [How to Use Apple Intelligence in a Capacitor App](./how-to-use-apple-intelligence-in-a-capacitor-app.md)
- [How to Use Gemini Nano in a Capacitor App](./how-to-use-gemini-nano-in-a-capacitor-app.md)
## Try Capawesome Cloud
The chatbot only runs in a native build, and the iOS side needs Xcode 26 or later to compile against the Foundation Models framework. Capawesome Cloud builds and signs your Capacitor app for Android and iOS without a Mac, so a test build with the current Xcode is a few minutes away. [How to Sign & Build Capacitor Apps in the Cloud](./how-to-sign-and-build-your-capacitor-app-in-the-cloud.md) shows the steps.
[Try Capawesome Cloud Free](https://capawesome.io){ .md-button .md-button--primary }
## Conclusion
Build the gate before the chat: on most devices today the model is not available, and a chatbot that only appears on the phones that can run it is better than one that fails on the first message. Stream from the first version, since a ten-second spinner is what users remember, and put the character budget in before the first long test session rather than after the first `GENERATION_FAILED`. Test on an iPhone 15 Pro or newer and on a Pixel 9; the Android Emulator has no AICore, and the iOS Simulator only runs the model on an Apple silicon Mac with macOS 26 and Apple Intelligence enabled. For the platform details behind each step, [How to Use Apple Intelligence in a Capacitor App](./how-to-use-apple-intelligence-in-a-capacitor-app.md) covers iOS and the Gemini Nano guide above covers Android.
If you have any questions, join us on the [Capawesome Discord server](https://discord.gg/VCXxSVjefW){:target="_blank"}. To stay updated on the latest news, subscribe to the [Capawesome newsletter](https://capawesome.io/newsletter/){:target="_blank"}.