# agentlib — full content dump

Source: https://agentlib.neullabs.com
License: MIT (content) · agentlib is MIT
Generated: 2026-07-16T09:04:38.702Z

This file is a single-pass dump of the agentlib marketing site for
AI-search crawlers. The authoritative version is at https://agentlib.neullabs.com.

---


## Voice in, vision out, device automation: building a pocket assistant with agentlib

URL: https://agentlib.neullabs.com/blog/voice-vision-automation
Published: 2026-06-09
Tags: voice, vision, automation, accessibility, tutorial
Cluster: guide

Wire voice input, vision-based screen understanding, accessibility-driven app automation, and on-device routing together. The pocket-assistant pattern in agentlib.

The product everyone wants — a real on-phone assistant that listens, sees, and acts — is finally buildable. agentlib
makes the wiring straightforward.

## What we're building

A demo "pocket assistant" agent that can:

1. **Listen** to a spoken command.
2. **See** the user's screen if context is needed.
3. **Act** by driving other apps through accessibility / AppIntents.
4. **Speak** the result back.

All on-device where possible. Cloud as a fallback. Same code on iPhone and Pixel.

## The CLIs

agentlib ships first-party CLIs for each pillar:

- `/bin/voice.listen` — start STT, return transcript.
- `/bin/voice.speak` — TTS, plays back text.
- `/bin/vision.describe` — describe an image or current screen.
- `/bin/vision.ocr` — extract text from an image.
- `/bin/apps.list` — list installed apps the assistant can drive.
- `/bin/apps.<bundle>.<action>` — perform an action in another app (accessibility-driven on Android,
  AppIntents/SiriKit on iOS).

## The orchestrator

```dart
final assistant = AgentSpec(
  name: 'pocket',
  instructions: '''
You are a private on-device assistant. When the user speaks:
1. Transcribe with `/bin/voice.listen`.
2. Understand the intent.
3. If you need to see the screen, call `/bin/vision.describe`.
4. Carry out the request through `/bin/apps.*` or other CLIs.
5. Speak the result with `/bin/voice.speak`.

Prefer on-device tools. Ask consent before any /apps.* action.
''',
  model: ModelRoute.preferOnDevice(
    onDevice: [FoundationModelsProvider(), GeminiNanoProvider(), FllamaProvider(llm: a, modelName: 'Qwen3-1.7B-Instruct')],
    fallback: AnthropicProvider(apiKey: '<KEY>'),
  ),
  tools: [...virtualFilesystemTools(), shTool, voiceTool, visionTool, appsTool],
);
```

## A turn end-to-end

User: "Tell my partner I'll be late."

Inside agentlib, the model emits an `Sh` pipeline:

```dart
Sh("voice listen | jq -r .text > /scratch/intent.txt")
// → /scratch/intent.txt now contains "Tell my partner I'll be late."
```

Then the model interprets intent and dispatches:

```dart
Sh("contacts find 'partner' | jq -r .number | xargs whatsapp send --message 'Running late, see you soon'")
```

The `PermissionRequestHook` surfaces a consent sheet for WhatsApp. The user confirms. agentlib snapshots before
dispatch (because `whatsapp.send` is `highImpact: true`). The send happens. The model emits:

```dart
Sh("voice speak --text 'Done, told them you\\'re running late.'")
```

Everything on-device, except the consent dialog. One Flutter app. Five tool calls.

## Vision when needed

User: "What's on the screen right now?"

```dart
Sh("vision describe --screen | jq -r .summary")
// → "A QR code with the URL https://example.com/event"
```

`/bin/vision.describe --screen` captures the current foreground screen (iOS Screen Capture API or Android
MediaProjection — both require one-time consent), runs it through Foundation Models' vision capability on iOS 19+ or
through Google ML Kit + an LLM caption pass on Android, and returns a structured description.

For dense text, `/bin/vision.ocr` runs Apple Vision or ML Kit OCR and returns string output. The model can then run
regex / jq over it.

## Driving other apps

Through Android's accessibility service or iOS's AppIntents + SiriKit, the assistant can perform actions in other
apps. agentlib normalises this behind `/bin/apps.*`:

```dart
Sh("apps gmail compose --to 'partner@example.com' --subject 'Late' --body 'Be there in 30'")
```

This works because agentlib registers a per-app CLI that wraps the appropriate native API. On Android, that's an
accessibility-service walk; on iOS, that's an AppIntent invocation. The model sees one consistent CLI.

## Battery + privacy

Wire the lifecycle hooks once and forget about them:

```dart
hooks
  ..register(OnLowBattery((e, _) async {
    if (e.percent < 15) {
      return HookOutcome.replace(modelOverride: FllamaProvider(...tiny));
    }
    return HookOutcome.carryOn();
  }))
  ..register(PermissionRequestHook((req, ctx) async {
    final ok = await showConsent(ctx, req);
    return ok ? HookOutcome.carryOn() : HookOutcome.deny('user denied');
  }))
  ..register(PostToolUse((call, result, ctx) async {
    await audit.log(call: call, result: result);
    return HookOutcome.carryOn();
  }));
```

## What it feels like to use

Latency is the main concern. End-to-end target on an iPhone 15 Pro: ~1.5 s from end-of-speech to start-of-TTS for a
simple "send message" intent. That's two on-device model calls (intent classification + planning) plus a native API
dispatch. Tight enough to feel snappy.

## Patterns to steal

**Audio routing.** Use `flutter_sound` or `record` to capture audio. Pass the buffer to `/bin/voice.listen` rather
than recording inside the CLI — the CLI shouldn't own the mic stream.

**TTS interruption.** Wire a "user is speaking again" hook that cancels in-flight TTS. Otherwise the assistant will
talk over the user.

**Wake-word.** Optional but nice: ship a tiny on-device wake-word model (e.g. Porcupine) that triggers
`voice.listen` only when called. Saves battery and respects the no-always-listening promise.

## Read next

- [On-device routing](/blog/on-device-agents-flutter) — the model side of the equation.
- [21 hooks](/blog/hooks-guardrails) — the consent + audit primitives this article relies on.
- [Install](/install) — get the CLIs wired in 60 seconds.

---


## MCP on mobile: HTTP and WebSocket transports for Model Context Protocol servers

URL: https://agentlib.neullabs.com/blog/mcp-on-mobile
Published: 2026-06-06
Tags: mcp, websocket, http, interop, tools
Cluster: guide

agentlib speaks Model Context Protocol over both HTTP and WebSocket. MCP tools surface as /bin/mcp.<server>.<tool> CLIs, ready for Sh pipelines. Notifications stream live.

The Model Context Protocol (MCP) is the wire format the agent world has converged on for tool servers. Anthropic
introduced it; Claude SDK, OpenAI, and most agent frameworks support some flavour. agentlib supports MCP through
two transports: HTTP and WebSocket.

## What MCP gives you

An MCP server is an external process (or service) that exposes a tool surface to your agent. Tools, resources,
prompts — all defined in a JSON-RPC schema. Connect once, list tools, call them. agentlib treats MCP tools the
same as native tools: they go into the `/bin` mount, they show up in `Sh` pipelines, hooks fire on them.

## Connecting over HTTP

```dart
final mcp = await McpClient.connect(
  HttpMcpTransport(Uri.parse('https://weather.mcp.example.com/rpc')),
);
final bin = BinMount()..registerMcp(mcp, prefix: 'weather');
```

Every tool from the server is now available as `/bin/mcp.weather.<tool>`. So the model can compose them in `Sh`:

```dart
Sh("weather forecast --city Tokyo | jq -r '.high_celsius' | tee /scratch/high.txt")
```

HTTP transport is request/response: each tool call is one POST. Latency is the network round trip + the server's
processing time. Fine for most tools.

## Connecting over WebSocket

For tools that emit notifications (live subscriptions, long-running streams, watcher events), use WebSocket:

```dart
final mcp = await McpClient.connect(
  WebSocketMcpTransport(Uri.parse('wss://watcher.mcp.example.com/ws')),
);

// Tools work as before.
final bin = BinMount()..registerMcp(mcp, prefix: 'watcher');

// But you also get a notifications stream.
mcp.notifications.listen((n) {
  print('MCP notification: ${n.method} ${n.params}');
});
```

WebSocket transport keeps the connection open and multiplexes request/response over the same socket. That's the
right shape for:

- **File watchers** — the server emits an event when something changes.
- **Subscriptions** — the agent says "tell me when X happens".
- **Long-running tools** — periodic progress events, not a single response.
- **Real-time data** — quotes, sensor feeds.

## Listing tools

After connection, `mcp.listTools()` returns the server's declared tool surface:

```dart
final tools = await mcp.listTools();
for (final t in tools) {
  print('${t.name}: ${t.description}');
}
```

`registerMcp` calls this for you and translates each MCP tool spec into a native `Tool` object. The model never sees
the MCP layer; it sees CLIs in `/bin`.

## Authentication

MCP doesn't dictate an auth scheme. agentlib's transports accept bearer tokens, custom headers, or query-string
params:

```dart
final mcp = await McpClient.connect(
  HttpMcpTransport(
    Uri.parse('https://gh.mcp.example.com/rpc'),
    headers: { 'Authorization': 'Bearer ${ghToken}' },
  ),
);
```

For tokens that refresh, register a callback on the transport and rotate as needed.

## Reconnection

WebSocket transports include exponential-backoff reconnect by default. If the socket drops, the transport reconnects
and replays in-flight requests. You can override with a custom `ReconnectPolicy`:

```dart
final transport = WebSocketMcpTransport(
  uri,
  reconnectPolicy: ReconnectPolicy.fixed(delay: Duration(seconds: 5), maxAttempts: 10),
);
```

## Patterns

**Tool fanout.** Connect to 3–4 MCP servers (weather, GitHub, Slack, the user's email) and register them all under
different prefixes. The model can pipe between them: `weather forecast | gh issue create`.

**Local MCP for OS access.** Run a small native helper service that exposes OS-level capabilities (file system,
sensors, share sheet) as MCP over a Unix socket. The Flutter app connects via the local transport.

**Backend MCP for shared tools.** A team-shared MCP server hosts company-specific tools (CRM, internal search,
calendar). Every Flutter device connects; the surface is consistent everywhere.

## Hooks still fire

Hooks fire on MCP tool calls the same as on native tools. So `PreToolUse` can deny external sends, audit hooks log
every call, `PostToolUse` rewrites results — independent of which transport the tool came from.

## Read next

- [CLI-first agent design](/blog/cli-first-agent-design) — how MCP tools compose with Sh.
- [Primitives](/primitives) — the full surface.

---


## Suspend, resume, and push-resume: agents that survive the OS lifecycle

URL: https://agentlib.neullabs.com/blog/suspend-resume-push
Published: 2026-06-04
Tags: lifecycle, suspend, resume, push-notifications, ios, android
Cluster: guide

iOS backgrounds apps. Android kills processes under memory pressure. agentlib's Runner state is serialisable, lifecycle hooks fire on every transition, and push notifications can wake the agent.

Mobile apps don't run all the time. The OS backgrounds them. Under memory pressure, it kills them. The user reopens
the app two hours later expecting state to be there. An agent SDK built for a server doesn't think about any of this.
agentlib is built for the device, so it does.

## Serialisable RunState

Every `Runner` exposes its state as a `RunState` value that can be serialised to JSON:

```dart
final state = runner.serializeState();
await persistedStore.write('agent.state', jsonEncode(state.toJson()));

// later, after a cold start:
final json = jsonDecode(await persistedStore.read('agent.state'));
final state = RunState.fromJson(json);
final runner = Runner.resume(state, config: ...);
```

`RunState` captures everything needed to continue: messages, tool history, scratch state, snapshot timeline pointers,
the current step in the agent loop. It does *not* capture transient state like open HTTP connections — those reset on
resume, which is what you want.

## The lifecycle hooks

Three hooks cover the common transitions.

### `OnSuspend`

Fires when the OS tells the app it's about to be backgrounded. iOS gives you ~5 seconds; Android often more. Use it
to persist state:

```dart
hooks.register(OnSuspend((event, ctx) async {
  final state = ctx.serializeState();
  await persistedStore.write('agent.state', jsonEncode(state.toJson()));
  return HookOutcome.carryOn();
}));
```

If the agent is mid-stream, `OnSuspend` waits for the current event to finish (a token, a tool result) and then
fires. Long-running tool calls are checkpointed too.

### `OnResume`

Fires when the app foregrounds again *and the process is still alive*. Refresh tokens, re-fetch any cached state that
might be stale, re-establish HTTP/2 connections:

```dart
hooks.register(OnResume((event, ctx) async {
  await tokenStore.refreshIfNearExpiry();
  return HookOutcome.carryOn();
}));
```

### Cold start

If the OS killed the process while backgrounded, there's no `OnResume` — just a fresh launch. Check for persisted
state at app boot:

```dart
Future<void> bootAgent() async {
  final raw = await persistedStore.read('agent.state');
  if (raw != null) {
    final state = RunState.fromJson(jsonDecode(raw));
    return Runner.resume(state, config: cfg);
  } else {
    return Runner.start(cfg);
  }
}
```

## Push-resume

The killer feature for long-running work. The backend can send an APNs (iOS) or FCM (Android) push that carries a
checkpoint payload. When the app receives the push, agentlib resumes from the checkpoint.

```dart
// Backend → device push payload:
//   {
//     "aps": { "alert": "..." },
//     "agentlib.resume": "<base64 checkpoint>"
//   }

// In the app's notification handler:
final payload = PushPayload.tryParse(notification.userInfo);
if (payload != null) {
  await Runner.resumeFromPush(payload, config: cfg);
}
```

This unlocks patterns like:

- **Server-side long ops finish → wake the device.** A backend agent finishes a 5-minute report; pushes a resume to
  the device; the device's agentlib instance picks up where it left off and shows the user the result.
- **Scheduled refresh.** Every morning at 7 AM, the backend pushes a "refresh your daily summary" checkpoint.
- **Multi-device sync.** User starts a conversation on iPhone, picks it up on iPad. The "pickup" is a push-resume
  from a snapshot synced to the backend.

## Idempotent tools

When you resume, the agent might re-call a tool it had partially run before suspension. Mark idempotent tools:

```dart
final fetchWeather = tool(
  name: 'weather.get',
  description: 'Fetch the current weather.',
  schema: Schema.object({ 'location': Schema.string() }),
  idempotent: true,    // safe to re-call
  execute: (args, ctx) async { ... },
);
```

For non-idempotent tools (sending messages, charging cards), pair with snapshots — auto-snapshot before dispatch so
revert is always available.

## What this changes in your UI

Two patterns are worth wiring:

**Show "resuming" briefly.** On cold start, while you're rehydrating state, show a small "picking up where you left
off" indicator. It tells the user the conversation isn't being lost.

**Streaming reconnect.** If the user was watching tokens stream when the app backgrounded, the rehydrated UI should
show the final state of that turn (or continue streaming if the turn was incomplete). The `RunEvent` stream naturally
replays from the resumed state.

## Hooks for diagnostic visibility

Pair the lifecycle hooks with analytics:

```dart
hooks.register(OnSuspend((event, ctx) async {
  await analytics.log('agent.suspended', {
    'turn': ctx.turnIndex,
    'messages': ctx.messageCount,
    'inflight_tool': ctx.inflightTool,
  });
  return HookOutcome.carryOn();
}));
```

You'll learn a lot from these numbers in the first week of production. Where do users tend to background mid-stream?
Are there tool calls that frequently hang? The lifecycle data tells you.

## Read next

- [Snapshots, revert, fork](/blog/snapshots-time-travel) — the storage primitive underneath suspend/resume.
- [21 hooks](/blog/hooks-guardrails) — the full hook surface.

---


## 21 hooks: the most fine-grained agent loop in any SDK

URL: https://agentlib.neullabs.com/blog/hooks-guardrails
Published: 2026-06-02
Tags: hooks, guardrails, lifecycle, audit
Cluster: guide

agentlib has more interception points than any other agent SDK. PreToolUse, PostToolUse, OnSuspend, OnLowBattery, OnNetworkChange — guardrails without prompt hacking.

If you've built an agent in production, you know the failure mode: the model "forgets" the rules in the system
prompt, calls a destructive tool, and there's nothing between the call and the user. agentlib's hook system is the
thing between.

There are 21 hooks. Each one is a deterministic interception point that runs in Dart, not in the prompt. Each one
returns a `HookOutcome` that tells the agent loop what to do next: carry on, deny, or replace with a different action.

## The full set

**Loop hooks** — fire around tool calls and the model turn.

- `PreToolUse` — before any tool dispatches. Can mutate args, deny, replace tool.
- `PostToolUse` — after a tool returns successfully. Can mutate result.
- `PostToolUseFailure` — after a tool throws or returns an error. Can retry, replace, propagate.
- `PermissionRequestHook` — when a tool with capabilities (microphone, camera, contacts) is about to run. Surface
  a native consent sheet.

**Subagent hooks** — fire around subagent dispatch.

- `SubagentStart` — before a subagent spawns. Log, mutate prompt, deny.
- `SubagentMessage` — each message the subagent emits.
- `SubagentStop` — when a subagent finishes.

**Compaction hooks** — fire around message compaction.

- `PreCompact` — before compaction runs. Can supply a custom compaction strategy.
- `PostCompact` — after compaction. Can review the new state.

**Lifecycle hooks** — fire on OS events.

- `OnSuspend` — app is about to background. Persist state.
- `OnResume` — app is in foreground again. Reload, refresh tokens.
- `OnLowMemory` — OS is warning you about memory. Trim caches, evict skills.
- `OnLowBattery` — battery dropped below threshold. Route to a smaller model.
- `OnNetworkChange` — connectivity changed. Re-route, retry, queue.

**Snapshot hooks**.

- `OnSnapshot` — a snapshot was taken. Mirror to remote, log.
- `OnRevert` — a revert happened. Reset UI state, log analytics.

**Shell hooks**.

- `OnShellParse` — an Sh pipeline was parsed but not yet dispatched. Audit the AST.

**Misc**.

- `UserPromptSubmit` — user submitted a prompt. Sanitise, prepend instructions.
- `StopHook` — agent finished (terminal stop). Final cleanup.
- `NotificationHook` — agent wants to surface a notification.
- `OnHandoff` — agent is about to hand off to another agent.

## What each does in practice

A few load-bearing examples.

### `PreToolUse` — deny a destructive action

```dart
hooks.register(PreToolUse((call, ctx) async {
  if (call.tool == 'whatsapp.send' && !userHasAcceptedTos) {
    return HookOutcome.deny('TOS not accepted');
  }
  if (call.tool == 'gmail.send' && (call.args['to'] as String).endsWith('@example.com')) {
    return HookOutcome.deny('External domain restricted');
  }
  return HookOutcome.carryOn();
}));
```

### `PermissionRequestHook` — native consent sheet

```dart
hooks.register(PermissionRequestHook((req, ctx) async {
  final granted = await showCupertinoModalPopup<bool>(
    context: ctx.uiCtx!,
    builder: (_) => ConsentSheet(capability: req.capability, reason: req.reason),
  );
  return granted == true
      ? HookOutcome.carryOn()
      : HookOutcome.deny('user denied');
}));
```

### `OnLowBattery` — re-route the model

```dart
hooks.register(OnLowBattery((event, _) async {
  if (event.percent < 20) {
    return HookOutcome.replace(
      modelOverride: FllamaProvider(llm: tinyAdapter, modelName: 'Qwen3-0.6B'),
    );
  }
  return HookOutcome.carryOn();
}));
```

### `OnNetworkChange` — drop cloud when offline

```dart
hooks.register(OnNetworkChange((event, ctx) async {
  if (!event.isReachable) {
    return HookOutcome.replace(modelOverride: ctx.config.model.withoutCloud());
  }
  return HookOutcome.carryOn();
}));
```

### `PostToolUse` — audit log

```dart
hooks.register(PostToolUse((call, result, ctx) async {
  await audit.log(
    runId: ctx.threadId,
    tool: call.tool,
    args: call.args,
    ok: result is ToolResultOk,
    at: DateTime.now(),
  );
  return HookOutcome.carryOn();
}));
```

## How they compose

Hooks fire in registration order. The first hook that returns `HookOutcome.deny(...)` short-circuits the chain — no
further hooks run, the tool doesn't dispatch. A `HookOutcome.replace(...)` mutates the call and continues. A
`HookOutcome.carryOn()` just observes and passes through.

This means *hook order matters*. Register policy hooks first (consent, deny lists) and observability hooks last
(audit, analytics).

## When to reach for a hook vs. a prompt instruction

Prompts give the model *intent*. "Don't send messages to external domains." The model usually does what the prompt
says — usually. Hooks give *guarantees*. The agent loop in Dart enforces them regardless of what the model decides.

Rule of thumb:

- **Prompt** for soft guidance: tone, style, framing.
- **Hook** for hard rules: deny-lists, consent, audit, anything safety-critical.

## Reference

- All hooks are sealed classes — pattern-match on them in your registry.
- `HookOutcome` is a sealed sum: `carryOn`, `deny(reason)`, `replace(...)`.
- Hooks run on the host thread. Keep them quick; if you need long-running work, dispatch it asynchronously and let
  the hook return early.

## Read next

- [Snapshots, revert, fork](/blog/snapshots-time-travel) — the other safety net.
- [Suspend, resume, push-resume](/blog/suspend-resume-push) — lifecycle hooks in production.

---


## Snapshots, revert, fork: time-travel debugging for mobile AI agents

URL: https://agentlib.neullabs.com/blog/snapshots-time-travel
Published: 2026-05-30
Tags: snapshots, fork, revert, safety, sqlite
Cluster: guide

agentlib's snapshot API is public, content-addressed, and SQLite-backed. snapshot(), revert(), fork() — and an auto-snapshot before any highImpact tool call so users can always undo.

Mobile agents drive devices. They send messages, edit files, post events to calendars. That's terrifying without an
undo button. agentlib's snapshot system is the undo button.

## The three operations

```dart
// 1. Take a snapshot. Returns the SHA-256 id.
final id = await ctx.snapshot(label: 'before-send');

// 2. Roll back to a previous snapshot.
await ctx.revert(id);

// 3. Branch — keep the original, start a parallel timeline from this point.
final forked = await ctx.fork(id, model: OpenAIProvider(apiKey: '<KEY>'));
```

That's the public API. Three operations. Content-addressed by SHA-256 of the serialised state, so identical states
produce identical ids — the store dedupes automatically.

## Auto-snapshots before high-impact actions

The most useful pattern: mark any destructive tool as `highImpact: true`, and agentlib auto-snapshots before dispatch.

```dart
final sendWhatsApp = tool(
  name: 'whatsapp.send',
  description: 'Send a WhatsApp message.',
  schema: Schema.object({
    'contact': Schema.string(),
    'message': Schema.string(),
  }),
  capabilities: const [Capability.accessibility],
  idempotent: false,
  highImpact: true,    // ← agentlib snapshots before this dispatches
  execute: (args, ctx) async { ... },
);
```

Now every time the model calls `whatsapp.send`, agentlib snapshots first. If the send goes wrong (typo'd message, wrong
contact, broken tool result), `revert()` rolls everything back. The external side effect persists (the message was
sent), but the agent's *understanding* of state goes back to before the send — so the next user turn doesn't proceed
as if everything succeeded.

For truly safe undo, pair snapshots with idempotency or transactional tools. Some `whatsapp.send` implementations
support recall-within-7-minutes; agentlib can detect a snapshot pre-recall and offer the user a one-tap "undo".

## Fork: branching to a different model

`fork(id, model: ...)` is the killer feature for evaluation and recovery. You don't lose the original timeline; you
start a parallel one from any past point with whatever changes you want.

Common uses:

**Try a different model.** "This Claude turn looked off — let me fork and see what GPT-4o would have done."

```dart
final alt = await ctx.fork(lastTurnId, model: OpenAIProvider(apiKey: '<KEY>'));
```

**Try a different system prompt.** "The instructions might be wrong — fork with a tweaked system prompt and see if
the result is better."

```dart
final alt = await ctx.fork(lastTurnId, instructions: 'New instructions...');
```

**Speculative execution.** During a long run, fork to a smaller model for an A/B comparison. Discard whichever is
worse.

## The store

In production you want `SqfliteSnapshotStore`:

```dart
final store = SqfliteSnapshotStore(database: db);
final ctx = RunConfig(snapshotStore: store, ...);
```

It's SQLite-backed. Each snapshot is a row keyed by SHA-256; large blobs (messages, scratch state) live in a
content-addressed blob table with foreign keys + cascade deletes. Calling `store.vacuum()` walks the blob table and
deletes orphans.

For tests, use `InMemorySnapshotStore`. Same shape, all in RAM, gone when the test finishes.

## Compaction interplay

When the message history gets too long, the agent loop runs *compaction* — summarises old turns into a shorter
representation. agentlib emits `PreCompact` and `PostCompact` hooks so you can intervene.

Compaction takes a snapshot before it runs. If the compaction summary is bad (lost important detail), the user
sees it and can `revert` to the pre-compaction snapshot.

## Hooks for snapshots

Two hooks fire around snapshots:

- **`OnSnapshot`** — when a snapshot is taken. Log it; sync to remote; mirror to backup.
- **`OnRevert`** — when a revert happens. Re-render UI; reset side effects; emit analytics.

```dart
hooks.register(OnSnapshot((event, _) async {
  await remoteBackup.upload(event.id, event.payload);
  return HookOutcome.carryOn();
}));
```

## Practical patterns

**Auto-snapshot every N turns.** Use `RunConfig(autoSnapshotEveryNTurns: 5)` if you want a baseline cadence in
addition to high-impact pre-snapshots.

**Named user snapshots.** Let users name a state ("save as 'planning v2'"). Surface them in a UI for revert + fork.

**Diff for time-machine UI.** `store.diff(idA, idB)` returns a structured diff between two snapshots — useful for a
"what changed?" view.

## What snapshots don't do

They don't undo external side effects. They don't time-travel device state — if the agent sent a notification, the
user got the notification. They don't replace consent: a `PermissionRequestHook` to confirm a destructive tool is
still the right primary safety net.

They do give you a deterministic recovery story when an agent goes off the rails — and that's the difference between
shipping autonomous agents that drive a device and shipping a "tap to retry" assistant.

## Read next

- [21 hooks: guardrails for the agent loop](/blog/hooks-guardrails) — pair snapshots with consent + audit hooks.
- [Suspend, resume, push-resume](/blog/suspend-resume-push) — snapshots are the same primitive that survives the OS.

---


## Subagents at scale: parallel and background dispatch on mobile

URL: https://agentlib.neullabs.com/blog/subagents-parallel-background
Published: 2026-05-28
Tags: subagents, parallel, background, orchestration
Cluster: guide

agentlib's subagents run sync, in parallel, or in the background — each with its own context window. Research + draft simultaneously; index in the background; survive backgrounding.

Single-agent loops are fine for short conversations. Once an agent has to research, plan, draft, and review, you want
specialisation: separate agents for separate jobs, each with its own context window so deep work doesn't pollute the
parent's.

agentlib ships three dispatch modes that cover the common patterns.

## The three modes

**Sync.** "Do this one thing and tell me the result." Parent blocks until the subagent finishes.

```dart
final result = await SubagentRunner.runOne(
  parent: ctx,
  def: researcherDef,
  prompt: 'Research the latest on Vision Pro UX patterns.',
);
```

**Parallel.** "Do these N things at once; come back when all are done." Each subagent runs in its own context, the
parent gathers results once they're all in.

```dart
final results = await SubagentRunner.runParallel(
  parent: ctx,
  defs: [researcherDef, drafterDef],
  prompts: ['research X', 'draft article about X'],
);
```

**Background.** "Start this; I'll come back later." Useful for indexing, long-running ingestion, or any work that
shouldn't block the user's current interaction.

```dart
final handle = SubagentRunner.dispatch(
  parent: ctx,
  def: indexerDef,
  prompt: 'Index the user\'s notes folder.',
);
// ... user does other things ...
final result = await handle.await();
// or: handle.onComplete((result) { ... });
```

## Why separate context windows

Two reasons. First, **cost**: subagent context doesn't multiply into the parent. A research subagent can read 200
documents and the parent never sees them — only the subagent's final structured summary. Second, **clarity**: the
parent's prompt stays focused on its job ("orchestrate the writing"), and each subagent's prompt stays focused on its
own.

## Defining a subagent

A subagent is described by an `AgentDefinition`:

```dart
final researcherDef = AgentDefinition(
  name: 'researcher',
  description: 'Researches a topic deeply from the workspace.',
  instructions: '''
You are a research assistant. Given a topic, search the workspace for relevant
notes, read the top 5, and emit a JSON summary with key points and citations.
''',
  tools: ['fs.read', 'fs.grep', 'fs.glob'],   // allow-list — researcher can only read
  modelOverride: AnthropicProvider(apiKey: '<KEY>'), // use a stronger model for research
);
```

Key fields:

- **`tools`** is an allow-list. Researcher gets fs read access; not write.
- **`modelOverride`** lets you pick a different (often cheaper or stronger) model.
- **`instructions`** is the subagent's system prompt. Independent from the parent's.
- **`permissionMode`** can be `ask`, `allow`, or `deny` — fine-grained capability gating.

## Dispatch from a model turn

Most subagent calls happen via the `Task` tool that the orchestrator agent has access to:

```dart
Sh("task dispatch researcher --prompt 'Vision Pro UX best practices' --mode parallel")
```

The model emits this; agentlib looks up the `researcher` definition; spawns it; waits (or returns immediately if
mode is background); injects the result back as a `ToolResult`.

## A research-and-draft example

```dart
final orchestrator = AgentSpec(
  name: 'writer',
  instructions: '''
You orchestrate writing. When a draft is requested:
1. Dispatch `researcher` in parallel with `outline`.
2. Wait for both to finish.
3. Dispatch `drafter` with the outline + research as input.
4. Return the draft.
''',
  model: AnthropicProvider(apiKey: '<KEY>'),
  subagents: [researcherDef, outlineDef, drafterDef],
);
```

The orchestrator's system prompt explains the workflow. The subagent definitions live alongside.

## Background work

Background dispatch is useful when:

- **Indexing**: walk the user's documents, build an embedding index. Don't block the chat.
- **Periodic refresh**: every hour, summarise new emails. The summary lives in `/memory`; the chat agent reads from
  it instantly.
- **Speculative work**: while the user is typing the next prompt, prefetch a likely response.

Background subagents survive parent suspension. If the app gets backgrounded while a subagent is running, the
subagent's state is checkpointed; on `OnResume`, agentlib resumes it. If the OS kills the process entirely, the
subagent picks up from its last snapshot.

## Hooks for subagents

Three hooks fire around subagents:

- **`SubagentStart`** — before dispatch. Log it, deny it, mutate the prompt.
- **`SubagentMessage`** — each message the subagent emits. Useful for live progress UI in the parent.
- **`SubagentStop`** — when the subagent finishes. Log the duration; check for errors.

```dart
hooks.register(SubagentStart((event, ctx) async {
  await analytics.log('subagent.start', { 'def': event.def.name });
  return HookOutcome.carryOn();
}));
```

## When subagents are wrong

Subagents are heavyweight. Each one is a full agent loop. If a piece of work is *procedural* (do this then this then
this), a [skill](/blog/progressive-disclosure-skills) is lighter. If it's a one-shot tool call, just call the tool
directly.

Reach for a subagent when:
- The work needs its own model (cheaper, smaller, specialised).
- The work needs its own context (deep reading, lots of intermediate state).
- The work needs to run in the background.
- The work is parallel with other work the orchestrator is doing.

## Read next

- [Suspend, resume, push-resume](/blog/suspend-resume-push) — how background subagents survive the OS.
- [Snapshots](/blog/snapshots-time-travel) — checkpoint your orchestrator before a risky parallel dispatch.

---


## Progressive-disclosure skills: 30 skills in 600 tokens

URL: https://agentlib.neullabs.com/blog/progressive-disclosure-skills
Published: 2026-05-25
Tags: skills, context, orchestration, claude-code
Cluster: guide

Skills are markdown bundles that load lazily. The orchestrator sees only name + description until invocation, so a 30-skill catalog costs ~600 tokens instead of 12 K.

There's a quiet cost to giving an agent lots of tools or workflows: each one consumes context. With ten skills at
~400 tokens each, you've burned 4 K tokens before the model has done anything. With thirty, you're at 12 K. On a
phone, where you're running a 1.7B–8B on-device model with a 4 K–8 K context, this is a problem.

Skills solve it with progressive disclosure.

## What the model sees

At the start of a run, the orchestrator builds a system prompt that includes a *skills index* — just the `name`
and `description` for each skill. That's typically 20–30 tokens per skill. With 30 skills, that's ~600 tokens.

```md
# Available skills

- `summarise-email` — Summarise an email into 3 bullet points.
- `triage-notifications` — Score notifications by urgency.
- `compose-reply` — Draft a reply in the user's voice.
- `extract-receipt` — Parse a photo of a receipt into structured fields.
- ... (26 more)
```

That's it. The bodies — the actual step-by-step instructions, the tool usage hints, the example outputs — are nowhere
in context. They sit on disk at `/workspace/.skills/<name>/SKILL.md`.

## What happens on invocation

When the model decides it wants a skill, it emits a tool call:

```dart
Sh("skill invoke summarise-email")
// or directly
{ "tool": "skill.invoke", "args": { "name": "summarise-email" } }
```

The `skill.invoke` tool reads the bundle, injects its body as a new system-message turn, and the model continues with
the skill's instructions front-and-center. After the skill finishes (often a single turn or a few), the body is
*evicted* from context — only the skill's *output* is retained.

So a 30-skill catalog uses 600 tokens at the index level, plus 400 tokens for whichever single skill is active.
1 K tokens total instead of 12 K.

## Writing a skill

A skill is a directory with at least a `SKILL.md`:

```md
---
name: summarise-email
description: Summarise an email into 3 bullet points.
---

# Steps

1. Read the email body via `/bin/gmail.show --id $1`.
2. Extract the subject, sender, and the 3 most important sentences.
3. Emit JSON: `{ subject, from, bullets: [...] }`.

# Tools

- `/bin/gmail.show`
- `/bin/jq`

# Example output

\`\`\`json
{
  "subject": "Q3 board prep",
  "from": "ceo@example.com",
  "bullets": [
    "Q3 numbers come in Tuesday — board deck due Wednesday morning.",
    "Marketing's launch slipped — replace section 4 with a roadmap slide.",
    "Investor list needs updating — add the two new participants from June."
  ]
}
\`\`\`
```

That's all that's needed. The orchestrator picks it up next time it scans `/workspace/.skills/`.

## Versioning and discovery

Skills are just files. Ship them in the app bundle, sync them from a server, let the user write their own. agentlib
doesn't care; it just scans the directory.

For app-shipped skills, copy them into the workspace on first launch:

```dart
final dir = Directory('${docsDir.path}/workspace/.skills/summarise-email');
await dir.create(recursive: true);
final asset = await rootBundle.loadString('assets/skills/summarise-email.md');
await File('${dir.path}/SKILL.md').writeAsString(asset);
```

For user-written or remotely-synced skills, just write the files; the next agent run sees them.

## Skill composition

A skill's body can call out to other skills:

```md
# Steps

1. If the email is from the user's manager, invoke `triage-urgent` first.
2. Otherwise summarise per the standard pattern.
3. If the summary is longer than 3 bullets, invoke `compress-bullets`.
```

The orchestrator handles the dispatch. This lets you build a small tree — one "front-door" skill that knows when to
delegate.

## When skills are wrong

Skills are great for *procedures* the agent does often. They're not the right shape for:

- **Data** (use the VFS — `/workspace/<file>`).
- **Tools** (register a CLI in `/bin/`).
- **System prompt material that should always be in context** (just put it in the orchestrator's `instructions`).
- **Per-user memory** (use `/memory` — the SQLite-backed mount).

If you find yourself loading the same skill on every turn, it's probably not a skill — it's an instruction.

## Reference

- The default skills directory is configured on `RunConfig`. Override with `RunConfig(skillsDir: ...)`.
- `SkillRegistry` is the in-memory cache. Call `registry.reload()` if you wrote a new skill mid-run.
- Skill invocation goes through the same hook pipeline as any tool call — `PreToolUse` and `PostToolUse` fire on
  `skill.invoke`.

## Read next

- [Subagents at scale](/blog/subagents-parallel-background) — when you need full agents, not just procedures.
- [CLI-first agent design](/blog/cli-first-agent-design) — the Sh idiom that pairs with skills.

---


## agentlib vs Claude SDK vs OpenAI Agents SDK vs Mastra vs LangChain: which to use

URL: https://agentlib.neullabs.com/blog/agent-sdk-comparison
Published: 2026-05-22
Tags: comparison, agent-sdk, claude-agent-sdk, openai-agents-sdk, mastra, langchain
Cluster: cornerstone

A practical decision tree for picking an agent SDK in 2026. The short answer: pick by where your agent runs — server, edge, or device. Here's the long answer.

There are now five credible AI agent SDKs you might pick in 2026, and most of them are good at very different things.
This article is a practical decision tree, written by people building agentlib but trying to be honest about where the
others shine.

## The contenders

**Claude Agent SDK** (Anthropic). Python and TypeScript. Cloud-only (Claude). Probably the most mature primitive set —
subagents, skills, snapshots, hooks. Tied tightly to Claude as the model. MIT licensed.

**OpenAI Agents SDK** (OpenAI). Python and JavaScript. Cloud-only (OpenAI + OpenAI-compatible). Handoff-first design.
Apache-2.0.

**Mastra** (Mastra Labs). TypeScript only. Node / Bun / Deno / Cloudflare Workers. Cloud-only, multi-provider. Agent
loops, workflows, MCP, RAG. MIT.

**LangChain** (LangChain Inc.). Python + JavaScript. Originally a chain-and-RAG framework that's grown agent
primitives. Multi-provider. MIT.

**agentlib** (Neul Labs). Dart only. Runs in Flutter apps natively — iOS, Android, macOS. On-device + cloud,
unified by `ModelRoute`. MIT.

## The 30-second decision

| Constraint                                  | Pick                           |
|---------------------------------------------|--------------------------------|
| Agent runs in a Flutter app                 | **agentlib**                   |
| Agent runs natively in an iOS or Android app | **agentlib**                   |
| Needs on-device LLM inference               | **agentlib**                   |
| Privacy-sensitive: no data leaves the device | **agentlib**                   |
| Python / TS backend, Claude is the model    | **Claude Agent SDK**           |
| Python / JS backend, OpenAI is the model    | **OpenAI Agents SDK**          |
| TypeScript backend, multi-provider          | **Mastra**                     |
| Edge runtime (Workers, Vercel)              | **Mastra** (TS edge support)   |
| Heavy RAG over a custom corpus              | **LangChain** + a vector store |
| Already invested in LangChain               | **LangChain** + LCEL           |

If your situation hits more than one row, you might want two of these together (see "Combining them" below).

## The longer version

### Native mobile + on-device

This is agentlib's territory. None of the others run inside an iOS app sandbox without a Node subprocess. None of the
others have a `FoundationModelsProvider` or `GeminiNanoProvider`. The closest is Mastra, which is TypeScript and could
in theory run in a webview embedded in a mobile app — but that's not really "native to the device" in the way Flutter
+ Dart is. If your agent lives on a phone, pick agentlib.

### Server-side, Claude-only

Pick the Claude Agent SDK. The primitive set is mature (subagents, skills, snapshots, hooks), the Anthropic team
maintains it, and you get features in lockstep with the model. The only reason to choose anything else here is if you
need explicit multi-provider support — Anthropic's SDK is, reasonably, Claude-first.

### Server-side, OpenAI-only, handoff-heavy

Pick the OpenAI Agents SDK. The handoff primitive is the cleanest of any framework — "this conversation is now the
specialist's problem" — and the SDK was designed around it. Less primitive surface than the Claude SDK (no
snapshots, fewer hooks) but it's right for the workflows it targets.

### Server-side, TypeScript, multi-provider

Pick Mastra. It's the cleanest TypeScript-first agent framework with multi-provider support. Good MCP integration,
solid workflow primitives, edge-runtime support out of the box. Younger than the Anthropic SDK so the surface evolves
fast — pin versions.

### Heavy RAG, pipelines, classifiers

Pick LangChain or LlamaIndex. The agent loop in LangChain has matured but the framework's center of gravity is still
retrieval + chains. If most of your work is "retrieve, transform, output", LangChain is more idiomatic than treating
the same pipeline as an "agent". If you go this way, expect to pin versions tightly — the API has churned a lot.

## Capability matrix

| Capability                            | agentlib | Claude SDK | OpenAI Agents | Mastra | LangChain |
|---------------------------------------|:--------:|:----------:|:-------------:|:------:|:---------:|
| Native iOS + Android                  | ✓        | ✗          | ✗             | ✗      | wrapped   |
| On-device models                      | ✓        | ✗          | ✗             | ✗      | wrapped   |
| Subagents (parallel/background)       | ✓        | sync       | via handoff   | ✓      | via LCEL  |
| Progressive-disclosure skills         | ✓        | ✓          | ✗             | ✗      | ✗         |
| Snapshots (fork / revert)             | ✓        | partial    | ✗             | partial| ✗         |
| Hooks                                 | 21       | ~10        | guardrails    | some   | ✗         |
| Unix-shell tool surface               | ✓        | bash       | ✗             | ✗      | ✗         |
| MCP (HTTP + WS)                       | ✓        | HTTP       | ✗             | HTTP   | ✗         |
| Suspend / resume + push-resume        | ✓        | ✗          | ✗             | partial| ✗         |
| Multi-provider out of box             | 7        | 1          | 1             | many   | many      |
| License                               | MIT      | MIT        | Apache-2.0    | MIT    | MIT       |

The honest read: agentlib has the deepest primitive surface specifically for mobile. The Claude SDK has the deepest
primitive surface for server-side Claude. The OpenAI Agents SDK is the cleanest handoff-first design. Mastra is the
strongest TypeScript story. LangChain is the strongest retrieval story.

## Combining them

We do this a lot, and we think you should too.

**Pattern A: cloud backend + on-device agent.** Run a Claude Agent SDK or Mastra pipeline on the server. Run agentlib
in the Flutter app. The device calls the backend for heavy reasoning; the agentlib agent handles UI orchestration,
on-device classification, offline fallback. They interoperate over JSON-RPC tool calls or MCP.

**Pattern B: orchestrator on the backend + sub-orchestrator on the device.** Claude Agent SDK's orchestrator spawns
mobile work; the agentlib runtime picks it up via push-resume; results stream back via webhooks or MCP notifications.

**Pattern C: retrieval pipeline (LangChain) + agent on top (Claude SDK or agentlib).** Use LangChain for what it's best
at (retrieval + transformation), and surface the results to an agent loop in whichever SDK fits your runtime.

## Migration notes

**From Claude Agent SDK to agentlib** (for mobile use):
- `AgentSpec` ≈ Anthropic's agent descriptor.
- Subagents map almost 1:1 (with parallel + background added).
- Skills shape is identical — same markdown bundle layout.
- Hooks are a superset — agentlib's 21 includes everything in the Claude SDK plus mobile-lifecycle hooks.
- Tool schemas: agentlib uses a Zod-style `Schema.object(...)`; mapping is mechanical.

**From OpenAI Agents SDK to agentlib**:
- The OpenAI handoff is `AgentSpec.handoffs` in agentlib — the v1.1.0 addition.
- Guardrails map onto `PreToolUse` + `PostToolUse` hooks.
- Tool schemas: same idea, different syntax.

**From LangChain to agentlib**:
- LCEL chains map onto a custom orchestrator agent + subagents. Don't expect a 1:1 port.
- LangChain retrievers become tools the agent calls.

## What we'd actually pick

Each row of this matrix is a real product we've worked on or talked to teams shipping. Honest picks:

- **Slack-style bot, server-side:** Claude Agent SDK + MCP.
- **Customer-support web app:** Mastra + Postgres + RAG.
- **Internal-tools agent that drives spreadsheets:** OpenAI Agents SDK + handoffs.
- **iOS / Android app with AI assistant features:** agentlib.
- **Voice-driven mobile assistant:** agentlib (voice CLIs are first-party).
- **Privacy-sensitive mobile assistant:** agentlib with `onDeviceOnly`.
- **Search-over-docs product:** LangChain (or LlamaIndex) + a vector DB.

The decision really is mostly about where the agent *runs*. Pick that first; everything else follows.

---


## agentlib 1.1.0: MCP WebSocket, SQLite snapshots, in-loop handoffs

URL: https://agentlib.neullabs.com/blog/v1-1-0-release
Published: 2026-05-22
Tags: release, mcp, websocket, snapshots, handoffs
Cluster: announcement

Three big additions in 1.1.0 — full-duplex MCP via WebSocket, a content-addressed SQLite snapshot store, and OpenAI-style in-loop handoffs between agents.

agentlib **1.1.0** is out. Three additions worth highlighting.

## 1. MCP WebSocket transport

The 1.0 release shipped HTTP MCP — fine for request/response tools, no good for tools that emit notifications.
1.1.0 adds a `WebSocketMcpTransport` with full-duplex JSON-RPC over a single socket, plus a notifications stream.

```dart
final mcp = await McpClient.connect(
  WebSocketMcpTransport(Uri.parse('wss://watcher.mcp.example.com')),
);

mcp.notifications.listen((n) {
  print('MCP: ${n.method} ${n.params}');
});
```

The transport handles reconnect with exponential backoff out of the box. In-flight requests replay after reconnect.
See [MCP on mobile](/blog/mcp-on-mobile) for the longer write-up.

## 2. SQLite-backed snapshot store

Snapshots have been a public API since 1.0, but the default store was in-memory only. 1.1.0 ships
`SqfliteSnapshotStore` — content-addressed, deduped via SHA-256, with cascade-deleting blob storage.

```dart
final store = SqfliteSnapshotStore(database: db);
final cfg = RunConfig(snapshotStore: store, ...);
```

Identical `RunState` produces the same content hash, so successive snapshots that differ by only a few tokens share
storage. Call `store.vacuum()` to garbage-collect orphan blobs. See [Snapshots](/blog/snapshots-time-travel).

## 3. In-loop handoffs

1.0 supported subagents (sync, parallel, background). 1.1.0 adds **handoffs** — the OpenAI Agents SDK pattern where
one agent declaratively transfers control to another on the same conversation thread.

```dart
final triage = AgentSpec(
  name: 'triage',
  instructions: 'Classify the request, then hand off.',
  model: AnthropicProvider(apiKey: '<KEY>'),
  handoffs: [
    Handoff(agent: billingAgentDef, when: 'request is about billing'),
    Handoff(agent: techAgentDef, when: 'request is about a technical issue'),
  ],
);
```

When the triage agent decides to hand off, a `HandoffEvent` fires (you can register `OnHandoff` to log or veto), and
the receiving agent picks up the conversation with the same message history and a fresh system prompt. Use the
`continueHandoff(handle)` helper to drive the next turn.

This is different from a subagent: a handoff *replaces* the agent for subsequent turns. A subagent is dispatched and
returns.

## Other changes

- New `Handoffs` concept guide in the repo's documentation.
- Loop guard now exposes a callback (`OnLoopGuard`) so you can surface "I'm in a loop" to the user.
- `Sh` builtins gained `wc -c` and `cut -f`.
- `FllamaProvider` includes a Phi-4 tool-call parser.
- Bug fix: `OnLowBattery` debounces correctly under rapid battery state changes.

## Upgrade

```yaml
dependencies:
  agentlib: ^1.1.0
```

Then `flutter pub upgrade agentlib`. No breaking changes from 1.0 — all additions are backward-compatible.

## What's next

The 1.2 milestone focuses on:

- **Streaming MCP tool results** — partial output for long-running MCP tools, surfaced as multiple `ToolResult` events.
- **Per-tool budget** — declare token / time budgets per tool, enforced by the loop.
- **Better Phi-4 + Gemma 3 parsers** — closing the on-device tool-call reliability gap.

If you have feature requests, file them on [GitHub](https://github.com/neul-labs/agentlib/issues). If you ship
something with agentlib, tell us — we love hearing about it.

---


## CLI-first agent design: why agentlib's Sh mini-shell matters

URL: https://agentlib.neullabs.com/blog/cli-first-agent-design
Published: 2026-05-18
Tags: sh, cli, tool-design, claude-code, agent-design
Cluster: cornerstone

Frontier LLMs are most fluent in Unix idioms — Read/Grep/Edit and cmd | jq | xargs. agentlib's Sh gives the model that surface inside a sandbox. Here's why it matters.

If you read a leaked Claude system prompt, an OpenAI tool spec, or a high-performing agent transcript, you'll notice
the same pattern: the model is fluent in *Unix*. `Read file.md`. `Grep TODO src/`. `cmd --flag value | jq ... > out`.

That's not an accident. It's the densest token-efficient idiom in the training data. Frontier LLMs have ingested a
giant corpus of `bash`-shaped commands and they reach for them naturally. An agent SDK that hands the model JSON-RPC
tool calls is asking it to translate out of its first language every turn.

agentlib's Sh mini-shell hands the model that idiom directly — sandbox-safely.

## What Sh is

Sh is a strict, sandboxed Unix shell that lives inside the Dart process. It parses:

- **Pipes** (`|`) — connect one command's stdout to the next's stdin.
- **Redirects** (`>`, `>>`) — write stdout to a VFS path.
- **Variables** (`$VAR`, `$1`, `$@`) — set via `RunConfig.env` or the `set` builtin.
- **Command substitution** (`$(cmd)`) — splice a command's stdout into another command.
- **Quoting** (single and double, escapes) — same as POSIX shell.

It does *not* parse:

- **`exec`** — no real binaries are spawned.
- **`&`** — no backgrounded subprocesses.
- **`eval`** — no dynamic shell construction.
- **`source`** — no script inclusion.
- **`function` definitions** — no shell-defined macros.

Every "command" Sh understands is a registered Dart function in the `/bin` VFS mount. Pipes connect streams in memory.
Redirects write to VFS paths. There's no escape to the host filesystem, no arbitrary code execution, no syscall surface
beyond what your registered CLIs explicitly expose.

## What it unlocks

Compare an agent SDK that hands the model JSON-RPC tools:

```json
// Turn 1: fetch contacts
{ "tool": "contacts.grep", "args": { "pattern": "Mom" } }
// Turn 2: extract number
{ "tool": "json.extract", "args": { "field": ".number" } }
// Turn 3: send message
{ "tool": "whatsapp.send", "args": { "contact": "...", "message": "Late" } }
```

That's three round trips, three tool calls, three opportunities for the model to lose state.

With Sh, the same intent is one model turn:

```dart
Sh("contacts grep 'Mom' | jq -r .number | xargs whatsapp send --message 'Late'")
```

The shell pipeline executes atomically. The hooks (`PreToolUse`, `PostToolUse`) still fire on each underlying CLI, so
audit logs and consent sheets work the same way. But the model emits one tool call, not three.

## How it composes with hooks

Sh isn't a bypass of the hook system. The execution path is:

1. Model emits `Sh("...")` tool call.
2. `OnShellParse` hook fires with the parsed AST. App code can inspect, mutate, or deny.
3. For each command in the pipeline, the normal tool dispatch path runs: `PreToolUse` → `PermissionRequestHook` (if
   the command declares capabilities) → tool exec → `PostToolUse`.
4. Streams connect: command N's stdout feeds command N+1's stdin.
5. Final stdout is returned to the model as `ToolResult.ok`.

So if `whatsapp.send` is `highImpact: true`, agentlib still snapshots before it runs. If `contacts.grep` requires the
`contacts` permission, the `PermissionRequestHook` still surfaces. The pipeline is sugar; the safety is not.

## Why frontier LLMs are good at this

Two reasons.

**First, training distribution.** The shape `cmd --flag value | jq -r .field | xargs cmd2 --arg` shows up everywhere
in GitHub, in Stack Overflow, in shell tutorials, in CI scripts. Models have ingested *a lot* of it. They reach for
`grep`, `jq`, `xargs`, `head`, `tail` without prompting.

**Second, density.** A three-step pipeline is one short string. The same intent expressed as JSON-RPC calls is three
nested objects, with field names that have to be remembered and called precisely. Less room for the model to drift.

## The builtins

agentlib ships twelve POSIX-ish builtins out of the box:

| Builtin | Purpose                                    |
|---------|--------------------------------------------|
| `cat`   | concatenate files                          |
| `echo`  | emit text                                  |
| `head`  | first N lines                              |
| `tail`  | last N lines                               |
| `cut`   | extract columns by delimiter               |
| `sort`  | sort lines                                 |
| `uniq`  | de-duplicate adjacent lines                |
| `xargs` | turn stdin into args for the next command  |
| `jq`    | JSON query (the inline subset most agents use) |
| `test`  | string / numeric tests for conditionals    |
| `ls`    | list directory contents                    |
| `wc`    | counts                                     |

That's enough to express most of what an agent needs to do with text. For the rest, the model emits a direct tool call.

## First-party CLIs

On top of the builtins, agentlib ships CLIs that drive the device:

| CLI             | Sub-commands                                              |
|-----------------|-----------------------------------------------------------|
| `fs`            | `read`, `write`, `edit`, `grep`, `glob`, `ls`             |
| `whatsapp`      | `send`, `list`, `read`                                    |
| `gmail`         | `send`, `list`, `read`, `search`                          |
| `calendar`      | `list`, `create`, `update`, `delete`                      |
| `browser`       | `open`, `tabs`, `screenshot`, `dom`                       |
| `phone`         | `call`, `sms`                                             |
| `notifications` | `list`, `dismiss`                                         |
| `vision`        | `describe`, `ocr`, `detect`                               |
| `task`          | `dispatch`, `await`, `list`                               |
| `skill`         | `invoke`, `list`                                          |

Each one is a registered CLI in the `/bin` mount. Each one is sandbox-safe. Each one composes in Sh pipelines.

## When *not* to use Sh

Sh is excellent for composition. It's overkill for one-shot tool calls. If the model needs to call one tool, it can
still emit a direct JSON-RPC tool call — there's no penalty. agentlib's prompt template encourages Sh for two-or-more
step pipelines and direct tool calls for one-shot work.

## Where to read next

Snapshots are the natural companion to Sh — when a pipeline includes a destructive CLI like `whatsapp.send`, you want
a snapshot before the dispatch so the user can undo. Read [snapshots, revert, fork](/blog/snapshots-time-travel) next.

Or jump to [/install](/install) and try a pipeline yourself. The five-line example below works as soon as you wire
a `Sh` and a `whatsappCli()` into your agent.

```dart
final bin = BinMount()
  ..register(fsCli())
  ..register(whatsappCli());
final vfs = Vfs()..mount('/workspace', WorkspaceMount(docsDir))..mount('/bin', bin);
final sh = Sh(binMount: bin, vfs: vfs, builtins: defaultShBuiltins());

final agent = AgentSpec(
  name: 'assistant',
  instructions: 'Use Sh for any multi-step tool work.',
  model: AnthropicProvider(apiKey: '<KEY>'),
  tools: [...virtualFilesystemTools(), sh.asTool()],
);
```

---


## On-device AI agents in Flutter: Foundation Models, Gemini Nano, llama.cpp

URL: https://agentlib.neullabs.com/blog/on-device-agents-flutter
Published: 2026-05-15
Tags: on-device, flutter, foundation-models, gemini-nano, llama-cpp, privacy
Cluster: cornerstone

agentlib runs agents on the phone. Four on-device providers — Apple Foundation Models, Gemini Nano, MediaPipe, Fllama — behind one ModelRoute.preferOnDevice([...]) call.

The big agent SDKs all assume cloud. They were built when on-device LLMs were a research curiosity. They're not
anymore. Apple Foundation Models ships in iOS 19. Gemini Nano ships in AICore. llama.cpp runs Qwen3-1.7B at 30+ tok/s
on a 2024 iPhone. MediaPipe runs Gemma 3 on most modern Android phones. agentlib treats them as first-class.

This article walks through the four on-device providers, the routing primitive that ties them together, and the
patterns that make on-device-first feasible in production.

## The four providers

**`FoundationModelsProvider` (iOS 19+).** Wraps Apple's on-device foundation model — the one that powers Apple
Intelligence. It supports native tool calling through Apple's `Tool` protocol. Available on iPhone 15 Pro and newer,
M-series iPads, and Apple Silicon Macs running macOS 26+.

```dart
final fm = FoundationModelsProvider();
// Auto-detects availability. Falls through silently when used in a ModelRoute.
```

**`GeminiNanoProvider` (Pixel 8+, S24+).** Wraps Google AICore. Tool calling works on `nano-v3`. agentlib detects
device support at startup; on unsupported devices, it returns capabilities indicating it can't serve the request, and
`ModelRoute` moves on.

```dart
final nano = GeminiNanoProvider(modelVariant: 'nano-v3');
```

**`MediaPipeProvider` (Android 12+).** Uses Google AI Edge LlmInference for Android. Supports a growing list of
models — Gemma 3, Phi-4, Llama 3.2, Qwen — that you ship as a `.task` file in your app bundle or download on first
launch. Cross-vendor: it runs on most modern Androids, not just Pixel.

```dart
final mp = MediaPipeProvider(
  modelAssetPath: 'assets/models/gemma-3-1b-it.task',
);
```

**`FllamaProvider` (any device).** Wraps Fllama, the Flutter package around llama.cpp. Any GGUF works. agentlib ships
family-aware tool-call parsers for Qwen, Llama 3.2, Phi-4, and Gemma 3 — each formats tool calls slightly differently
and agentlib normalises them.

```dart
final llama = FllamaProvider(
  llm: fllamaAdapter,
  modelName: 'Qwen3-1.7B-Instruct',
);
```

## One routing call

The on-device picture is dramatically simpler than it looks because `ModelRoute` handles dispatch:

```dart
final agent = AgentSpec(
  name: 'offline-helper',
  instructions: 'You are a private on-device assistant.',
  model: ModelRoute.preferOnDevice(
    onDevice: [
      FoundationModelsProvider(),                   // iOS 19+
      GeminiNanoProvider(modelVariant: 'nano-v3'),  // Pixel 8+, S24+
      FllamaProvider(llm: fllamaAdapter, modelName: 'Qwen3-1.7B-Instruct'),
    ],
    fallback: AnthropicProvider(apiKey: '<KEY>'),
  ),
);
```

`ModelRoute` queries each provider's `capabilities` getter — does it stream? Does it support tools? Vision? Thinking?
— and picks the first provider that can serve the current request. If none can, it falls back to the cloud provider
you specified. The agent code never branches.

## When to prefer on-device

You almost always want on-device first for these workloads:

- **Classification** (is this email urgent? is this photo a receipt?). 1.7B model is great at this.
- **Summarisation** of short-to-medium documents (under ~2 K tokens of context).
- **Intent routing** (which subagent should handle this prompt?).
- **On-text editing** (rewrite this sentence in formal English).
- **Voice command parsing** (extract intent + slots from a spoken sentence).
- **Anything privacy-sensitive** where the user's data shouldn't leave the device.

You probably still want cloud for these:

- **Long-context reasoning** (multi-document analysis, long codebases).
- **Vision tasks beyond what the local model handles** (Foundation Models has vision; Gemini Nano doesn't).
- **Multi-step tool use that needs sophisticated planning** (frontier cloud models still plan better than 1.7B locals).
- **First-class structured output** with complex schemas. (On-device is catching up here, but still patchy.)

## Patterns that work

**Pattern 1: tier on-device → cloud.** Most apps end up here. Smaller on-device for cheap routine work; cloud for the
hard 20%.

**Pattern 2: on-device only for sensitive flows.** Health, finance, kids, legal. Use `ModelRoute.onDeviceOnly([...])`
— it throws rather than silently falling back to cloud.

```dart
final medicalAgent = AgentSpec(
  name: 'medical-helper',
  instructions: 'You are an on-device medical assistant. No PHI leaves the device.',
  model: ModelRoute.onDeviceOnly([
    FoundationModelsProvider(),
    GeminiNanoProvider(),
    FllamaProvider(llm: fllamaAdapter, modelName: 'Qwen3-1.7B-Instruct'),
  ]),
);
```

**Pattern 3: battery-aware re-routing.** Below 20%, switch to a tinier model. The `OnLowBattery` hook handles this in
five lines.

```dart
hooks.register(OnLowBattery((event, ctx) async {
  if (event.percent < 20) {
    return HookOutcome.replace(
      modelOverride: FllamaProvider(llm: tinyAdapter, modelName: 'Qwen3-0.6B'),
    );
  }
  return HookOutcome.carryOn();
}));
```

**Pattern 4: network-aware re-routing.** On a flaky cell signal, prefer on-device.

```dart
hooks.register(OnNetworkChange((event, ctx) async {
  if (!event.isReachable) {
    // Drop cloud from the route entirely.
    return HookOutcome.replace(modelOverride: ctx.config.model.withoutCloud());
  }
  return HookOutcome.carryOn();
}));
```

## Capability gating in detail

Each provider exposes a `capabilities` getter that returns a `ModelCapabilities` value:

```dart
class ModelCapabilities {
  final bool streaming;
  final bool toolCalling;
  final bool vision;
  final bool thinking;
  final int maxContextTokens;
}
```

`ModelRoute` matches the *current request's* needs against this. So a vision agent will skip Gemini Nano
(text-only) and try Fllama (if you've configured a vision GGUF) or fall through to cloud. A simple text-summarisation
request will use whichever on-device provider's available first.

## Bring your own runtime

If none of the four providers fits, implement `ModelProvider` yourself:

```dart
class MyRuntimeProvider implements ModelProvider {
  @override
  ModelCapabilities get capabilities => const ModelCapabilities(
    streaming: true,
    toolCalling: true,
    vision: false,
    thinking: false,
    maxContextTokens: 8192,
  );

  @override
  Stream<ProviderEvent> stream(ProviderRequest req) async* {
    // Wrap your runtime's HTTP or FFI API here.
  }
}
```

That's the full contract. Two methods. Wrap any ONNX runtime, any local HTTP API, any FFI binding — agentlib's
agent loop will treat it the same as a built-in provider.

## What changes in your UI

Nothing, ideally. The streaming event API is the same regardless of which provider serves the request. Your
`StreamBuilder<RunEvent>` works the same whether the model is in your bundle or in Anthropic's data centre.

What you can add, optionally:

- A small "running on-device" badge in your UI when `ctx.activeProvider.kind == ProviderKind.onDevice`. Users like
  knowing.
- Per-provider analytics so you can see how often the cloud fallback kicks in.
- A user setting: "Prefer on-device even if slower" → flip to `onDeviceOnly` mode.

## Performance reference points

These are rough numbers on consumer 2024–2025 devices. Your mileage varies.

| Provider             | Device          | Tokens/sec | Cold start |
|----------------------|-----------------|-----------:|-----------:|
| FoundationModels     | iPhone 15 Pro   | ~40        | ~200 ms    |
| Gemini Nano (v3)     | Pixel 8 Pro     | ~30        | ~150 ms    |
| MediaPipe + Gemma 3 1B | Pixel 7       | ~22        | ~500 ms    |
| Fllama + Qwen3 1.7B  | iPhone 15 Pro   | ~28        | ~1.2 s     |
| Fllama + Qwen3 0.6B  | iPhone XR (2019)| ~14        | ~2.4 s     |

In every case the time-to-first-token is much better than a cloud round trip, especially on cell networks.

## Bottom line

Running models on the device used to be a science project. It isn't anymore. Foundation Models and Gemini Nano are
ready for production today; Fllama covers everything else with a 1–2 GB ship-cost; MediaPipe gives you the Android
breadth. agentlib treats them uniformly so your agent code doesn't care which one ran.

Install with one pubspec line, plug a `preferOnDevice([...])` into your `AgentSpec`, and ship a private agent that
works on a plane.

---


## Why agentlib is the only mobile-native AI agent SDK

URL: https://agentlib.neullabs.com/blog/mobile-native-agent-sdk
Published: 2026-05-12
Tags: mobile, flutter, agent-sdk, claude-agent-sdk, openai-agents-sdk
Cluster: cornerstone

Every other agent SDK assumes a Node or Python runtime that doesn't fit inside an iOS app sandbox. agentlib is pure Dart, with native bridges for Apple Foundation Models and Gemini Nano.

If you've tried to ship an AI agent inside a Flutter app, you've hit the same wall. The major agent SDKs — Anthropic's
[Claude Agent SDK](https://docs.claude.com/en/api/agent-sdk/overview), OpenAI's
[Agents SDK](https://github.com/openai/openai-agents-python), Mastra, LangChain — are all built for a server. They
assume Python or Node. They spawn MCP subprocesses. They expect to live in a process you can `kill -9`.

None of that ports cleanly to an iOS app sandbox. Or, more bluntly: it doesn't port at all. Apple's app review will
reject a binary that ships a Node interpreter inside your bundle. You can't `exec` in an iOS sandbox. Even on Android,
where you technically can, you're carrying a runtime that has nothing to do with what your app is for.

**agentlib closes that gap.** It is the only open-source agent SDK that runs natively inside a Flutter app — pure Dart,
no subprocess, no `exec`, with native bridges to Apple Foundation Models and Gemini Nano. Same API shape as the Claude
Agent SDK, but the code ships *in your app binary*, not on a server it talks to.

## What "mobile-native" actually means

A lot of frameworks claim to "support" mobile. They usually mean "you can call our HTTP API from a mobile app". That's
fine — you can call any HTTP API from a mobile app — but it's not the same thing as an agent SDK on the device. An
agent SDK on the device:

1. **Doesn't need a server.** The orchestrator, the tool dispatcher, the snapshot timeline, the hooks — all of it runs
   in your app process.
2. **Calls on-device models when it can.** Apple Intelligence's foundation model, Gemini Nano on Pixel, llama.cpp via
   Fllama, Gemma 3 via MediaPipe — all without a network round trip.
3. **Survives the OS.** Backgrounding, suspension, push wake-up, low battery — every one is a lifecycle hook you can
   wire into.
4. **Plays nice with reviewers.** No subprocess, no shell escape, no `exec` of arbitrary code. The sandbox stays sealed.
5. **Lets the UI react in real time.** Streaming events go straight into a Dart `Stream<RunEvent>`, which slots into
   `StreamBuilder`, Riverpod, BLoC, or whatever state management the team already uses.

agentlib does all five.

## What the alternatives look like

It helps to be specific about what "use the Claude SDK in a Flutter app" actually involves.

**Option A: Spawn a Node process on the device.** You bundle a Node binary, stream the Claude SDK's stdout back to
Flutter via platform channels, and pray Apple doesn't reject the app. Some teams do this. They also spend most of
their engineering time on the integration rather than the product.

**Option B: Wrap the SDK behind your own HTTP server.** You run the Claude Agent SDK on a backend, expose a thin
HTTP wrapper, and have the app call it. This works, but: every request needs a network round-trip, no offline use,
you carry server cost, and you've added a whole new service to operate.

**Option C: Re-implement the agent loop in Dart.** Some teams do this from scratch — call the model API, parse tool
calls, dispatch tools, manage messages. That's exactly what agentlib is, except agentlib already did the hard parts:
streaming parsers for Anthropic / OpenAI / Google, on-device provider implementations, snapshots, hooks, subagents,
skills, the Sh mini-shell. Tens of thousands of lines of agent loop you don't have to write.

agentlib is option D: ship the SDK *in* your app, in the language you already write.

## What's in agentlib that's worth knowing about

If you've used the Claude Agent SDK, the shape will feel familiar. agentlib explicitly mirrors the primitives that
Anthropic and OpenAI converged on, plus a few mobile-only additions.

**`AgentSpec` + `Runner`.** A value-class descriptor and a streaming event-driven loop. Compose `AgentSpec`s; pass them
to `Runner.runStreamed(...)`; consume `Stream<RunEvent>`. Events include `TextDelta`, `ThinkingDelta`, `ToolCalled`,
`ToolResult`, `SnapshotCreated`, `Finished`.

**Subagents.** Sync, parallel, or background. Each gets its own context window so deep research doesn't pollute the
parent. Dispatch via the `Task` tool or the `task` CLI builtin.

**Skills.** Markdown bundles at `/workspace/.skills/<name>/SKILL.md`. The orchestrator sees only `name + description`
until invocation — then the body loads. A 30-skill catalog costs ~600 tokens in the orchestrator, not 12 K.

**Snapshots.** `snapshot()`, `revert()`, `fork()`. Content-addressed (SHA-256). SQLite-backed in production. Auto
before `highImpact: true` tool calls. Users can undo. You can fork to try a different model mid-run.

**21 hooks.** `PreToolUse`, `PostToolUse`, `PermissionRequestHook`, `OnSuspend`, `OnResume`, `OnLowBattery`,
`OnNetworkChange`, `OnHandoff`, `UserPromptSubmit`, `StopHook`… everything you'd want to observe is observable.

**Vfs + Sh.** A virtual filesystem (`/workspace`, `/scratch`, `/system`, `/apps`, `/bin`, `/memory`, `/snapshots`)
plus a sandboxed Unix mini-shell that parses pipes, redirects, `$VAR`, `$(cmd)`. No real `exec`, no `&`, no `eval`.

**ModelRoute.** Capability-aware fallback. `ModelRoute.preferOnDevice([FoundationModelsProvider(), GeminiNanoProvider(),
FllamaProvider(...)], fallback: AnthropicProvider(...))` walks the list and picks the first provider that can serve
the request.

**MCP.** HTTP and WebSocket transports. MCP tools surface as `/bin/mcp.<server>.<tool>` CLIs in `Sh` pipelines.

## A first agent

```dart
import 'package:agentlib/agentlib.dart';

final agent = AgentSpec(
  name: 'helper',
  instructions: 'You are a helpful mobile assistant.',
  model: ModelRoute.preferOnDevice(
    onDevice: [FoundationModelsProvider(), GeminiNanoProvider()],
    fallback: AnthropicProvider(apiKey: '<KEY>'),
  ),
);

final run = Runner.runStreamed(
  agent.toRunConfig(vfs: Vfs(), threadId: 'main'),
  input: 'Why is agentlib mobile-native?',
);
await for (final e in run.events) {
  if (e is TextDelta) stdout.write(e.delta);
}
```

That's the full agent. On an iPhone 15 Pro running iOS 19, this never leaves the device — Apple Foundation Models
serves the request. On a Pixel 8, Gemini Nano serves it. On a 2019 iPhone XR with iOS 16, the cloud fallback kicks in.
Your code is identical.

## Why not just embed `MLC LLM` or `gpustack` directly?

You can. agentlib is not trying to be a model runtime — it's an *agent* SDK on top of model runtimes. The runtimes it
wraps (FoundationModels, AICore, MediaPipe LlmInference, Fllama / llama.cpp) are the right primitives for inference.
agentlib adds the loop: tool calling, hooks, snapshots, subagents, the streaming event surface, MCP transports.

If you want the loop in Dart but a different runtime, implement the `ModelProvider` interface — it's two methods —
and plug it in.

## Bottom line

If your agent runs on a server, use the Claude Agent SDK or Mastra. If your agent runs on the device, use agentlib.
A surprising number of products do both — Claude SDK on the backend, agentlib on the device — and we built MCP
transports in agentlib precisely so they can interoperate.

Install is one pubspec line. The hello-agent example is 79 lines. The five quickstart Flutter apps in the
[`example/quickstarts`](https://github.com/neul-labs/agentlib/tree/main/example/quickstarts) folder cover the
common cases — hello agent, multi-tool, on-device, voice, mobile automation.

Read [the primitives in detail](/primitives) next, or jump to [/install](/install) and ship something.