Offline and on-device agents with cloud fallback
Some agents must keep working with no signal, or must never send certain data off the device. agentlib treats on-device models as first-class ModelProviders and uses ModelRoute to escalate to the cloud only when a request exceeds local capability — with no branching in your app code.
Four on-device providers, one interface
Apple Foundation Models, Gemini Nano, llama.cpp (via Fllama), and MediaPipe each implement the same ModelProvider shape as AnthropicProvider. That means an on-device model and a cloud model are interchangeable in an AgentSpec. See the on-device deep dive for the full provider matrix.
Capability-aware routing
ModelRoute.preferOnDevice([...]) picks the first capable on-device provider and only reaches for a cloud provider when the local one can't satisfy the request (context length, tool use, vision). Privacy-sensitive turns stay local; heavy reasoning escalates — automatically.
Graceful degradation, not failure
Because routing is capability-aware, an offline device doesn't error — it runs the on-device path. When connectivity returns, the OnNetworkChange hook lets you re-enable cloud escalation. The agent's behaviour degrades gracefully instead of breaking.
final model = ModelRoute.preferOnDevice([
FoundationModelsProvider(), // iOS 19+
GeminiNanoProvider(), // Pixel 8+, S24+
FllamaProvider(model: 'qwen2.5-1.5b'),
], fallback: AnthropicProvider(
apiKey: const String.fromEnvironment('ANTHROPIC_API_KEY'),
));Related use cases
- An in-app AI assistant that ships inside your Flutter app — Build a Claude-Code-grade assistant that runs in your Flutter app on iOS and Android — no Node subprocess, no backend agent runtime.
- Bring MCP servers to a mobile agent — Connect any Model Context Protocol server over HTTP or WebSocket and expose its tools to an on-device agent as Sh CLIs.
See how these fit together in the architecture overview, or browse all eight primitives in depth.