Designing a tool-calling agent that survives going offline

The pitch for EchoList was simple: manage your tasks by talking to them. "Move the lab report to Friday" should just work — no date pickers, no forms. The implementation turned into the best lesson in agent design I've had, mostly because Android refused to cooperate with the fantasy version.

Tool calling beats asking for JSON#

First instinct: prompt the model to return structured JSON, parse it, done. This works in demos and fails in production, because a language model asked to generate a structure will occasionally be creative about it — a trailing comma here, an extra field there, a due date that's subtly a string when you expected a number.

Tool calling inverts the contract. Instead of asking the model to produce data that happens to be structured, I give it functions with typed arguments and let it choose:

val tools = listOf(
  Tool("create_task", CreateTaskArgs(title, dueDate, tags, priority)),
  Tool("update_task", UpdateTaskArgs(id, field, value)),
  Tool("query_tasks", QueryTasksArgs(filter)),
  Tool("complete_task", CompleteTaskArgs(id)),
)

The model's job shrinks from "be a serializer" to "pick the right function". Failures become typed and recoverable: a malformed date fails validation at the tool boundary, where I can return a structured error and let the model retry — not silently in the user's task list three screens later.

The fallback ladder#

Here's the part Android taught me. Networks on a moving phone are a spectrum, not a boolean. EchoList degrades in explicit named levels:

1. Online — GLM tool-calling with a streaming status line ("thinking…", "creating task…"), so the user sees why it takes two seconds.
2. Degraded — requests time out at 2.5 seconds; one retry; if that fails, drop a level and say so.
3. Offline — a deterministic on-device parser handles the common patterns: "tomorrow", "next Monday", "!high", "#project". The banner is honest: "Offline — simplified commands only."

The ladder forced a design constraint I now apply everywhere: every agent capability must have a non-agent implementation of its top 3 uses. If the feature only works when the model works, the feature doesn't work.

Trust details that made it feel finished#

None of these are AI features. All of them are why testers trusted the AI:

• Undo on everything. Every agent-initiated change is a visible diff — "due date: Thursday → Friday" — with one-tap revert. People forgive mistakes they can see and reverse.
• Optimistic UI. Tasks appear instantly and reconcile when the round trip lands. The network never gets to block a tap.
• A tool-call debug screen. Every model decision is inspectable: prompt, tool chosen, arguments, result, latency. Two weeks of bugs were diagnosed there — hallucinated task IDs, timezone double-conversions — without ever guessing "what did the model do?"
• Confidence in the copy. "Added to your list" when confident; "I think you meant Friday — tap to confirm" when parsing was fuzzy. The interface admits uncertainty instead of hiding it.

What I took away#

The agent is maybe 20% of an agent product. The other 80% is the boring envelope: timeouts, retries, fallback tiers, undo, logging, and honest status text. That ratio holds for anything LLM-powered I've built since.

If you're building your first agent feature: start with the offline behaviour. Design what happens when the intelligence is absent — the rest is much easier to make trustworthy once the absent case already is.