How Consolidating 22 Tools into 5 Fixed Our Agent Hallucinations

In the early hype of agentic AI frameworks, developers were encouraged to treat large language models as universal dispatchers: define twenty or thirty microscopic tools, dump their JSON schemas into the model’s context window, and let the agent figure out what to call.

When building Panda FPL, our conversational Fantasy Premier League assistant on WhatsApp, we fell directly into this trap.

Our initial backend provided tools for every distinct action you could imagine: get_bank_balance, get_free_transfers, get_chips_remaining, get_player_stats, get_differentials, get_fixture_swings, calculate_transfer_hit, suggest_captain, compare_players, and more than a dozen others. On paper, this felt modular and clean.

In production, it was an operational disaster.

Exposing an LLM to 22 separate tools caused severe decision paralysis. The model frequently hallucinated arguments, picked the wrong tool for ambiguous queries, executed redundant multi-turn loops, and took seven to nine seconds just to formulate an execution plan.

Here is how we redesigned our agent architecture from the ground up: collapsing 22 microscopic tools into 5 high-order action interfaces and introducing an upstream intent filter that slashed latency by more than half while pushing tool-call accuracy past 98%.

The Hidden Cost of Tool Proliferation#

When an autonomous agent has access to dozens of tools, three distinct engineering problems emerge under real traffic:

1. Context Window Tax and Schema Bloat: Every tool declaration includes a name, description, and nested JSON schema for its parameters. Injecting 22 tools consumed over 2,200 input tokens on every single message turn before the user’s actual question or squad context was even read.
2. Semantic Ambiguity and Parameter Hallucination: When tools have overlapping boundaries (such as get_player_stats versus compare_players versus get_differentials), small variations in user phrasing cause the model to hesitate. It would frequently invent arguments that did not exist in the schema or call a read tool when the user intended a state mutation.
3. Multi-Turn Call Cascades: If a manager asked: "I have £0.8m in the bank, who is the best replacement for my injured defender?", the model would invoke get_bank_balance, wait for the tool output, invoke get_squad_injuries, wait again, and finally invoke get_transfer_targets. Each network roundtrip added 1.5 to 2 seconds of latency, pushing total response time past twelve seconds.

On a messaging platform like WhatsApp, twelve seconds of silence without streaming tokens is completely unacceptable.

The Tool Consolidation Architecture#

To solve this, we stepped back and analyzed how human domain experts reason. A veteran fantasy strategist does not perform five separate disconnected lookups; they think in terms of consolidated operational workflows.

We collapsed 22 brittle tools into 5 comprehensive, high-order domain interfaces:

The Five Consolidated Domain Tools#

Instead of treating tools as simple database getters, each tool represents an autonomous subsystem capable of multi-step evaluation in a single execution:

1. manage_squad_state: Handles all user-directed roster mutations, including transfer tracking, active mini-league switches, captaincy changes, and bench ordering in one unified action.
2. audit_user_squad: Evaluates complete squad health in a single pass, diagnosing flagged injuries, calculating defensive and attacking balance ratings, evaluating captaincy leverage, and identifying immediate transfer risks.
3. optimize_chips_and_drafts: Executes combinatorial squad optimization for Wildcard and Free Hit chips, compares expected points across chips, and evaluates long-term chip timing roadmaps.
4. analyze_fixtures: Computes rolling difficulty swings, isolates domestic league matches from cup congestion, and projects clean sheet and goal probabilities across a defined gameweek horizon.
5. track_live_and_rivals: Pulls live matchday points, tracks mini-league rank movements, and performs rival radar analysis against both immediate podium leaders and pack competitors.

Upstream Intent Filtering to Inject Only What Matters#

Consolidating tools down to five solved multi-turn cascades, but sending all five tool definitions into every prompt still imposed unnecessary cognitive overhead on casual turns.

To make the agent razor-sharp, we introduced an Upstream Intent Filter.

Before the primary reasoning model is invoked with heavy tool loadouts, a dedicated, low-latency LLM classification call (running with temperature 0.1 and strict JSON schema enforcement) inspects the incoming message alongside active session context. It categorizes the turn into structured operational modes:

• Direct Chat and Profile Reads (DIRECT_CHAT): Casual banter ("Hey Panda", "Thanks"), basic account inquiries ("How much money do I have in the bank?", "How many free transfers do I have?"), or season point tallies answered directly by profile data. Zero tools are injected.
• Roster and League Mutation (STATE_MUTATION): When the manager reports an action ("I sold Saka for Palmer", "Switch active league to PESA OTAS"), the filter binds manage_squad_state.
• Squad Audit and Health (SQUAD_AUDIT): Rating the team, diagnosing injury flags, or bench order decisions binds audit_user_squad.
• Combinatorial Optimization (WILDCARD_CHIPS): Free Hit squads, Wildcard overhauls, or multi-week chip comparisons bind optimize_chips_and_drafts alongside analyze_fixtures.
• Live Matchday and Mini-League Radar (LIVE_AND_RIVALS): Live points, ongoing matches, and rival tracking bind track_live_and_rivals alongside analyze_fixtures.

By using a structured JSON classification call upfront, the architecture avoids both fragile regex matching and bloated prompt schemas. If a turn is simple conversational banter, the primary agent receives zero tools, completely eliminating tool hallucination and slashing token overhead. When complex tactical inquiries arrive, the agent receives only the precise one or two tools required for the task.

Production Impact and Benchmarks#

We measured the performance difference before and after consolidating tools and adding the upstream intent filter across 5,000 live production messages:

Production Gotchas#

When consolidating tools, two architectural traps require careful handling:

1. Over-Consolidation: Merging read operations with irreversible state mutations into a single vague tool can cause unintended side effects. Keep analytical simulations strictly isolated from real state persistence.
2. Schema Description Precision: Because consolidated tools handle broader responsibilities, parameter descriptions must be unambiguous. Document enum values and optional fields explicitly so the model never defaults to ambiguous placeholders.

By treating tool design as an exercise in domain modeling rather than a reflection of database tables, we built an agent that is faster, significantly cheaper to operate, and consistently accurate under live traffic.

Related Deep Dives in This Series#

• Part 1 — Behind the Multi-Agent Architecture of Panda FPL
• Part 3 — Why Generic AI Architecture Fails in High-Context Real-Time Domains
• Part 4 — Engineering Sub-10-Second Tactical AI on WhatsApp
• Part 5 — Preventing Runaway LLM Costs with Atomic Credit Metering
• Case Study — Panda FPL Systems Architecture Case Study

Designing specialized AI agents, optimizing function-calling pipelines, or scaling real-time conversational systems? Connect directly at jackmkimbo.com/#contact.