Preventing Runaway LLM Costs with Atomic Credit Metering

Building a high-throughput conversational AI assistant on WhatsApp is technically exhilarating—until you examine your cloud and API inference invoices.

Unlike traditional web-based SaaS platforms where users encounter paywalls, mandatory credit card forms, or explicit session limits before taking actions, messaging platforms like WhatsApp invite frictionless, casual banter. A single passionate football fan can easily send 40 messages on a Saturday morning, comparing complex transfer combinations, asking for expected goals (xG) breakdowns, and simulating wildcards across multiple gameweeks.

Each of those turns might trigger multiple tool calls, query external sports data feeds, and consume thousands of tokens across multimodal LLMs. While we currently utilize Google Gemini for core development and specific tasks like multimodal voice note transcription and embeddings, our financial engine must remain strictly model-agnostic. Without rigorous real-time metering, a popular AI agent will quickly turn into an unsustainable financial liability.

Furthermore, traditional Stripe subscriptions or credit card forms experience extreme friction in African markets where mobile money dominates daily commerce. To commercialize Panda FPL, we needed an airtight financial engine that:

1. Enforces Atomic Pre-Flight Deduction: Verifies and deducts user balances before invoking expensive inference pipelines to eliminate race conditions.
2. Executes Graceful Compensation Hooks: Automatically refunds credits if an upstream sports API or LLM provider fails mid-turn.
3. Supports Sub-Second Mobile Money Top-Ups: Integrates Safaricom M-Pesa STK Push, allowing managers in Kenya to top up in 5 seconds directly from their mobile phone keyboards.
4. Guarantees Idempotent Webhook Processing: Prevents double-crediting and replay attacks via HMAC signature verification and atomic database locks.

Here is the engineering blueprint behind our transactional metering and billing system.

High-Level Financial State Flow#

To protect our API margins while delivering an effortless payment experience, financial state transitions must be completely deterministic:

1. The Pre-Flight Deduction Pattern vs. Post-Billing#

A common anti-pattern in early AI applications is post-billing: generating the AI response, counting the total tokens consumed, calculating the cost, and then attempting to deduct the user's account balance.

In production messaging environments, post-billing collapses under real-world concurrency:

• The Double-Tap Race Condition: If a user fires three rapid queries within 800 milliseconds, parallel webhook workers read the same positive balance, proceed to execute all three expensive LLM runs, and push the user's account deep into negative credits.
• Uncollectible Debt: If a database connection hiccups after the LLM response has already been dispatched to WhatsApp, the user received expensive compute for free, and you absorb the loss.

Instead, we enforce Atomic Pre-Flight Deduction:

// Conceptual Atomic Credit Metering Engine
import { getPayload } from "payload";
import config from "@/payload-config";

export class CreditManager {
  /**
   * Pre-flight atomic reservation.
   * Atomically verifies balance >= cost and decrements in a single operation
   * to eliminate concurrent webhook double-spending race conditions.
   */
  static async deduct(
    userId: string,
    cost: number,
    featureName: string,
    breakdown?: Record<string, any>,
  ): Promise<boolean> {
    const payload = await getPayload({ config });

    try {
      // 1. Single Atomic Conditional Decrement via MongoDB driver
      const userModel =
        (payload.db as any)?.collections?.["users"] ||
        (payload.db as any)?.models?.["users"];

      if (userModel?.findOneAndUpdate) {
        const updated = await userModel.findOneAndUpdate(
          { _id: userId, credits: { $gte: cost } },
          { $inc: { credits: -cost } },
          { new: true },
        );

        if (!updated) {
          // If no document matched, balance was insufficient (< cost)
          return false;
        }
      }

      // 2. Immutable Usage Audit Ledger
      await payload.create({
        collection: "usage-logs",
        data: {
          user: userId,
          feature: featureName,
          cost,
          breakdown: breakdown || {},
          timestamp: new Date().toISOString(),
        },
      });

      return true;
    } catch (error) {
      console.error("[CreditManager] Atomic deduction failed:", error);
      return false;
    }
  }
}

Every analytical tool in our consolidated 5-tool architecture declares its exact unit cost:

• Account status and squad viewing (audit_user_squad): 0 Credits (Free basic tier)
• Matchday live tracking and mini-league rival monitoring (track_live_and_rivals): 1 Credit
• Rolling fixture difficulty and Elo modeling (analyze_fixtures): 2 Credits
• Volatile squad mutation and transfer drafting (manage_squad_state): 3 Credits
• Combinatorial knapsack chip optimization (optimize_chips_and_drafts): 5 Credits

If CreditManager.deduct() returns false, execution halts immediately before contacting LLM inference or external data endpoints. The assistant responds with the user's current balance and interactive top-up buttons.

2. Graceful Rollbacks with Compensation Hooks#

What happens if a user's credits are successfully deducted, but the Premier League data API drops the connection, or the LLM provider experiences a temporary 503 outage during final synthesis?

Silently consuming a customer's paid credits for an error screen destroys user trust instantly.

To maintain transactional integrity across asynchronous boundaries, we wrap the execution pipeline in a Two-Phase Compensation Hook:

1. Pre-Flight Reservation: CreditManager.deduct() reserves the required credits before any downstream network request is dispatched.
2. Execution Boundary: The analytical workload executes inside a protected try/catch block.
3. Automatic Compensation on Failure: If an unhandled exception or upstream timeout occurs, the catch block triggers CreditManager.refund(), which executes an atomic MongoDB $inc: { credits: +cost } and appends an audit record to usage-logs noting the failure reason (UPSTREAM_API_TIMEOUT).
4. Transparent User Messaging: Instead of an unhelpful error trace, the user receives an immediate apology on WhatsApp confirming their credits were not charged and inviting them to retry in a moment.

This compensation pattern ensures that credits are only permanently consumed when the user receives complete, actionable intelligence.

3. Sub-Second Mobile Money with M-Pesa STK Push via Paystack#

Credit cards represent less than 5% of digital consumer transactions in Kenya and East Africa. If you force an FPL manager to leave WhatsApp, open a mobile browser, enter a 16-digit card number, CVV, and 3D Secure OTP just to buy a weekend pass of AI analysis, abandonment rates exceed 90%.

We engineered our production billing around Paystack as our unified payment gateway orchestrator, bridging directly into Safaricom's Daraja M-Pesa STK Push rail.

Why the Orchestrator Architecture Matters#

Rather than integrating raw telco APIs with custom retry daemons, Paystack acts as the resilient transaction orchestrator:

• Direct Telco Bridge: Paystack coordinates the mobile money payment request directly to Safaricom's Daraja gateway in Kenyan Shillings (KES).
• Automated Failure Fallback: Handles carrier-level timeouts, network delays, and transient banking connection drops automatically.
• Encrypted Webhook Ingress: Dispatches cryptographically signed event notifications to our backend upon settlement.

How M-Pesa STK Push Operates in Practice#

1. The user taps an interactive WhatsApp package button or types a natural top-up request.
2. Our backend dispatches an encrypted charge request to Paystack specifying the user's phone number and the KES package amount.
3. Within 1.5 seconds, Safaricom displays a native SIM toolkit modal overlaid directly across the user's mobile screen:
   • "Do you want to pay Panda FPL? Enter M-Pesa PIN: [ **** ]"
4. The user enters their 4-digit PIN. The money transfers immediately, Safaricom issues an instant SMS receipt, and Paystack dispatches an HMAC-signed webhook to our server.

Natural Language and Sheng Intent Recognition#

Users rarely format payment commands identically. Beyond interactive button taps, our parser extracts payment intent from natural conversational cues:

• Localized Slang and Numbers: Capturing Kenyan Sheng phrases such as "soo mbili" (200), "mia tano" (500), "50 bob", or "weka 200".
• Package Aliases: Recognizing names like "weekend pass", "monthly plan", or "mini-league champion".
• Voice Note Audio: Audio voice notes are transcribed using multimodal AI models before passing into the numerical intent extractor.

(For detailed credit bundles and pricing structures, explore the Panda FPL Pricing Section).

4. Webhook Security with HMAC-SHA512 Verification and Idempotency#

When processing monetary transactions, webhooks must be treated as hostile until cryptographically proven authentic. Our webhook ingress pipeline follows four strict security phases:

1. Cryptographic Signature Verification: We extract the x-paystack-signature header and compute an HMAC-SHA512 hash over the raw request payload buffer using our secure environment secret. If the computed hash does not match byte-for-byte, the request is rejected immediately with a 401 Unauthorized before any JSON parsing occurs.
2. Idempotency Guard via Transaction Ledger: Payment gateways operate on an at-least-once delivery model, meaning network retries can deliver identical webhooks multiple times. Before crediting any account, we query our transactions collection by the unique gateway reference. If a completed record already exists, we return 200 OK immediately, preventing balance inflation.
3. Atomic Balance Increment: Once verified as a new transaction, a transaction record is created and the user's balance is incremented using MongoDB's atomic $inc: { credits: creditsToAdd } operator.
4. Asynchronous WhatsApp Confirmation: Within milliseconds of transaction settlement, our WhatsApp service dispatches a formatted receipt directly to the user's chat, confirming their new balance and suggesting their next tactical question.

5. Expanding Beyond M-Pesa with Polar.io#

Currently, our production billing operates on M-Pesa via Paystack, providing seamless mobile money checkout for our domestic community in Kenya.

However, Fantasy Premier League is a global phenomenon. As our community expands to managers in the UK, Nigeria, South Africa, and international diaspora leagues, our next billing milestone is integrating Polar.io:

1. Merchant-of-Record Compliance: Polar handles international VAT, sales tax, and automated invoicing out of the box, removing cross-border compliance burdens.
2. Global Payment Rails: Allows international users to top up seamlessly using Apple Pay, Google Pay, and international cards with the same frictionless link delivery directly inside WhatsApp.
3. Unified Credit Ledger: Regardless of whether credits are purchased via M-Pesa in Nairobi or Apple Pay in London, all tokens flow into the same atomic CreditManager ledger.

6. Key Production Takeaways#

• Never run inference without pre-authorization: Treat LLM tokens and external data queries like debit card authorizations.
• Implement compensation hooks: If downstream tool calls time out, automatically refund the balance immediately to preserve customer trust.
• Eliminate checkout friction: Meeting users on their native payment rail (M-Pesa STK push today, Polar.io globally tomorrow) drives conversions that traditional web checkouts cannot match.
• Normalize phone numbers aggressively: Strip spaces, formatting characters, and enforce international E.164 formats before hitting telco gateways.
• Keep basic navigation free: Let users check point-in-time account status for free to maintain daily engagement, reserving credit deductions for high-value combinatorial solvers and live tracking tools.

The Complete Engineering Series#

Explore the full 5-part architecture series behind Panda FPL:

• Part 1 — Behind the Multi-Agent Architecture of Panda FPL
• Part 2 — How Consolidating 22 Tools into 5 Fixed Our Agent Hallucinations
• Part 3 — Why Generic AI Architecture Fails in High-Context Real-Time Domains
• Part 4 — Engineering Sub-10-Second Tactical AI on WhatsApp
• Part 5 — Preventing Runaway LLM Costs with Atomic Credit Metering

Curious to test Panda FPL's real-time analytics in action? Start chatting with Panda FPL on WhatsApp, or review our comprehensive Panda FPL Engineering Case Study.

Need help designing billing state machines, M-Pesa mobile money integrations, or LLM cost control architectures? Let's discuss your project architecture.