Building Production-Ready AI Agents with RAG

AI agents do not just answer questions - they take action. Given a goal, an agent reasons through what it needs, calls tools, observes results, and repeats until the task is done. Add RAG, and the agent stops guessing at company-specific information and starts retrieving it - your actual policies, live order data, current documentation.

That combination powers real use cases: a support agent that resolves refund requests end-to-end, an HR tool that answers policy questions from your actual handbook, an internal assistant that pulls from your live knowledge base instead of making things up.

The hard part is not building the demo. It is making it work in production. Most teams hit the same wall: stale knowledge retrieved, agents looping on tool failures, no escalation path when the answer is not there.

This guide is about closing that gap.

We will cover how to build AI agents that are grounded in your company's actual knowledge, built around reliable tool execution patterns, and designed to handle the failure modes that only show up when real users hit your system. Every section includes working Node.js code you can adapt for your own stack.

What you will learn:

• What makes an AI agent fundamentally different from a chatbot - and why that difference changes everything about how you build

• Architecture patterns for two core production agent types: customer support and internal knowledge tools

• How to integrate RAG into an agent loop so your agent retrieves company knowledge intelligently, not blindly

• Tool calling patterns, error handling, guardrails, and fallback mechanisms that keep agents reliable under real load

• Monitoring, observability, and cost optimisation strategies for agents running at scale

Who this is for: Engineering teams and technical decision-makers at startups and SMEs who have experimented with AI and are now serious about production deployment. If you have already built a basic chatbot or RAG pipeline, this guide is your next step.

Building on our RAG foundation: This guide extends our earlier work on metadata filtering in production RAG systems. If you have not read that guide, we recommend starting there. The Supabase pgvector setup, staged hybrid filtering approach, and Node.js patterns from that guide appear throughout this one.

Section 1: What Makes an AI Agent Different From a Chatbot

Before we write a single line of code, we need to be precise about what an agent actually is. The word gets used loosely sometimes to describe a simple chatbot with a system prompt, sometimes to describe a fully autonomous multi-step reasoning system. That ambiguity causes real problems when you are making architecture decisions.

Here is a definition that holds up in production:

An AI agent is a system that uses an LLM to decide what to do next, executes actions using tools, observes the results, and repeats until the task is complete.

Three words in that definition do the most work: decide, execute, and repeat. A chatbot does none of those things. It receives input, generates output, and stops. The interaction is stateless and linear. An agent operates in a loop.

The Chatbot vs. Agent Distinction

Consider a user asking: "Can I get a refund for my order from last Tuesday?"

A chatbot:

User:  "Can I get a refund for my order from last Tuesday?"
Bot:   "Our refund policy allows returns within 30 days of purchase.
        Please contact support at support@company.com to initiate a return."

The chatbot retrieved a policy from its context and generated a response. It did not check whether the order exists, whether it is within the return window, or whether this specific product is eligible. It gave a generic answer and stopped.

An agent handling the same request:

User:   "Can I get a refund for my order from last Tuesday?"

Agent thinks: I need to look up this order before I can answer.

→ Calls lookupOrder({ userId: "u_123", dateRange: "last 7 days" })
  Result: Order #4821, placed 6 days ago, Product: Pro Plan subscription

→ Calls checkRefundEligibility({ orderId: "4821", productType: "subscription" })
  Result: Eligible. Subscription refunds allowed within 14 days.

→ Calls initiateRefund({ orderId: "4821", reason: "user_request" })
  Result: Refund #R-9923 created. $49 returned in 3–5 business days.

Agent responds: "I have initiated a refund for your Pro Plan subscription
                (Order #4821). You will receive $49 back within 3–5 business
                days. Your refund reference is R-9923."

The agent made decisions, executed actions across three different tools, and produced an outcome - not just an answer. That is the distinction that matters.

The Perception-Reasoning-Action Loop

Every production agent runs on the same underlying loop:

                    ┌─────────────────────┐
                    │    User Message /    │
                    │    Task Input        │
                    └──────────┬──────────┘
                               │
                               ▼
                    ┌─────────────────────┐
                    │   PERCEIVE          │
                    │   Read context,     │
                    │   conversation      │
                    │   history, state    │
                    └──────────┬──────────┘
                               │
                               ▼
                    ┌─────────────────────┐
              ┌────▶│   REASON            │
              │     │   What do I know?   │
              │     │   What do I need?   │
              │     │   Which tool next?  │
              │     └──────────┬──────────┘
              │                │
              │                ▼
              │     ┌─────────────────────┐
              │     │   ACT               │
              │     │   Call a tool,      │
              │     │   retrieve context, │
              │     │   execute function  │
              │     └──────────┬──────────┘
              │                │
              │          Tool result returned
              │                │
              └────────────────┘
                               │
                    Task complete?
                               │
                    ┌──────────▼──────────┐
                    │   Final Response     │
                    │   to User            │
                    └─────────────────────┘

The agent perception-reasoning-action loop

The Five Components of a Production Agent

┌──────────────────────────────────────────────────────────────────┐
│                        PRODUCTION AGENT                          │
│                                                                  │
│   ┌──────────────┐    ┌──────────────┐    ┌──────────────────┐  │
│   │  LLM Core    │    │    Tools     │    │     Memory       │  │
│   │  Reasoning   │◀──▶│  Functions   │    │  Short-term:     │  │
│   │  Planning    │    │  the agent   │    │  conversation    │  │
│   │  Deciding    │    │  can call    │    │  history         │  │
│   └──────────────┘    └──────────────┘    │  Long-term: RAG  │  │
│                                           └──────────────────┘  │
│   ┌──────────────┐    ┌──────────────┐                          │
│   │  Guardrails  │    │ Orchestrator │                          │
│   │  Input/out   │    │  Loop ctrl,  │                          │
│   │  validation  │    │  step limit, │                          │
│   │  Safety      │    │  routing     │                          │
│   └──────────────┘    └──────────────┘                          │
└──────────────────────────────────────────────────────────────────┘

The five components of a production AI agent

LLM Core - The reasoning engine. It reads the conversation, decides what to do next, and either generates a response or issues a tool call.

Tools - Functions the LLM can invoke. Each tool has a name, a description the model reads to decide when to call it, an input schema, and an execution function.

Memory - Short-term memory is conversation history within a session. Long-term memory is your vector database - the RAG knowledge base that grounds the agent in your company's actual information.

Guardrails - Constraints that prevent the agent from doing things it should not. Input validation, output filtering, topic scope limiters, human escalation thresholds. In production, guardrails are not optional.

Orchestrator - The loop controller. It decides when to call the LLM again, enforces the maximum number of steps, handles retries, and routes between agents if your system uses more than one.

What "Production-Ready" Actually Means

An agent is production-ready when it satisfies four criteria:

Reliable - It handles failure gracefully. API timeouts, empty retrieval results, malformed tool arguments - none of these should cause an uncaught exception or a hallucinated answer.

Grounded - Its responses are tied to verifiable sources. For business automation, this means RAG. Hallucinated policy answers are not just wrong - they are a liability.

Observable - You can see what it did, why it did it, and what it cost. Every tool call, every retrieval, every LLM invocation should be logged.

Bounded - It knows what it is not supposed to do, and says so clearly. Scope limits, access controls, and human escalation paths are architecture decisions, not afterthoughts.

The POC-to-Production Gap: The most common issues we see: no max iteration guard (agent loops indefinitely when a tool fails), retrieval with no metadata filtering (agent answers 2025 questions with 2022 data), and no fallback for zero retrieval results (agent hallucinates rather than saying "I don't know"). Every section of this guide is designed to close these gaps before you deploy.

Section 2: Architecture Patterns for Production Agents

There is no single correct architecture for an AI agent. The right pattern depends on what the agent needs to do, how it will fail, and how much autonomy you can afford to give it. For most businesses deploying agents, two patterns cover the vast majority of use cases: customer support and internal knowledge tools. We focus on these two because they represent different ends of the production complexity spectrum - one is customer-facing with live operational data, the other is internal-facing with a documentation-heavy knowledge base.

Pattern 1: Customer Support Agent

The support agent handles inbound user questions, looks up order and account information, checks policies, and either resolves the issue or escalates to a human. This is the most common starting point for businesses deploying agents because the value is immediately measurable: fewer tickets reaching your human team.

The RAG requirement here is critical. Your agent needs live access to your product documentation, return policies, and FAQs - and that knowledge changes frequently. An agent answering refund questions with a six-month-old policy is worse than no agent at all.

The hardest production challenge is not getting the right answer. It is knowing when the agent should stop trying and hand off to a human. This threshold - when to escalate - is one of the most important design decisions you will make, and it belongs in your system prompt, not your code.

Pattern 2: Internal Knowledge Agent

The internal agent answers employee questions about company policies, processes, HR documents, and internal procedures. This is often the fastest to build (internal users are more forgiving than customers) and the cheapest to run (lower volume, lower latency requirements).

The RAG requirement is the entire value proposition. The agent is only as good as its knowledge base.

The hardest challenge is keeping that knowledge base current - stale internal docs lead to agents confidently citing policies that no longer exist. Metadata filtering by document date and version is not optional here; it is the core reliability mechanism.

Agent Type Comparison

Both agent types share one non-negotiable requirement: RAG. The difference is in operational complexity - the support agent writes to external systems (tickets, order updates) while the knowledge agent stays read-only. That distinction drives almost every other architecture decision.

The next section covers exactly how to integrate RAG into the agent loop - not as a preprocessing step, but as a tool the agent chooses to call.

Section 3: RAG Integration - Grounding Your Agent in Company Knowledge

Your agent knows everything the model was trained on. It knows nothing about your refund policy, your product catalogue, or the support ticket that came in this morning. Without RAG, your agent hallucinates. With basic RAG, it retrieves - but it still does not decide. The difference between a POC and a production agent is teaching it when to retrieve, what to retrieve, and when to stop.

Foundation: This section builds directly on our metadata filtering guide. The Supabase match_documents function, staged hybrid filtering approach, and pgvector setup from that guide are used in the code below. If you have not implemented that foundation, start there.

Standard RAG vs. Agentic RAG

In a standard RAG pipeline, retrieval is always step one. Every user query goes through the same sequence regardless of what the question actually needs:

STANDARD RAG (always retrieves):
User Query → Retrieve → Augment → LLM → Answer
             [happens every time, no decision made]

In an agent, retrieval is a tool the agent chooses to call - or not:

AGENTIC RAG (agent decides):
User Query → Agent thinks →
  ├── Has enough context already? → Answer directly
  ├── Needs company knowledge?   → Call retrieve() → Answer
  └── Needs live data + knowledge? → retrieve() + lookupAPI() → Answer

This is the architectural shift that matters. The agent treats your vector database like any other tool. That means retrieval now has a schema, validation, error handling, and retry logic - exactly like your API calls do.

Designing the Retrieval Tool

The most important decision in your retrieval tool is the description field. The LLM reads this to decide when and whether to call the tool. Write it the way you would brief a new employee: be specific about what the tool does, when to use it, and what it returns.

Code Snippet 1 - Retrieval Tool with Staged Hybrid Filtering (Node.js + Vercel AI SDK):

// tools/retrieveKnowledge.js
require('dotenv').config();
const { tool } = require('ai');
const { z } = require('zod');
const { createClient } = require('@supabase/supabase-js');

const supabase = createClient(
  process.env.SUPABASE_URL,
  process.env.SUPABASE_ANON_KEY
);

const retrieveKnowledgeTool = tool({
  description: `Search the company knowledge base to answer questions about 
    products, policies, procedures, and internal documentation. 
    Use this tool when the user asks about company-specific information 
    that may not be in your training data. Do NOT use for general knowledge 
    questions. Always use this before answering policy or product questions.`,

  parameters: z.object({
    query: z.string().describe('The search query derived from the user question'),
    department: z.enum(['support', 'billing', 'engineering', 'hr', 'general'])
      .optional()
      .describe('Filter to a specific department knowledge base'),
    maxResults: z.number().min(1).max(10).default(5)
      .describe('Number of results to return'),
  }),

  execute: async ({ query, department, maxResults = 5 }, { messages }) => {
    try {
      // Get embedding for the query
      const { OpenAI } = require('openai');
      const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

      const embeddingRes = await openai.embeddings.create({
        model: 'text-embedding-3-small',
        input: query,
      });
      const queryEmbedding = embeddingRes.data[0].embedding;

      // Stage 1 + 2: Pre-filter by metadata + vector search
      // Uses the match_documents RPC from our RAG guide
      const { data: results, error } = await supabase.rpc('match_documents', {
        query_embedding: queryEmbedding,
        match_count: maxResults * 2, // retrieve extra for post-filter
        filter_department: department || null,
        filter_max_access_level: 2, // internal content only
      });

      if (error) throw new Error(`Retrieval failed: ${error.message}`);

      // Stage 3: Post-filter - verified content, minimum length
      const filtered = (results || [])
        .filter(r => r.author_verified && r.word_count > 20)
        .slice(0, maxResults);

      // Explicit "not found" - never return empty silently
      if (filtered.length === 0) {
        return {
          found: false,
          message: 'No relevant content found in the knowledge base for this query.',
          results: [],
        };
      }

      return {
        found: true,
        results: filtered.map(r => ({
          content: r.content,
          source: r.source_url,
          department: r.department,
          similarity: Math.round(r.similarity * 100) / 100,
        })),
      };

    } catch (err) {
      // Return structured error - never throw from a tool
      return {
        found: false,
        error: true,
        message: `Knowledge base temporarily unavailable: ${err.message}`,
        results: [],
      };
    }
  },
});

module.exports = { retrieveKnowledgeTool };

Two production details worth highlighting. First, the tool returns { found: false } explicitly when there are no results - not an empty array.

This matters because an empty array gives the LLM no signal about why retrieval failed, and it may hallucinate an answer to fill the gap. An explicit found: false with a message tells the agent to say "I don't have information on this" rather than guess.

Second, errors are caught and returned as structured objects rather than thrown. A thrown error inside a tool can break the agent loop in ways that are hard to debug; a structured error lets the agent decide how to recover.

RAG Inside the Agent Decision Loop

                    ┌─────────────────────────────────┐
                    │         User Message             │
                    └────────────────┬────────────────┘
                                     │
                                     ▼
                    ┌─────────────────────────────────┐
                    │          Agent (LLM)             │
                    │   "Do I need company context?"   │
                    └──────┬──────────────┬────────────┘
                           │              │
                    YES: retrieve    NO: answer directly
                           │
                           ▼
                    ┌──────────────────────┐
                    │  retrieveKnowledge() │
                    │                      │
                    │ 1. Pre-filter        │ ← dept, date, access level
                    │    (Supabase)        │
                    │                      │
                    │ 2. Vector Search     │ ← semantic similarity
                    │    (pgvector)        │
                    │                      │
                    │ 3. Post-filter       │ ← verified, word count
                    │                      │
                    └──────────────────────┘
                           │
                    { found: true, results: [...] }
                    or { found: false, message: "..." }
                           │
                           ▼
                    ┌─────────────────────────────────┐
                    │   Agent generates grounded       │
                    │   answer with source citation    │
                    └─────────────────────────────────┘

RAG inside the agent decision loop

When the Agent Should NOT Retrieve

Most guides tell you when to use RAG. Almost none tell you when to skip it. Knowing when retrieval is wasteful - or actively harmful - is a production insight that matters for both reliability and cost.

Conversational follow-ups. If the user says "Got it, thanks" or "Can you explain that differently?", the agent should use conversation history, not re-query the vector database. A retrieval call here wastes tokens and adds latency with no benefit.

Out-of-scope requests. If retrieval returns found: false with no results above your similarity threshold, the agent should say "I don't have information on this" rather than trying harder or guessing. Repeated retrieval attempts on a query that consistently returns nothing are a sign the question is outside your knowledge base - not a retrieval failure.

Repeated context within a session. If the user asks three follow-up questions about the same policy, the agent can cache the retrieval result for the session rather than querying the vector database three times. This is straightforward to implement with in-memory session state and meaningfully reduces per-session cost.

Section 4: Tool Calling and Function Execution Patterns

Tools are how your agent interacts with the world. RAG retrieval is one tool. But production agents need more: they create tickets, look up orders, book meetings, send notifications. How you define, execute, and validate these tool calls determines whether your agent is reliable or a liability.

How Function Calling Works

When you provide tools to an LLM, you give it a list of function schemas - JSON objects describing what each tool does, what arguments it takes, and what it returns. The model reads these descriptions and decides, based on the conversation, whether to generate a text response or issue a tool call.

┌─────────────────────────────────────────────────────────────┐
│                   TOOL CALLING FLOW                         │
│                                                             │
│  1. Send prompt + tool schemas to LLM                       │
│                                                             │
│  2. LLM responds with either:                               │
│     a) Text response (task complete)                        │
│     b) Tool call: { name: "lookupOrder", args: {...} }      │
│                                                             │
│  3. Your code executes the function                         │
│                                                             │
│  4. Result appended to conversation history                 │
│                                                             │
│  5. LLM called again with updated history                   │
│                                                             │
│  6. Repeat until text response or max steps reached         │
└─────────────────────────────────────────────────────────────┘

The description field on each tool is doing more work than most developers realise. It is the only signal the model has for deciding which tool to call in a given situation. Vague descriptions produce unreliable tool selection. Write descriptions the way you would document an API for a junior developer: what it does, when to use it, what it returns.

Code Snippet 2 - Multi-Tool Agent (Node.js + Vercel AI SDK)

// agent/supportAgent.js
require('dotenv').config();
const { generateText } = require('ai');
const { anthropic } = require('@ai-sdk/anthropic');
const { tool } = require('ai');
const { z } = require('zod');
const { retrieveKnowledgeTool } = require('../tools/retrieveKnowledge');

// Tool: Look up a customer order
const lookupOrderTool = tool({
  description: `Look up a customer's order by order ID or by user ID and 
    date range. Use when the customer references an order, purchase, 
    subscription, or transaction. Returns order details including status, 
    product, date, and amount.`,
  parameters: z.object({
    orderId: z.string().optional().describe('Specific order ID if known'),
    userId: z.string().optional().describe('Customer user ID'),
    dateRange: z.enum(['today', 'last_7_days', 'last_30_days', 'last_90_days'])
      .optional().describe('Date range to search'),
  }),
  execute: async ({ orderId, userId, dateRange }) => {
    // In production: replace with your actual orders API call
    const mockOrders = {
      'ORD-4821': {
        id: 'ORD-4821', product: 'Pro Plan', amount: 49,
        status: 'active', date: '2025-02-13', userId: 'u_123'
      }
    };
    if (orderId && mockOrders[orderId]) return { found: true, order: mockOrders[orderId] };
    return { found: false, message: 'No orders found matching criteria.' };
  },
});

// Tool: Create a support ticket
const createTicketTool = tool({
  description: `Create a support ticket for issues that cannot be resolved 
    automatically. Use when: the user's problem requires human review, 
    the resolution involves a refund over $200, or the user explicitly 
    requests to speak with a human. Do NOT use for simple FAQ questions.`,
  parameters: z.object({
    userId: z.string().describe('Customer user ID'),
    subject: z.string().describe('Brief description of the issue'),
    priority: z.enum(['low', 'medium', 'high', 'urgent']),
    summary: z.string().describe('Detailed summary of the issue and steps taken'),
  }),
  execute: async ({ userId, subject, priority, summary }) => {
    // In production: replace with your ticketing API (Zendesk, Linear, etc.)
    const ticketId = `TKT-${Date.now()}`;
    return {
      created: true,
      ticketId,
      message: `Ticket ${ticketId} created. A human agent will follow up within 
        ${priority === 'urgent' ? '1 hour' : '24 hours'}.`,
    };
  },
});

// The agent itself
async function runSupportAgent(userMessage, userId, sessionHistory = []) {
  const messages = [
    ...sessionHistory,
    { role: 'user', content: userMessage },
  ];

  const { text, toolCalls, steps } = await generateText({
    model: openai('gpt-5.2'),
    system: `You are a customer support agent for Acme Corp.
      You help customers with orders, billing, and product questions.
      Always search the knowledge base before answering policy questions.
      Create a support ticket if you cannot resolve the issue, or if 
      the customer asks to speak with a human.
      Never guess at policies - if you don't find it in the knowledge base, 
      say so clearly and create a ticket.`,
    messages,
    tools: {
      retrieveKnowledge: retrieveKnowledgeTool,
      lookupOrder: lookupOrderTool,
      createTicket: createTicketTool,
    },
    maxSteps: 8, // prevent infinite loops
  });

  return { text, toolCalls, steps };
}

module.exports = { runSupportAgent };

Tool Output Validation

One failure mode that rarely gets discussed: the LLM receives a tool result and uses it uncritically. If your orders API returns malformed data, the agent will incorporate that malformed data into its answer. Build validation into the execute function, not as an afterthought after the fact.

The pattern is: validate tool inputs (Zod handles this automatically via the schema), validate tool outputs before returning them to the agent, and return structured errors rather than throwing.

Section 5: Error Handling, Fallback Mechanisms, and Safety Guardrails

Production agents fail in ways that demo agents never do. An API times out. The vector database returns zero results. The LLM generates a tool call with an argument that violates your schema.

A user tries to push the agent outside its intended scope. Each of these failure modes needs a defined response - not a crash, not a hallucination, and not a silent failure that your team discovers a week later from a customer complaint.

The Five Failure Modes to Handle

LLM API timeout or rate limit. Your LLM provider will occasionally be slow or unavailable. Implement exponential backoff with a maximum of three retries before returning a graceful fallback message.

Tool execution failure. Any external call can fail - the orders API, the ticketing system, your vector database. Tools should catch all errors internally and return structured error objects. The agent can then decide to retry, use a fallback, or inform the user.

Agent stuck in a loop. When a tool repeatedly returns errors, an agent without a step limit will keep calling that tool indefinitely. Always set maxSteps - typically 8 to 12 for production agents. If the agent hits the limit, return a graceful message rather than an error.

Hallucinated or malformed tool arguments. The LLM will occasionally generate tool arguments that pass schema validation but make no logical sense - an orderId of "none", a priority of "medium" for an account deletion request. Build semantic validation into your execute functions, not just schema validation.

Out-of-scope request. Users will ask your customer support agent to write their cover letter, help with their homework, or find information about your competitors. Define your agent's scope in the system prompt explicitly, and handle out-of-scope detection with a lightweight classifier or a topic guard.

Code Snippet 3 - Retry Logic, Loop Guard, and Graceful Fallback

// utils/agentRunner.js
const { generateText } = require('ai');
const { openai } = require('@ai-sdk/openai');

const MAX_RETRIES = 3;
const RETRY_DELAY_MS = 1000;

async function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

async function runAgentWithGuards(agentConfig, userMessage, sessionHistory = []) {
  let attempt = 0;

  while (attempt < MAX_RETRIES) {
    try {
      const result = await generateText({
        model: openai(agentConfig.model || 'gpt-5.2'),
        system: agentConfig.systemPrompt,
        messages: [
          ...sessionHistory,
          { role: 'user', content: userMessage },
        ],
        tools: agentConfig.tools,
        maxSteps: agentConfig.maxSteps || 10,

        // Called before each step - log for observability
        onStepFinish: ({ stepType, toolCalls, toolResults, usage }) => {
          console.log(JSON.stringify({
            event: 'agent_step',
            stepType,
            toolsCalled: toolCalls?.map(t => t.toolName),
            tokensUsed: usage?.totalTokens,
            timestamp: new Date().toISOString(),
          }));
        },
      });

      return { success: true, result };

    } catch (err) {
      attempt++;
      const isRateLimit = err.status === 429;
      const isTimeout = err.code === 'ETIMEDOUT' || err.status === 504;

      if ((isRateLimit || isTimeout) && attempt < MAX_RETRIES) {
        const delay = RETRY_DELAY_MS * Math.pow(2, attempt); // exponential backoff
        console.log(`Retry ${attempt}/${MAX_RETRIES} after ${delay}ms: ${err.message}`);
        await sleep(delay);
        continue;
      }

      // Exhausted retries or non-retryable error
      console.error(JSON.stringify({
        event: 'agent_error',
        error: err.message,
        attempt,
        timestamp: new Date().toISOString(),
      }));

      return {
        success: false,
        fallbackMessage: `I'm having trouble processing your request right now. 
          Please try again in a moment, or contact support@company.com 
          if this continues.`,
      };
    }
  }
}

// Topic guard - lightweight scope check
// ⚠️  IMPORTANT: This keyword approach is a starting placeholder only.
// It will miss real-world inputs like "My subscription is acting weird"
// (no direct keyword match). For production, replace this with an
// LLM-based classifier as described in the guardrails section above -
// a single fast call to GPT-5.2 with reasoning.effort: low
// is more reliable and takes under 100ms.
function isOutOfScope(userMessage, allowedTopics) {
  const lower = userMessage.toLowerCase();
  const topicKeywords = {
    support: ['order', 'refund', 'billing', 'subscription', 'account', 'payment', 'cancel'],
    hr: ['policy', 'leave', 'holiday', 'benefits', 'contract', 'payroll'],
    sales: ['pricing', 'plan', 'upgrade', 'demo', 'trial', 'enterprise'],
  };

  const relevantKeywords = allowedTopics.flatMap(t => topicKeywords[t] || []);
  return !relevantKeywords.some(kw => lower.includes(kw));
}

module.exports = { runAgentWithGuards, isOutOfScope };

Safety Guardrails

Guardrails are the architectural layer that enforces what your agent is and is not allowed to do. They operate at three levels:

Input guardrails - applied before the agent runs. Check for prompt injection attempts (instructions embedded in user input designed to override the system prompt), topic scope violations, and content policy violations. A fast, cheap classifier model (GPT-5.2 with reasoning.effort: low) can handle this in under 100ms without blocking the main agent execution.

Tool-level guardrails - applied before and after each tool call. Before: validate that the arguments make semantic sense, not just schema sense. After: validate that the result is safe to pass back to the LLM. A tool that returns a large blob of unstructured data should be post-processed before the agent sees it.

Output guardrails - applied to the final agent response before it reaches the user. Check for PII exposure, policy violations, or content that falls outside your acceptable use definition.

Human-in-the-loop: For high-stakes actions - account deletions, refunds above a threshold, data exports - always implement a human approval gate. The Vercel AI SDK supports this natively with the needsApproval flag on individual tools. This is not a nice-to-have for regulated industries; it is a requirement.

Section 6: Monitoring and Observability for Agent Systems

You cannot operate what you cannot see. Traditional application monitoring - uptime, error rate, response time - is necessary but not sufficient for AI agents. An agent can return a 200 status code and produce a useless or wrong answer. You need a second layer of observability that tracks the quality of what the agent did, not just whether the request completed.

What to Log for Every Agent Interaction

At minimum, every agent session should produce a structured log containing:

• Session ID - to trace a full multi-turn conversation

• Each LLM call - model, prompt tokens, completion tokens, latency, step number

• Each tool call - tool name, input arguments, output, execution time, success/failure

• Each retrieval - query sent, number of results, similarity scores, filters applied

• Final response - text returned to user

• Total session cost - calculated from token usage × current model pricing

• Session outcome - resolved, escalated, failed, out-of-scope

Code Snippet 4 - Logging Middleware for Agent Calls

// utils/observability.js

// Pricing as of February 2026. Model prices change frequently.
// Always check current rates at: https://openai.com/api/pricing
const MODEL_PRICING = {
  'gpt-5.2':      { input: 1.75, output: 14.00 }, // per 1M tokens
  'gpt-5.2-chat-latest': { input: 1.75, output: 14.00 },
};

function calculateCost(model, inputTokens, outputTokens) {
  const pricing = MODEL_PRICING[model];
  if (!pricing) return null;
  return (
    (inputTokens / 1_000_000) * pricing.input +
    (outputTokens / 1_000_000) * pricing.output
  );
}

class AgentLogger {
  constructor(sessionId, userId) {
    this.sessionId = sessionId;
    this.userId = userId;
    this.steps = [];
    this.startTime = Date.now();
    this.totalInputTokens = 0;
    this.totalOutputTokens = 0;
  }

  logStep({ model, stepType, toolsCalled, inputTokens, outputTokens, latencyMs }) {
    const stepCost = calculateCost(model, inputTokens, outputTokens);
    this.totalInputTokens += inputTokens || 0;
    this.totalOutputTokens += outputTokens || 0;

    const entry = {
      sessionId: this.sessionId,
      userId: this.userId,
      event: 'agent_step',
      stepType,
      model,
      toolsCalled,
      inputTokens,
      outputTokens,
      stepCost,
      latencyMs,
      timestamp: new Date().toISOString(),
    };

    this.steps.push(entry);
    console.log(JSON.stringify(entry)); // send to your log pipeline
  }

  logRetrieval({ query, resultsFound, topSimilarity, filtersApplied, latencyMs }) {
    const entry = {
      sessionId: this.sessionId,
      event: 'retrieval',
      query,
      resultsFound,
      topSimilarity,
      filtersApplied,
      latencyMs,
      timestamp: new Date().toISOString(),
    };
    console.log(JSON.stringify(entry));
  }

  logSessionEnd({ outcome, finalResponse }) {
    const totalCost = calculateCost(
      'gpt-5.2',
      this.totalInputTokens,
      this.totalOutputTokens
    );
    const summary = {
      sessionId: this.sessionId,
      event: 'session_end',
      outcome,        // 'resolved' | 'escalated' | 'failed' | 'out_of_scope'
      totalSteps: this.steps.length,
      totalInputTokens: this.totalInputTokens,
      totalOutputTokens: this.totalOutputTokens,
      totalCost,
      totalLatencyMs: Date.now() - this.startTime,
      finalResponse: finalResponse?.slice(0, 200), // truncate for logs
      timestamp: new Date().toISOString(),
    };
    console.log(JSON.stringify(summary));
    return summary;
  }
}

module.exports = { AgentLogger };

Choosing an Observability Tool

For most startups and SMEs, two tools cover the majority of your observability needs:

Langfuse (open-source, self-hostable) is the best choice if you want full control over your data, generous free tier, and detailed trace visualisation across multi-step agent sessions. It captures prompts, responses, costs, and execution traces. Strongly recommended for teams that need to debug complex agent behaviour.

Helicone is the best choice if you need something running in production today with minimal setup - it works via a single URL change to your API base URL and immediately logs every LLM call with cost tracking. Less depth than Langfuse for complex agent traces, but extremely fast to instrument.

For teams already on Datadog or similar enterprise APM tools, their LLM observability modules let you add AI monitoring without introducing another vendor.

What to Alert On

Set up alerts for: session error rate exceeding 5%, average session cost exceeding your per-session budget, retrieval returning zero results more than 20% of the time (usually signals a knowledge base freshness problem), and agent hitting the maxSteps limit (usually signals a tool is consistently failing).

Section 7: Cost Optimisation Strategies

Agents cost more than chatbots. A single user request can trigger planning, multiple tool calls, retrieval, and a final generation step - easily 5× the token usage of a single-turn chatbot response. If you are not actively managing cost from the start, you will discover the problem in your monthly invoice.

The good news is that most agent cost problems are architectural, not inevitable. The following strategies can reduce per-session costs by 40–60% without compromising quality.

Know Your Cost Baseline First

Before optimising anything, instrument your agent and measure actual token usage per session type. The formula is straightforward:

// utils/costEstimator.js

function estimateMonthlyCost({
  requestsPerDay,
  avgTokensPerRequest = 3000,
  model = 'gpt-4o-mini',
  cacheHitRate = 0.30,      // 30% of queries answered from cache
  agentMultiplier = 3.0,    // agents use ~3x tokens vs single-turn chatbots
}) {
  const PRICING = {
    // Pricing as of February 2026 - verify latest at https://openai.com/api/pricing
    'gpt-5.2':             { input: 1.75, output: 14.00 }, // per 1M tokens
    'gpt-5.2-chat-latest': { input: 1.75, output: 14.00 },
  };

  const rates = PRICING[model];
  const monthlyRequests = requestsPerDay * 30;
  const cachedRequests  = monthlyRequests * cacheHitRate;
  const llmRequests     = monthlyRequests - cachedRequests;

  // Agents: 60% input tokens (system prompt + history + RAG context), 40% output
  const inputTokens  = avgTokensPerRequest * 0.6 * agentMultiplier;
  const outputTokens = avgTokensPerRequest * 0.4 * agentMultiplier;

  const llmCost = llmRequests * (
    (inputTokens  / 1_000_000) * rates.input +
    (outputTokens / 1_000_000) * rates.output
  );

  return {
    model,
    requestsPerDay,
    monthlyRequests,
    cachedRequests: Math.round(cachedRequests),
    llmRequests: Math.round(llmRequests),
    estimatedMonthlyCost: `$${llmCost.toFixed(2)}`,
  };
}

// Example: customer support agent, 500 requests/day
console.log(estimateMonthlyCost({
  requestsPerDay: 500,
  model: 'gpt-5.2',
  cacheHitRate: 0.30,
}));
// → { estimatedMonthlyCost: '$~55/month' }

Strategy 1: Model Routing

Not every agent step needs your most capable model. Classification tasks - is this request in scope? which tool should I call first? - can run on GPT-5.2 with reasoning.effort: low at significantly lower cost than full reasoning.

Reserve higher reasoning effort for the steps that require genuine reasoning: complex multi-step resolution, handling ambiguous requests, generating final responses for high-stakes situations.

The pattern: use reasoning.effort: low for the orchestration layer and switch to reasoning.effort: high only when confidence is below a threshold or the task complexity warrants it.

Strategy 2: RAG Reduces Context Cost

This is the most underappreciated cost benefit of RAG in an agent context. Without RAG, you would need to include your entire knowledge base in the system prompt - thousands of tokens on every request. With good RAG and metadata filtering, you retrieve only the 3–5 most relevant chunks per query. For a knowledge base of 10,000 documents, this difference in context size can reduce per-request token cost by 70% or more.

This is why the staged hybrid filtering approach from our earlier guide matters for cost, not just accuracy. A pre-filter that eliminates 90% of documents before vector search means your retrieval returns a tight, relevant set - and that tight set keeps your agent's context window lean.

Strategy 3: Semantic Caching

Many agent queries are semantically identical even when worded differently. "What is your refund policy?" and "How do I get my money back?" will retrieve the same documents and produce near-identical responses. A semantic cache stores recent responses as vectors and returns a cached answer when incoming query similarity exceeds a threshold (typically 0.90–0.95).

Production systems with predictable query patterns consistently achieve 20–40% cache hit rates. At 500 requests per day, a 30% cache hit rate eliminates 4,500 LLM calls per month.

Strategy 4: Session-Level Retrieval Caching

Within a single multi-turn session, if the user asks three follow-up questions about the same policy, there is no reason to query your vector database three times. Cache retrieval results in session memory for the duration of the conversation. This is simple to implement and eliminates a meaningful percentage of Supabase query costs in longer sessions.

Section 8: Real-World Case Studies

Theory is useful. Seeing the architecture applied to a real problem is more useful. The following examples are drawn from production work at MTechZilla.

Case Study 1: Customer Support Agent for a SaaS Platform

The problem: A B2B SaaS company with 2,000 customers was handling 150–200 support tickets per day with a two-person support team. Average first response time was 6 hours. Most tickets were questions already answered in the documentation.

The architecture: A customer support agent built on Node.js and the Vercel AI SDK, connected to three tools: retrieveKnowledge (Supabase pgvector over 800 documentation pages), lookupAccount (company's internal API), and createTicket (Zendesk). The agent was given explicit escalation instructions: create a human ticket for any billing dispute, any account deletion request, and any question where retrieval returned found: false.

The RAG setup: Documentation chunked at 400 tokens with paragraph boundary detection. Metadata schema included product_area (6 categories), doc_type (guide, API reference, FAQ, release note), and last_updated. Pre-filter by product_area inferred from the user's account type. Post-filter excluding release notes older than 90 days.

The outcome: 68% of tickets resolved without human involvement in the first month. First response time for escalated tickets dropped to 45 minutes because the agent's ticket summary gave the human agent full context. Estimated 12 hours per week returned to the support team.

What almost went wrong: The first version had no topic guard. Users discovered the agent would happily answer general programming questions and write SQL queries unrelated to the product. Adding a lightweight topic classifier in the input guardrail layer fixed this in the second week.

Case Study 2: Internal HR Knowledge Agent

The problem: A 120-person company's HR team was spending 6–8 hours per week answering the same employee questions about leave policies, benefits, and onboarding procedures. The employee handbook was 180 pages across 14 documents, updated quarterly.

The architecture: A read-only internal agent accessible via Slack, built on Node.js with a single tool: retrieveKnowledge. No write access to any system. Human approval required before any document could be added to the knowledge base.

The RAG setup: Documents segmented by policy category (leave, benefits, payroll, conduct, onboarding). Metadata included effective_date and version. Pre-filter by effective_date within the last 12 months eliminated outdated policy versions automatically - solving the most common failure mode before it happened.

The outcome: HR team reported a 70% reduction in repetitive policy questions within 6 weeks. The agent answered 85% of queries without escalation. The remaining 15% - requests for exceptions, edge cases, sensitive situations - were appropriately escalated to HR with a clear summary of what the employee asked.

What almost went wrong: Version 1 had no access control. Any employee could query any document, including compensation band information that was restricted to managers. Role-based metadata filtering (access_level field) was implemented in week 2, restricting documents to the requesting employee's role.

Conclusion: Your Production Readiness Checklist

Building a production-ready AI agent is not one decision - it is a series of decisions made across architecture, implementation, observability, and operations.

The difference between an agent that works in a demo and one that works reliably at scale comes down to whether you have made all of those decisions deliberately, or left them to chance.

Before you ship, run through this checklist:

Agent fundamentals

• [ ] System prompt defines scope explicitly - what the agent does and does not handle

• [ ] maxSteps is set (8–12 for most production agents)

• [ ] Every tool returns structured errors, not thrown exceptions

• [ ] Graceful fallback message defined for all failure modes

RAG and retrieval

• [ ] Retrieval is implemented as a tool, not a preprocessing step

• [ ] Metadata schema designed before ingestion begins

• [ ] Pre-filter applied for date, category, and access control fields

• [ ] found: false returned explicitly when no results match

• [ ] Knowledge base freshness process defined (who updates it, how often)

Error handling and guardrails

• [ ] Retry logic with exponential backoff for API failures

• [ ] Input guardrail: topic scope check before agent runs

• [ ] Human escalation path defined with explicit trigger conditions

• [ ] High-stakes tool actions (writes, deletions) require validation or approval

Observability

• [ ] Every LLM call logged with tokens, cost, latency, and step number

• [ ] Every tool call logged with name, inputs, outputs, and execution time

• [ ] Every retrieval logged with query, result count, and similarity scores

• [ ] Session-level cost tracked and alertable

• [ ] Error rate alert configured

Cost

• [ ] Baseline cost per session measured before launch

• [ ] Model routing implemented for classification vs. resolution steps

• [ ] Semantic cache in place for high-volume query patterns

• [ ] Monthly cost projection reviewed against budget

Build It With MTechZilla

Building a production-ready AI agent takes more than the right architecture. It takes experience with the failure modes that only appear under real load, integration patterns across the tools your business already uses, and the operational discipline to keep it running as your data and requirements change.

At MTechZilla, we build production AI agents as a service for startups and growing companies - from initial architecture through deployment and monitoring. Our AI Assistant as a Service offering covers the full stack: knowledge base setup with proper metadata design, agent implementation in Node.js, observability instrumentation, and ongoing knowledge base maintenance.

If you are moving from a proof-of-concept to a production deployment and want to get it right the first time, get in touch - we will walk through your use case in a free discovery call.

This guide builds on our earlier work on production RAG systems and metadata filtering. If you found this useful, that guide covers the retrieval foundation in greater depth.