RAG + Agent · Project 2

IT Support Agent

Diagnoses and fixes VPN and tool access issues. Introduces RAG — searching company docs before acting — and parallel tool calls for faster resolution.

Node.js · React · Anthropic Claude · JSON RAG GitHub ↗

New Concepts This Project

RAG — search company docs before acting. LLM never guesses policy. Parallel tools — check VPN + Jira in the same turn. Fewer turns, faster resolution.

System Prompt

You are an IT Support Agent for Acme Corp.

RULES — follow in this exact order:
1. ALWAYS call search_knowledge_base first with the user's issue
2. Read the policy docs before taking any action
3. Call check_* tools to see current status
4. Call enable_* tools only if needed
5. Never enable access without checking status first

You can call multiple tools in parallel when they are independent.

Tool Definitions

[
  {
    "name": "search_knowledge_base",
    "description": "Search company IT policy docs. ALWAYS call this first.",
    "input_schema": {
      "properties": {
        "query": { "type": "string", "description": "Issue description" }
      },
      "required": ["query"]
    }
  },
  {
    "name": "check_vpn_status",
    "description": "Check if employee VPN is active or expired",
    "input_schema": {
      "properties": {
        "employee_id": { "type": "string" }
      }
    }
  },
  {
    "name": "enable_vpn",
    "description": "Renew expired VPN certificate for employee",
    "input_schema": {
      "properties": {
        "employee_id": { "type": "string" }
      }
    }
  },
  {
    "name": "check_tool_access",
    "description": "Check if employee has access to a specific tool",
    "input_schema": {
      "properties": {
        "employee_id": { "type": "string" },
        "tool_name":   { "type": "string" }
      }
    }
  },
  {
    "name": "enable_tool_access",
    "description": "Re-enable expired tool access for employee",
    "input_schema": {
      "properties": {
        "employee_id": { "type": "string" },
        "tool_name":   { "type": "string" }
      }
    }
  }
]

End-to-End Flow

👤 User Input

"My VPN is not working and I can't access Jira"

🧠 LLM Response — Turn 1 (RAG first)

{
  "stop_reason": "tool_use",
  "content": [
    { "type": "text", "text": "Let me search our IT docs first." },
    {
      "type": "tool_use", "id": "tool_001",
      "name": "search_knowledge_base",
      "input": { "query": "VPN not working Jira access" }
    }
  ],
  "usage": { "input_tokens": 480, "output_tokens": 65 }
}

System prompt says "ALWAYS call search_knowledge_base first" — LLM follows it every time.

📚 RAG Result (keyword search)

// Keywords: "vpn", "jira", "not working", "access"
// Searched against in-memory JSON docs
// Tags weighted 2x vs content matches

Results returned:
[
  "VPN certificates expire every 90 days. Renewal is
   automatic if employee is active in HR system.",
  "Jira access expires after 60 days of inactivity.
   IT admin can re-enable via provisioning system."
]

🧠 LLM Response — Turn 2 (parallel check)

{
  "stop_reason": "tool_use",
  "content": [
    {
      "type": "tool_use", "id": "tool_002",
      "name": "check_vpn_status",
      "input": { "employee_id": "EMP-4521" }
    },
    {
      "type": "tool_use", "id": "tool_003",
      "name": "check_tool_access",
      "input": { "employee_id": "EMP-4521", "tool_name": "Jira" }
    }
  ],
  "usage": { "input_tokens": 720, "output_tokens": 120 }
}

Two tools called in ONE turn. Both execute simultaneously. Fewer turns = faster resolution.

🔧 Tool Results — Turn 2

check_vpn_status  → { status: "expired", expired_on: "2026-07-15" }
check_tool_access → { status: "expired", expired_on: "2026-06-30" }

🧠 LLM Response — Turn 3 (parallel fix)

{
  "stop_reason": "tool_use",
  "content": [
    {
      "type": "tool_use", "id": "tool_004",
      "name": "enable_vpn",
      "input": { "employee_id": "EMP-4521" }
    },
    {
      "type": "tool_use", "id": "tool_005",
      "name": "enable_tool_access",
      "input": { "employee_id": "EMP-4521", "tool_name": "Jira" }
    }
  ],
  "usage": { "input_tokens": 960, "output_tokens": 130 }
}

🧠 LLM Response — Turn 4 (Final)

{
  "stop_reason": "end_turn",
  "content": [{
    "type": "text",
    "text": "Fixed both issues!\n\n✅ VPN renewed — valid until Oct 22, 2026\n✅ Jira access restored\n\nBoth had expired. Your VPN certificate is renewed\nautomatically. Jira access was inactive for 60+ days."
  }],
  "usage": { "input_tokens": 1180, "output_tokens": 68 }
}

Why RAG Matters Here

Without RAG: LLM might guess "try restarting your computer" — wrong and unhelpful. With RAG: LLM reads actual company policy first → knows VPN expires every 90 days → checks status → fixes root cause. System prompt design ("ALWAYS call search first") forces grounded reasoning on every turn.

RAG — Keyword Search Implementation

In-memory JSON — no vector DB needed for small doc sets

const docs = [
  {
    tags: ["vpn", "certificate", "expired"],
    content: "VPN certificates expire every 90 days..."
  },
  {
    tags: ["jira", "tool", "access", "expired"],
    content: "Jira access expires after 60 days of inactivity..."
  }
];

function searchKnowledgeBase(query) {
  const keywords = query.toLowerCase().split(" ");
  return docs
    .filter(doc => keywords.some(kw => doc.tags.includes(kw)))
    .map(doc => doc.content);
}

Token Cost

Turn

Input

Output

Cost

Turn 1 (RAG)

480

$0.0000024

Turn 2 (check)

720

120

$0.0000038

Turn 3 (fix)

960

130

$0.0000048

Turn 4 (final)

1180

$0.0000046

Total

3,340

383

~$0.000016