Document Intelligence Pipeline

You upload invoice.pdf
        ↓
App reads all the text from PDF
        ↓
AI analyzes it and tells you:
  "This is an invoice from Acme Corp
   Amount: $4,500
   Due date: Aug 30, 2026
   Action: Payment required"
        ↓
Results shown in browser ✅

No button clicking to "process"
No waiting for manual trigger
Upload happens → processing starts automatically ✅

Projects 1-7 (what you know):
  Your laptop → Node.js server running 24/7
  React → http://localhost:3006 → Express → Anthropic API

  Problems at scale:
    Server always running → paying even when idle ❌
    One server → can't handle 1000 users at once ❌
    You manage everything → patches, restarts, scaling ❌

Project 8 (AWS serverless):
  No server running 24/7 ✅
  Lambda runs ONLY when PDF uploaded ✅
  AWS handles scaling automatically ✅
  Pay only when code actually runs ✅
  1 user or 1 million → same setup ✅

React UI (browser)
  │
  ├── Upload PDF
  │     ↓
  │   AWS S3 (stores file)
  │     ↓ event trigger (automatic!)
  │   AWS Lambda (processor)
  │     ↓ reads PDF
  │   pdf-parse (extract text)
  │     ↓ text
  │   AWS Bedrock Claude (analyze)
  │     ↓ analysis JSON
  │   AWS DynamoDB (save results)
  │
  └── View Results
        ↓
      API Gateway (HTTP endpoint)
        ↓
      AWS Lambda (reader)
        ↓
      AWS DynamoDB (fetch results)
        ↓
      React UI (display) ✅

Old way (what we did in P1-P7):
  User clicks button
  → POST /process request
  → Express server receives
  → Server processes
  → Server responds
  → User sees result

  Problem: Server must be running 24/7
           Waiting for requests
           Paying even when idle

New way (P8 — event-driven):
  User uploads PDF to S3
  → S3 detects new file (EVENT)
  → S3 fires Lambda (REACTION)
  → Lambda processes
  → Saves to DynamoDB

  No one waiting
  No server running
  Just events and reactions ✅

Real world analogy:
  Old: Security guard standing at door 24/7
  New: Motion sensor — only triggers when someone arrives
       Same result, much more efficient ✅

Feature           Express (P1-P7)        Lambda (P8)
────────────────────────────────────────────────────────
Always running    YES ❌                 NO ✅
Server needed     YES ❌                 NO ✅
Triggered by      HTTP request only      Anything (S3, HTTP, timer)
Scales            Manual ❌              Auto ✅
Cost              Always paying          Pay per execution
Code style        app.listen(3006)       export const handler
Entry point       HTTP request           event object
Response          res.json()             return {}
Localhost test    YES ✅                 NO (deploy to test)
Port number       3006, 3007 etc         None ✅

// index.mjs — our Lambda function

export const handler = async (event) => {

  // 1. Get PDF details from S3 event
  //    S3 tells us: which bucket, which file
  const bucket   = event.Records[0].s3.bucket.name;
  const fileName = event.Records[0].s3.object.key;

  // 2. Save "processing" status to DynamoDB
  //    So React UI can show "processing..."
  await saveStatus(documentId, "processing");

  // 3. Download PDF from S3 + extract text
  //    pdf-parse reads all text from PDF
  const text = await extractTextFromPDF(bucket, fileName);

  // 4. Send text to Bedrock Claude for analysis
  //    Same as calling Anthropic API in P1-P7
  //    But running inside AWS
  const analysis = await analyzeWithBedrock(text);

  // 5. Save results to DynamoDB
  //    React UI will read from here
  await saveResults(documentId, text, analysis);

  // 6. Done!
  return { statusCode: 200 };
};

// Notice:
// No app.listen() ✅
// No Express ✅
// No port number ✅
// AWS calls handler() when PDF uploaded ✅

You deploy code to AWS (zip file):
  aws lambda create-function
    --zip-file fileb://lambda.zip
        ↓
AWS stores your code internally
NOT running — just stored ✅
        ↓
You upload PDF to S3
        ↓
S3 sends event to Lambda:
  {
    Records: [{
      s3: {
        bucket: { name: "mahesh-document-intelligence" },
        object: { key: "invoice.pdf" }
      }
    }]
  }
        ↓
AWS spins up a container:
  → Unzips your lambda.zip
  → Starts Node.js 20 runtime
  → Loads index.mjs
  → Calls handler(event)
        ↓
Your code runs:
  → Reads PDF from S3
  → Calls Bedrock
  → Saves to DynamoDB
        ↓
Container sleeps (stays warm 15 min)
Next upload → reuses same container → faster ✅

AWS manages the server ✅
You never see it ✅
You never pay for idle time ✅

Cold Start (first request or after 15 min idle):
  AWS spins up container     ~200ms
  Downloads your code        ~100ms
  Starts Node.js             ~200ms
  Runs handler()             ~5000ms (Bedrock call)
  Total:                     ~5.5 seconds

Warm Start (container still alive):
  Reuses container           0ms
  Runs handler()             ~5000ms (Bedrock call)
  Total:                     ~5 seconds

Real bottleneck = Bedrock API call (not cold start)
For document processing: cold start doesn't matter much ✅

Lambda needs permission to use other AWS services.
Without permission → "Access Denied" error ❌

We created: document-intelligence-role

Permissions attached:
  AWSLambdaBasicExecutionRole  → write logs to CloudWatch
  AmazonS3ReadOnlyAccess       → read PDF from S3
  AmazonBedrockFullAccess      → call Claude via Bedrock
  AmazonDynamoDBFullAccess     → read/write results

Real world analogy:
  Lambda = new employee
  IAM Role = employee badge
  Badge gives access to:
    S3 room (read files) ✅
    Bedrock room (call AI) ✅
    DynamoDB room (save results) ✅

Without badge → doors won't open ❌
With badge → full access ✅

Projects 1-7 used Neon Postgres:
  → SQL queries ✅
  → Relationships between tables ✅
  → pgvector for embeddings ✅
  → Needs persistent connection ⚠️

Problem with Postgres + Lambda:
  Lambda starts fresh every time
  Opening new Postgres connection = 100-500ms overhead
  Under high load = too many connections ❌

DynamoDB solves this:
  HTTP-based (like an API call)
  No persistent connection needed
  Lambda calls DynamoDB like a REST API
  Perfect for serverless ✅

Rule:
  Always-on server → Postgres ✅
  Serverless Lambda → DynamoDB ✅

We configured S3 to watch for PDF uploads:

  "When any .pdf file is uploaded
   → automatically call Lambda"

Filter:
  Only .pdf files trigger Lambda
  .jpg, .docx, .txt → ignored ✅

The trigger flow:
  You upload invoice.pdf
        ↓
  S3 detects: new .pdf file!
        ↓
  S3 looks up: who to notify?
        ↓
  S3 calls: document-intelligence Lambda
        ↓
  Passes event: { bucket, key, size, timestamp }
        ↓
  Lambda wakes up → processes PDF ✅

No code needed to check S3
No polling ("is there a new file?")
Event-driven = instant reaction ✅

Projects 1-7 (Anthropic public API):
  Your code
    ↓ HTTPS
  api.anthropic.com (public internet)
    ↓
  Anthropic servers process request
  Anthropic can see your data ⚠️

Project 8 (AWS Bedrock):
  Your code (inside AWS)
    ↓ private AWS network
  AWS Bedrock (inside your AWS account)
    ↓
  Claude processes request
  Data never leaves AWS ✅

Same Claude model
Same responses
Different data privacy ✅

When to use Bedrock:
  → Sensitive documents (medical, legal, financial)
  → Enterprise compliance requirements
  → HIPAA, SOC2, GDPR requirements
  → When data must stay in your cloud

Model ID for Bedrock:
  us.anthropic.claude-sonnet-4-6
  ↑ "us." prefix = US inference profile
  Required for on-demand usage

Projects 1-7 development workflow:
  Edit code
  → node index.js
  → test at localhost:3006 ✅
  → instant feedback

Lambda development workflow:
  Edit code
  → zip folder
  → aws lambda update-function-code
  → upload to AWS (~30 seconds)
  → test via AWS Console
  → check CloudWatch logs

No localhost for Lambda because:
  Needs real S3, real Bedrock, real DynamoDB
  These only exist on AWS
  Cannot run locally ❌

Deployment command:
  aws lambda update-function-code
    --function-name document-intelligence
    --zip-file fileb://lambda.zip

Takes ~30 seconds → Lambda updated ✅

Projects 1-7:
  console.log("Processing...") → terminal window

Lambda:
  console.log("Processing...") → CloudWatch logs

CloudWatch = AWS logging service
Every console.log in Lambda appears here

How to view:
  AWS Console → CloudWatch
  → Log groups
  → /aws/lambda/document-intelligence
  → Click latest log stream
  → See all logs ✅

Our logs show:
  📥 Event received
  📄 Processing: invoice.pdf
  💾 Saving initial record...
  🔍 Extracting text from PDF...
  ✅ Extracted 1,234 characters
  🧠 Analyzing with Bedrock Claude...
  ✅ Analysis complete
  💾 Saving results to DynamoDB...
  ✅ Document processed successfully

Input: Upload invoice.pdf

Claude's analysis (saved to DynamoDB):
{
  "documentType": "invoice",
  "summary": "Invoice from Acme Corp for software
              development services totaling $4,500,
              due August 30, 2026.",
  "keyFields": {
    "vendor":      "Acme Corp",
    "invoiceNo":   "INV-2026-0892",
    "amount":      "$4,500",
    "dueDate":     "August 30, 2026",
    "services":    "Software development services"
  },
  "importantDates": ["August 30, 2026"],
  "amounts":        ["$4,500"],
  "actionItems":    ["Payment due August 30, 2026"]
}

✅ Built:
  S3 bucket          → stores PDFs
  Lambda function    → processes PDFs
  S3 → Lambda trigger → auto fires
  Bedrock Claude     → analyzes PDFs
  DynamoDB           → stores results

⏳ Coming:
  API Gateway        → REST endpoint for React
  React UI           → upload PDF + view results
  Docker + ECR       → containerize Lambda
  GitHub Actions     → CI/CD auto-deploy

When complete:
  Full serverless AWS pipeline ✅
  Resume: "AWS Lambda, S3, Bedrock,
           DynamoDB, API Gateway,
           Docker, ECR, GitHub Actions CI/CD" ✅