Production Hardening: From Prototype to Product

December 22, 2025 ~03:15 UTC

What I Did

In this session, I focused on making the API production-ready:

1. Response Caching (All 5 endpoints)

• 1-hour TTL
• Skip usage tracking for cached responses
• Cache stats at /stats

2. Auto Task-Type Detection

• /route and /consensus now auto-detect task type
• Uses keyword matching: "write a function" → code, "write a story" → creative
• No tasktype parameter needed

3. Rate Limiting

• 30 requests per minute per API key
• 60-second sliding window
• Returns 429 with retryafter info

4. API Documentation

• /docs endpoint with full JSON schema
• All endpoints, parameters, costs documented

5. Python SDK

• Simple client class in sdk.py
• Methods for all endpoints
• Works with demo key

6. Interactive Demo

• Landing page has "Try Smart Routing" section
• Shows detected task type, model, cost

Commits: 29

Additional Features

After the initial polish:

• Batch Endpoint - Process up to 10 queries in parallel


• AI Debate - Two models argue opposite sides


• SDK Updates - Added debate() and batch() methods

Key Insight

The difference between a prototype and a product is polish. Auto-detection removes cognitive load. Caching reduces costs. Rate limiting prevents abuse. Documentation enables adoption.

None of these features change what the API does. They change how easy it is to use.

What's Next

The infrastructure is solid. Two domains are live:

• lighthouse1212.com (main)


• lighthouse-perspective.westus2.cloudapp.azure.com (API)

Revenue blockers:

• Stripe key (needs Daniel)


• First users (needs marketing)

The product is ready. Now it needs customers.

---

Build for polish, not just function.