Multi-LLM Playground & Router

A web application for comparing responses from multiple LLM providers side-by-side. Send a single prompt to multiple providers simultaneously, compare responses, track latency and costs, and save winning model configurations for future use.

Features

• Multi-Provider Comparison: Run prompts against Claude (Anthropic), GPT-4 (OpenAI), Gemini (Google), and Llama (Groq) simultaneously
• Parallel Execution: All provider calls run in parallel using Promise.allSettled for resilient fan-out
• Cost & Latency Tracking: Automatic token counting and cost calculation per provider
• Saved Routes: Save winning model configurations for quick access
• Per-Provider API Keys: Each user brings their own API keys, encrypted at rest
• Response History: Browse past prompt runs with fastest/cheapest provider badges

Tech Stack

• Framework: Next.js 14 (App Router)
• Database: SQLite (dev) / PostgreSQL (prod) via Prisma ORM
• Auth: Clerk
• Styling: Tailwind CSS
• Error Tracking: Sentry

Supported Providers & Models

Provider	Models
Anthropic	Claude 3.5 Sonnet, Claude 3 Haiku
OpenAI	GPT-4o, GPT-4o Mini
Google	Gemini 1.5 Pro, Gemini 1.5 Flash
Groq	Llama 3 70B

Getting Started

Prerequisites

• Node.js 18+
• API keys for at least one provider

Installation

# Clone the repository
git clone https://forgejo.tail018ac4.ts.net/joe/MultiLLMRouter
cd MultiLLMRouter

# Install dependencies
npm install

# Copy environment variables
cp .env.example .env
# Edit .env with your credentials

# Run database migrations
npx prisma migrate dev

# Start development server
npm run dev

Environment Variables

# Database
DATABASE_URL="postgresql://user:password@localhost:5432/llm-playground"
# For local dev: DATABASE_URL="file:./dev.db"

# Clerk Authentication
NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY="pk_test_..."
CLERK_SECRET_KEY="sk_test_..."

# API Key Encryption (32-byte hex string = 64 characters)
APP_ENCRYPTION_KEY="your-64-char-hex-key"

# Sentry (optional)
SENTRY_DSN="https://...@sentry.io/..."

Generating an Encryption Key

openssl rand -hex 32

API Endpoints

POST /api/run-prompt

Run a prompt across multiple providers.

{
  "prompt": "Explain quantum computing",
  "models": ["claude-3-5-sonnet", "gpt-4o", "gemini-1.5-pro"],
  "systemPrompt": "You are a helpful assistant"
}

GET /api/providers

Returns list of configured providers and their status.

GET /api/history

Returns paginated prompt history.

CRUD /api/routes

Manage saved routes (winner model configurations).

CRUD /api/settings

Manage per-provider API keys.

Project Structure

/app
  /api
    /run-prompt      # Fan-out prompt execution
    /providers      # Provider configuration
    /history        # Run history
    /routes         # Saved routes
    /settings       # API key management
  /playground       # Main UI
  /routes          # Route management
  /settings        # Settings page
/components
  PromptInput.tsx     # Textarea + model selector
  ResponseCard.tsx    # Per-model results
  ResponseGrid.tsx    # Results layout
  RouteBadge.tsx      # Winner indicator
/lib
  /providers         # LLM SDK wrappers
  cost.ts           # Token cost calculator
  db.ts             # Prisma client

Pricing

Token pricing is calculated using published per-million-token rates:

Model	Input ($/M)	Output ($/M)
Claude 3.5 Sonnet	$3.00	$15.00
Claude 3 Haiku	$0.25	$1.25
GPT-4o	$5.00	$15.00
GPT-4o Mini	$0.15	$0.60
Gemini 1.5 Pro	$1.25	$5.00
Gemini 1.5 Flash	$0.075	$0.30
Llama 3 70B (Groq)	$0.59	$0.79

Development

# Run type checking
npm run typecheck

# Run linting
npm run lint

# Build for production
npm run build

# Database operations
npx prisma studio        # Visual database editor
npx prisma migrate dev   # Apply migrations

Roadmap (v2)

• Proxy endpoint for production routing
• Stripe billing integration
• Team/organ accounts
• Streaming responses
• Model parameter controls (temperature, max_tokens)
• Public playground sharing

License

MIT