Ask HN: Best AI Gateway?
TL;DR
An HN user is looking for an AI gateway for an agent running on val.town and currently calls Anthropic's router directly. The shortlist includes OpenRouter, Vercel AI Gateway, Cloudflare AI Gateway, or something else. OpenRouter seems broad on models, but adds a 5.5% pay-as-you-go platform fee. The real issue is cost pass-through to customers, missing features such as prompt caching, and a market flooded with vendor-written comparison posts.
Nauti's Take
The market is pitching gateways as the next mandatory layer in the AI stack. Maybe, but not by default.
If a team already runs reliably through Anthropic, it should not switch because of provider FOMO, but because the requirements are clear: cost control, caching, audit logs, fallbacks, rate limits, and data residency. Without that checklist, a gateway is just another middleman that takes a fee and makes debugging harder.
Briefingshow
AI gateways promise less provider mess, centralized billing, fallbacks, logging, and easier model switching. For teams that pass inference costs to customers, the gateway layer quickly becomes a business decision: a 5.5% fee, missing prompt caching, or opaque routing rules can directly affect margin and performance.
Sources