Building With Infra: A Practical System
A layer that picks the cheapest capable model for each request, saving cost and latency. An AI router inspects the incoming prompt and dispatches it to the smallest model that can handle it — a small local model for classification, a mid-tier for drafting, a frontier model only for hard reasoning. This guide pulls together everything on Onexial tagged infra — 8 connected nodes across definitions, workflows, tool stacks, comparisons, prompts and applied use cases — and orders it the way you would actually learn it: vocabulary first, then process, then tooling, then execution. Every item below links to a full node with its own examples and connections, so you can go as deep as you need without losing the map.
Core concepts behind Infra
Before wiring anything together, the vocabulary has to be precise. These 4 definitions cover the terms that show up in almost every Infra discussion — each one links to a full entry with an example and its own connections inside the graph.
AI Router
A layer that picks the cheapest capable model for each request, saving cost and latency.
Quantization
Shrinking a model by lowering weight precision.
Inference
Running a trained model to produce outputs.
Model Routing
Sending each request to the cheapest model that can handle it.
Trade-offs and comparisons
Most Infra decisions are trade-offs rather than right answers. These 4 comparisons break down the real differences, when each option wins, and the recommendation for the common case.
Vercel vs Cloudflare Workers for AI Apps
Two edge platforms, two philosophies: DX-first vs infra-first. Both ship AI apps well.
Groq vs Together AI
Fast open-model inference: throughput vs breadth.
Chroma vs Qdrant vs Pinecone
Open-source local vs managed cloud vector databases.
Cloudflare Workers vs Vercel Functions
Two edge runtimes for AI apps.
Frequently asked questions
- What is AI Router?
- An AI router inspects the incoming prompt and dispatches it to the smallest model that can handle it — a small local model for classification, a mid-tier for drafting, a frontier model only for hard reasoning. Products: OpenRouter, Portkey, LiteLLM.
- What is an example of AI Router?
- A support agent routes 80% of tickets to GPT-4o mini, 15% to GPT-4o, and 5% to Claude Opus, cutting bill by ~70% with no quality drop.
- Why does AI Router matter for AI and automation?
- A layer that picks the cheapest capable model for each request, saving cost and latency. It connects to the workflows, prompts and tool stacks linked on this page, so you can move from definition to execution without leaving Onexial.
- What is the difference in Vercel vs Cloudflare Workers for AI Apps?
- Vercel wins on developer experience and framework integration. Cloudflare wins on cost, edge coverage and integrated primitives (KV, D1, R2, Vectorize, Workers AI).
- What are the main points of comparison?
- DX: Vercel: best-in-class, Next.js native vs Cloudflare: improving fast (Wrangler, Pages) · AI primitives: AI SDK, AI Gateway vs Workers AI, AI Gateway, Vectorize · Cost at scale: Higher (per-invocation bandwidth) vs Lower (generous free tiers) · Cold starts: Fast, but paid tier for consistency vs V8 isolates: near-zero cold start · Data / storage: Vercel KV, Postgres via partners vs KV, D1, R2, Durable Objects natively
- Which one should I choose?
- Ship prototypes on Vercel. Move heavy inference or high-QPS endpoints to Cloudflare Workers.
- What is Quantization?
- Quantization converts 16-bit weights to 8-bit, 4-bit, or lower. It shrinks model size 2-4x with a small accuracy hit — the key trick behind running Llama-scale models on a laptop.
- What is an example of Quantization?
- A 70B model quantized to 4-bit fits in ~40GB VRAM and runs on a single H100.