Drop-in compatible
Point your existing OpenAI-compatible client at our base URL and you are live.
Platform · Inference Endpoints
Production-grade inference, one API away.
Serve leading open-source models through managed, autoscaling endpoints with an OpenAI-compatible API. Start with 20M free tokens and scale to billions without touching a server.
$600 in launch credit + 20M free inference tokens. No commitment.
Engineered for
- 13 global regions
- CPU & GPU compute
- Self-healing orchestration
- Zero-trust security
- OpenAI-compatible API
- Sovereign & air-gapped options
Inference Endpoints
Why teams choose it.
Managed, autoscaling endpoints for leading open-source models behind an OpenAI-compatible API.
Fast at any scale
Autoscaling endpoints with streaming responses keep latency low from first token to last.
Open models, your choice
Choose from a curated catalog of leading open-source models for chat, code, embeddings and vision.
Capabilities
Everything production inference needs.
Every capability is available through the dashboard, the API and the AI ops copilot.
OpenAI-compatible API
POST /api/v1/chat/completions with streaming responses. Existing SDKs and frameworks work by changing a base URL and an API key.
Curated model catalog
List available models at any time with GET /api/v1/models.
Autoscaling
Capacity follows demand automatically, from a trickle to a flood.
Fine-tuning
Adapt open models to your domain and deploy them as private endpoints.
Scoped API keys
Create, rotate and revoke keys in Dashboard → Security.
Usage analytics
Tokens, latency and spend per key and per model, in real time, so you always know exactly what each feature costs and how it performs.
- Free inference tokens
- 20M
- Global regions
- 13
- Failover target
- <60s
- Usage metering
- Per token
Figures describe platform capabilities and service-level targets. Contracted commitments are set out in our Service Level Agreement.
Global cloud fabric
Inference Endpoints, everywhere you operate.
Available across 13 regions on six continents, connected by a private, encrypted fabric with automatic failover and per-region data residency.
- US East · Ashburn
- US West · Hillsboro
- Canada Central · Montréal
- South America · São Paulo
- EU West · Dublin
- EU Central · Frankfurt
- EU North · Stockholm
- Middle East · Dubai
- Africa South · Johannesburg
- Asia South · Mumbai
- Asia Southeast · Singapore
- Asia Northeast · Tokyo
- Australia East · Sydney
Use cases
What teams build with Inference Endpoints.
Starting points that deliver value in days.
Chat assistants
Customer-facing and internal assistants with streaming responses.
Retrieval-augmented generation
Ground answers in your own documents and data.
Coding assistants
Code completion, review and explanation inside your tools.
Extraction and summarization
Turn unstructured documents into structured data.
Agents
Multi-step reasoning loops with tool calls and memory.
Translation
Localize content and conversations across dozens of languages.
Security & compliance
Secure at every layer.
Zero-trust access, encryption everywhere and immutable audit trails are standard on every product, on every plan.
- Prompts and completions are encrypted in transit and never used to train shared models.
- Encryption at rest and in transit, with customer-managed keys available.
- SSO, role-based access control and scoped API keys.
- Your data is never used to train shared models.
- Built to align with SOC 2, ISO 27001, HIPAA and FedRAMP-moderate control frameworks.
Zero-trust by default
Every request authenticated, authorised and encrypted.
Built to align with control frameworks
- SOC 2
- ISO 27001
- HIPAA
- GDPR
- FedRAMP Moderate
- PCI DSS
AES-256
at rest
TLS 1.3
in transit
Immutable
audit trail
FAQ
Frequently asked questions
Straight answers from the team that runs the fabric. Still curious? Our architects reply fast.
Is the API really OpenAI-compatible?
Yes. The chat completions and models endpoints follow the familiar request and response shapes, so most SDKs work by changing the base URL and API key.
Which models are available?
A curated catalog of leading open-source models for chat, code and more. Call GET /api/v1/models for the current list.
Is my data used for training?
No. Prompts and completions are processed to serve your request and are never used to train shared models.
How are tokens billed?
Your first 20M tokens are free. After that, usage is metered per token and invoiced every 30 days.
Ship your first endpoint before lunch.
Start with $600 in launch credit and 20M free inference tokens, double the typical big-cloud free trial. No commitment, live in minutes.
Double the typical big-cloud free trial · Per-second metering · Live in minutes