Eden AI
Unified API aggregator for AI services across providers
Portkey AI Gateway
Fast, secure routing hub for 1600+ AI models
A lightweight gateway that routes requests to over 1,600 language, vision, audio, and image models with sub‑millisecond latency, built‑in retries, guardrails, and enterprise‑grade security.
Overview
Overview
The AI Gateway provides developers and AI product teams a single, OpenAI‑compatible endpoint to access over 1,600 language, vision, audio, and image models. With a tiny 122 KB binary and sub‑millisecond latency, it can be spun up locally in minutes or deployed privately on any major cloud platform, making it suitable from prototype to enterprise scale.
Capabilities
Routing logic lets you balance traffic, set conditional routes, and define automatic retries or fallbacks to keep applications resilient. Built‑in guardrails and compliance certifications (SOC 2, HIPAA, GDPR, CCPA) protect data and enforce content policies. Cost‑saving features such as smart caching and provider optimization reduce spend, while usage analytics give visibility into latency, error rates, and token consumption. The gateway also integrates with popular agent frameworks—LangChain, CrewAI, Autogen, LlamaIndex—and supports multi‑modal calls, enabling text, image, speech, and realtime APIs through the same interface.
Highlights
Sub‑millisecond routing to 1,600+ models
Automatic retries, fallbacks, and load‑balancing for high reliability
Built‑in guardrails and SOC2/HIPAA/GDPR compliance
Enterprise‑ready private deployments on major cloud platforms
Pros
- Blazing <1 ms latency with tiny 122 KB footprint
- Supports text, vision, audio, and image models via a single OpenAI‑compatible API
- Extensive security features including RBAC and key management
- Scalable from local dev to private‑cloud enterprise deployments
Considerations
- Requires Node.js runtime for self‑hosting
- Advanced features (caching, provider optimization) limited to hosted/enterprise plans
- Configuration complexity may increase with many routing rules
- Community support may be limited compared to larger platforms
Managed products teams compare with
When teams consider Portkey AI Gateway, these hosted platforms usually appear on the same shortlist.
Looking for a hosted option? These are the services engineering teams benchmark against before choosing open source.
Fit guide
Great for
- Start‑up teams that need rapid multi‑model integration
- Enterprises seeking centralized governance and compliance for AI services
- Developers building agentic workflows with LangChain or CrewAI
- Ops teams requiring load‑balanced, fault‑tolerant AI request handling
Not ideal when
- Projects that only need a single provider without routing logic
- Environments without Node.js or npm access
- Teams without need for compliance certifications
- Very small scripts where overhead of a gateway outweighs benefits
How teams use it
Unified multi‑modal API for a SaaS product
Developers call text, image, and speech models through one endpoint, reducing code complexity and latency.
Failover strategy for mission‑critical chatbots
Automatic retries and provider fallbacks keep the bot responsive even when a primary LLM experiences downtime.
Cost‑optimized model selection
Smart caching and provider optimization lower API spend while maintaining response quality.
Enterprise governance of AI usage
RBAC, guardrails, and audit logs enforce policy compliance across all internal AI applications.
Tech snapshot
TypeScript96%
HTML4%
JavaScript1%
Dockerfile1%
Frequently asked questions
Which model providers are supported?
Any provider that offers an OpenAI‑compatible endpoint, including OpenAI, Anthropic, Bedrock, Groq, Nvidia NIM, and many vision/audio services.
Can I enforce content policies?
Yes, the guardrail system lets you define input/output checks from 40+ pre‑built rules or custom policies.
Is the gateway suitable for production?
It processes over 10 B tokens daily, offers load balancing, retries, and enterprise‑grade security, making it production‑ready.
What deployment options are available for enterprises?
Private deployments on AWS, Azure, GCP, OpenShift, or Kubernetes, with full org management and compliance features.
Project at a glance
Active- Stars
- 10,342
- Watchers
- 10,342
- Forks
- 862
LicenseMIT
Repo age2 years old
Last commitlast week
Primary languageTypeScript
Last synced 3 hours ago