Eden AI
Unified API aggregator for AI services across providers
Open-Source Projects
Discover top open-source software, updated regularly with real-world adoption signals.
Discover top open-source software, updated regularly with real-world adoption signals.
Unified gateway for secure, scalable generative AI traffic
Envoy AI Gateway routes, authenticates, and rate‑limits traffic to multiple LLM providers, offering a two‑tier architecture for centralized control and fine‑grained model access.
Envoy AI Gateway provides a cloud‑native, Envoy‑based solution for routing client requests to a wide range of generative AI services. By handling authentication, top‑level routing, and global rate limiting at a centralized Tier One gateway, it gives operators a single control point for policy enforcement across providers such as OpenAI, Azure OpenAI, Google Gemini, Anthropic, and many others.
A second Tier Two gateway sits closer to self‑hosted model clusters, offering fine‑grained access control and an endpoint picker that can direct inference traffic to the most suitable model instance. The two‑tier pattern lets organizations combine SaaS LLM APIs with on‑premise deployments while maintaining consistent security and observability. Deployment follows standard Envoy Gateway practices on Kubernetes, and the project includes quick‑start guides, documentation, and an active CNCF‑aligned community.
When teams consider Envoy AI Gateway, these hosted platforms usually appear on the same shortlist.
Looking for a hosted option? These are the services engineering teams benchmark against before choosing open source.
Enterprise AI platform consolidating OpenAI, Azure, and internal models
Provides a single entry point with unified auth, rate limits, and routing, simplifying policy enforcement.
Multi‑tenant SaaS offering LLM‑powered features
Isolates tenant traffic via Tier Two gateways, enabling per‑tenant model selection and usage quotas.
Edge deployment routing requests to on‑premise LLM cluster
Uses Tier Two gateway to pick optimal inference endpoints, reducing latency and cost.
Rapid prototyping across diverse AI providers
Allows developers to switch providers without code changes, leveraging the same gateway configuration.
The gateway runs on Envoy Gateway and is configured via YAML/Envoy resources; the control plane is written in Go but any language can interact through standard HTTP/HTTPS.
Yes, the Tier One gateway supports authentication plugins (e.g., JWT, OAuth) that can be configured to validate incoming client requests.
Custom providers can be integrated by defining a new upstream cluster and routing rules, leveraging Envoy’s extensible filter architecture.
Currently the project is self‑hosted; no official managed service is offered.
It selects the most appropriate model endpoint based on routing criteria such as latency, version, or load, allowing fine‑tuned optimization.
Project at a glance
ActiveLast synced 4 days ago
