AWS Cloud9
Cloud IDE to write, run, and debug code in the browser
envd
Instantly spin up reproducible AI/ML container environments
envd is a CLI tool that creates isolated, OCI‑compatible container environments for AI/ML projects, supporting local, Kubernetes, and remote builds with caching and reusable Python‑based manifests.
Overview
Overview
envd streamlines the creation of AI/ML development environments by letting developers declare dependencies in a simple Python‑style manifest. With a single envd up command, it builds an OCI‑compatible image that isolates all required tools—Python, Conda, CUDA, shells, and more—eliminating the fragile, manual Dockerfile process.
Who it's for & How it works
Targeted at data scientists, ML engineers, and research teams, envd works on local machines, Kubernetes clusters, or remote build servers. It leverages BuildKit for fast, cached builds, reusing previously downloaded packages from PyPI or APT caches. Images can be pushed to any container registry, enabling easy sharing across teams.
Deployment flexibility
Whether you need a quick local sandbox, a scalable cluster‑based training job, or a remote build on a high‑performance server, envd provides a consistent workflow. The tool also supports importing reusable functions from Git repositories, fostering knowledge reuse within organizations.
Highlights
Python‑based manifest eliminates Dockerfile complexity
OCI‑compatible images can be pushed to any registry
BuildKit‑powered caching and remote build support
Hybrid execution on local machines or Kubernetes clusters
Pros
- Fast, reproducible builds with automatic caching
- Simple CLI integrates with existing Python workflows
- Portable images work across environments and registries
- Supports remote builds for low‑resource machines
Considerations
- Requires Docker 20.10+ and BuildKit enabled
- Limited to declarative Python syntax for manifests
- Advanced customizations may still need manual Dockerfiles
- Learning curve for BuildKit and OCI concepts
Managed products teams compare with
When teams consider envd, these hosted platforms usually appear on the same shortlist.
Looking for a hosted option? These are the services engineering teams benchmark against before choosing open source.
Fit guide
Great for
- Data scientists needing quick, reproducible experiment environments
- ML teams that share container images across developers
- Projects that run on both local workstations and Kubernetes clusters
- Workflows that benefit from build caching to reduce iteration time
Not ideal when
- Users without Docker or BuildKit support on their host
- Projects requiring extensive non‑Python custom Docker logic
- Environments where a full Dockerfile is already standardized
- Teams that prefer GUI‑based environment managers
How teams use it
Prototype a new model locally
Spin up an isolated container with required Python packages and CUDA in minutes, ensuring the prototype runs identically on any machine.
Scale training on a Kubernetes cluster
Build the environment once, push the OCI image, and run distributed training jobs without re‑configuring dependencies.
Share reproducible research
Publish the envd image to a registry, allowing collaborators to pull and reproduce experiments with a single command.
Offload heavy builds to a cloud server
Use remote build mode to compile the environment on a powerful server, then retrieve the cached image for local development.
Tech snapshot
Go91%
Python4%
Shell3%
Makefile1%
Dockerfile1%
Frequently asked questions
What platforms can envd run on?
envd works on any system with Docker 20.10+ and supports local execution, Kubernetes clusters, and remote build servers.
Do I need to write Dockerfiles?
No. envd uses a Python‑style manifest to declare packages and configuration, handling Dockerfile generation internally.
How does caching improve build speed?
BuildKit caches pip, conda, and apt downloads, so repeated builds reuse existing layers instead of re‑downloading packages.
Can I share the built environment with others?
Yes. The resulting OCI image can be pushed to any container registry (Docker Hub, Harbor, etc.) for team-wide reuse.
Is envd limited to Python projects?
While manifests are written in Python, envd also supports R, Julia, and other languages via appropriate install functions.
Project at a glance
Active- Stars
- 2,178
- Watchers
- 2,178
- Forks
- 167
LicenseApache-2.0
Repo age3 years old
Last commit2 weeks ago
Primary languageGo
Last synced 2 hours ago