Anubis
Lightweight web firewall that blocks AI scrapers with challenge tests
Anubis is a lightweight Go-based web firewall that challenges incoming requests to filter AI-driven crawlers, protecting small sites without relying on third-party services like Cloudflare.
Overview
Anubis is a Go-written web firewall designed for small-to-medium sites that need protection against AI-driven scraper bots. It evaluates each incoming HTTP request using one or more configurable challenges, allowing you to block unknown crawlers while permitting legitimate traffic.
Capabilities & Deployment
The utility offers a fine-grained policy system where known good bots can be explicitly allowlisted, and unknown agents are challenged or denied. Its lightweight architecture ensures minimal CPU and memory overhead, making it suitable for budget-constrained environments. Anubis can be run as a native binary or containerized via Docker, and integrates easily with existing web servers as a reverse-proxy layer.
When to Use
If you cannot or prefer not to use a managed service like Cloudflare, Anubis provides a self-hosted alternative for mitigating unwanted automated traffic while keeping control over bot policies.
Highlights
Pros
- Lightweight and easy to run on modest hardware
- No reliance on external SaaS providers
- Fine-grained control over bot policies
- Simple configuration using files or environment variables
Considerations
- May block legitimate bots such as the Internet Archive by default
- Requires manual policy management for allowlisting
- Not a full CDN or DDoS mitigation solution
- Limited to HTTP-layer protection only
Fit guide
Great for
- Small community websites seeking self-hosted bot protection
- Projects that cannot use Cloudflare or similar services
- Developers who prefer a Go-based, low-overhead firewall
- Low-budget deployments needing basic scraper mitigation
Not ideal when
- High-traffic enterprises requiring comprehensive DDoS defense
- Sites that depend heavily on search engine indexing
- Environments where false positives must be avoided at all costs
- Users who prefer fully managed security services
How teams use it
Personal blog protection
Blocks AI content scrapers while keeping human visitors uninterrupted.
Open-source documentation site
Prevents automated data harvesting without paying for third-party firewall.
Community forum with limited budget
Provides self-hosted bot filtering, reducing server load from malicious crawlers.
Prototype API endpoint
Ensures only vetted clients reach the backend during early development.
Tech snapshot
Frequently asked questions
How does Anubis differentiate bots?
It uses configurable challenge mechanisms and policy definitions to assess request characteristics, allowing you to block unknown AI crawlers while allowlisting known good bots.
Do I need Cloudflare if I use Anubis?
Anubis is an alternative for sites that cannot or choose not to use Cloudflare; it does not replace full CDN or DDoS services.
Can I allow specific bots like the Internet Archive?
Yes, you can add explicit allowlist entries in the bot policy configuration.
What runtime environments are supported?
Anubis is written in Go and can be run as a native binary or via Docker container.
Is there a GUI for managing rules?
Currently configuration is file-based; a web UI is not provided out of the box.
Project at a glance
Active- Stars
- 17,481
- Watchers
- 17,481
- Forks
- 521
Last synced yesterday