Anubis (software)
 Mascot of Anubis | |
| Original author(s) | Xe Iaso |
|---|---|
| Initial release | January 19, 2025[1] |
| Stable release | 1.18.0[2] 
/ 9 May 2025 |
| Repository | github |
| Written in | Go |
| License | MIT License |
| Website | anubis |
Anubis is a software program that makes web scraping harder by using a proof of work mechanism.[3][4]
It was created by Xe Iaso in response to Amazon's web crawler overloading their Git server, as it did not respect the robots.txt exclusion protocol and would work around restrictions.[3]
It is used by a number of projects, including[5]:
- The GNOME Project's GitLab instance.[3]
- The Linux Kernel Mailing list archives and git server[6][7]
- Sourcehut[8][6]
- FFmpeg[6]
- Wine[6]
- UNESCO[6]
- The Science Olympiad Student Center[6]
- Enlightenment[6]
- FreeCAD[9]
References
[edit]- ^ Iaso, Xe (19 January 2025). "Block AI scrapers with Anubis". xeiaso.net. Retrieved 2 May 2025.
- ^ "Release 1.18.0". 9 May 2025. Retrieved 11 May 2025.
- ^ a b c Edwards, Benj (2025-03-25). "Open source devs say AI crawlers dominate traffic, forcing blocks on entire countries". Ars Technica. Retrieved 2025-05-02.
- ^ Bort, Julie (2025-03-27). "Open source devs are fighting AI crawlers with cleverness and vengeance". TechCrunch. Retrieved 2025-05-02.
- ^ "List of known websites using Anubis".
- ^ a b c d e f g "Anubis works". xeiaso.net. Retrieved 2025-05-09.
- ^ "K. Ryabitsev 🍁 (@monsieuricon@social.kernel.org)". social.kernel.org. 2025-04-03. Retrieved 2025-05-09.
- ^ "You cannot have our user's data". sourcehut.org. Retrieved 2025-05-09.
- ^ "Forum and Wiki Performance; Anubis Deployed". FreeCAD News. 2025-04-30. Retrieved 2025-05-11.