ZuwaduBot

Zuwadu is a guide to places to eat and drink, built around people's own recommendations. It is coming soon at zuwadu.com.

ZuwaduBot is the crawler Zuwadu uses to read venues' public websites, so that each place is represented accurately: its opening hours, its menu, what it serves. It reads only what a venue publishes about itself, and nothing else.

How it behaves

• It identifies itself in every request with the User-Agent ZuwaduBot/<version> (+https://zuwadu.com; dev@remiali.com).
• It makes at most one request per second to any one site.
• It honours robots.txt and text-and-data-mining reservations (TDM Reservation Protocol).
• It backs off when a site answers with 429 Too Many Requests and respects Retry-After.
• It signs its requests (RFC 9421 HTTP Message Signatures, Web Bot Auth) so that sites and their CDNs can verify that a request really comes from ZuwaduBot. The public key is published on this host.

Blocking it

To keep ZuwaduBot off your site, add these lines to your robots.txt:

User-agent: ZuwaduBot
Disallow: /

The change takes effect the next time ZuwaduBot visits.

Contact

Questions, or a request to stop reading a site: dev@remiali.com.