WaybookBot

A small, polite crawler reading café and restaurant websites.

Waybook is a free route guide for cyclists. When a café or restaurant already on the map has its own website, WaybookBot visits it, at most a handful of pages, to read opening hours and a photo — the things a rider actually needs standing outside a closed door. What it finds is shown on the place’s Waybook page with a link back to the venue’s own site.

It identifies itself with the user agent WaybookBot/0.1 (+https://waybook.cc/bot), honors robots.txt, and keeps at least two seconds between requests to the same site. It never fills in a form, never creates an account, and never reads anything behind a login.

To keep it out entirely, add to your site’s robots.txt:

User-agent: WaybookBot
Disallow: /

Questions, or want your venue removed: jakub.foglar@gmail.com. More about the project at waybook.cc.