VendarBot

VendarBot is the crawler that reads company websites for Vendar profiles. It identifies itself as VendarBot/1.0 (+https://www.vendar.org/bot).

Last updated 2026-11-02

What it reads

Public pages of agencies listed on Vendar: case studies, portfolio and project pages, services, pricing and about pages. It reads a few dozen pages per company at most, every few months, to keep profiles current. It never logs in and never submits forms.

How it behaves

  • It follows robots.txt.
  • It requests pages from one site one at a time, with a pause between requests.
  • What it reads is quoted with its address on the company's profile, so you can see exactly what we used.

How to limit or block it

Add to your robots.txt:

User-agent: VendarBot
Disallow: /

Blocking it means your profile is built only from independent sources. To correct your profile instead, see Contact or write to hello@vendar.org.