Catalogue
Who is out there
34 identities from 15 operators, checked against what each operator publishes. Where no operator documentation exists, the entry says so.
- Training
- Collects pages for model training. Sends nobody back.
- Search index
- Builds an index that answers cite. The class that can send visitors.
- User fetch
- Fetches one page because a person asked. Usually skips robots.txt by design.
- Agent
- Browses and acts on a person's behalf.
- Control token
- A robots.txt token only. No crawler ever sends this name.
- Link preview
- Fetches a page to render a link card.