Anthropic updated its crawler documentation to list separate Claude bots for training, search indexing, and user requests, ...
Anthropic explains how its bots handle AI training, live queries, and search results, and what opting out means for visibility.