# Traffic Control Dashboard

The Traffic Control dashboard lets you configure which AI agents and bots can access your website. Rules are managed centrally in Spyglasses and automatically enforced by your site integration.

For comprehensive Traffic Control documentation including implementation details and example configurations, see [Traffic Control Guide](/docs/getting-started/traffic-control).

## What You'll Learn

In this guide, you'll learn:

- How to configure bot blocking settings
- How to create custom block and allow rules
- How rules sync to your site
- Best practices for bot management

## How It Works

Traffic Control is managed centrally in Spyglasses at the property level. Your site integration (middleware, plugin, or worker) automatically syncs the rules and enforces them at the edge. You don't need to ship code changes to update your block/allow lists.

### Rule Enforcement

1. You configure rules in Spyglasses
2. Your integration periodically syncs rules
3. When bots visit, rules are applied automatically
4. Blocked bots receive HTTP 403 Forbidden responses
5. Allowed bots access your content normally

## Global Settings

Enable broad protections with a single toggle:

### Block AI Model Trainers

When enabled, this automatically blocks known model-training crawlers:
- GPTBot (OpenAI)
- ClaudeBot (Anthropic)
- Meta-ExternalAgent
- Google-Extended
- CCBot (Common Crawl)
- And others

This prevents your content from being used in future AI training while still allowing AI assistants to access your content for current conversations.

### Other Global Settings

Depending on your configuration options:
- Block all scrapers
- Block suspicious user agents

### Browse Categories

View bot categories and their associated patterns:
- AI Model Trainers
- AI Assistants
- Search Crawlers
- SEO Bots
- Social Bots
- Scrapers

For each category, you can set Block or Allow.

### Browse Specific Patterns

Drill down to specific bots or user agent patterns:
- Set rules for individual bots
- Create pattern-based rules
- Override category-level settings

### Rule Priority

More specific rules override broader rules:
1. Specific bot rules (highest priority)
2. Pattern-based rules
3. Category rules
4. Global settings (lowest priority)

## Rules Summary

The Rules Summary view shows exactly what's being enforced:

- All blocked categories and patterns
- All allowed overrides
- Effective rule set

This is what your site actually enforces—review it to ensure your configuration is correct.

## Creating Rules

### To Block a Bot or Category

1. Find the bot or category in the browse view
2. Click to select it
3. Choose "Block"
4. Save changes

### To Allow a Blocked Bot

If a bot is blocked by category but you want to allow it:
1. Find the specific bot
2. Choose "Allow"
3. Save changes

The allow rule overrides the category block.

### Pattern-Based Rules

For advanced filtering:
1. Create a custom pattern
2. Enter a user agent substring or regex
3. Set Block or Allow
4. Save changes

## Path Exclusions

You can optionally exclude certain paths from Traffic Control in your code. This is useful for:
- Health check endpoints
- Admin pages
- Internal APIs
- Public assets

See [Traffic Control Guide](/docs/getting-started/traffic-control) for implementation details.

## Testing Your Configuration

After configuring rules:

1. **Check Spyglasses dashboard**: Review blocked vs. allowed traffic
2. **Monitor server logs**: Verify blocked requests are handled correctly
3. **Test important bots**: Ensure Googlebot and other critical bots can access
4. **Verify search access**: Check Google Search Console for crawl issues

## Best Practices

### Start Conservative

Begin with minimal blocking:
1. Enable "Block AI Model Trainers" if desired
2. Monitor traffic for a week
3. Add additional rules based on what you observe

### Always Allow Search Engines

Ensure legitimate search engines can access your content:
- Googlebot (required for Google Search)
- Bingbot
- DuckDuckBot
- YandexBot (if targeting that market)

Blocking search engines harms your visibility.

### Don't Block AI Assistants

If you want AI visibility, don't block AI assistant bots. Block only training crawlers if you want to prevent future AI training while maintaining current AI visibility.

### Review Regularly

Periodically review:
- What's being blocked
- Any new bots that appeared
- Whether rules are still appropriate

### Monitor Impact

Watch your analytics after making changes:
- Did bot traffic decrease as expected?
- Are there unintended side effects?
- Is legitimate traffic affected?

## Troubleshooting

### Bot Still Getting Through

If blocked bots still access your site:
1. Check if they match an allow rule
2. Verify your patterns are correct
3. Ensure your integration synced rules

### Legitimate Traffic Blocked

If you're accidentally blocking legitimate traffic:
1. Add specific allow rules for important bots
2. Review custom patterns for overly broad matches
3. Check category blocks for unintended coverage

### Rules Not Syncing

If changes aren't taking effect:
1. Verify your integration is running
2. Check for sync errors in logs
3. Force a manual sync if available
4. Verify API connection

## Related

- [Traffic Control Guide](/docs/getting-started/traffic-control) - Comprehensive implementation guide
- [Bot Traffic](/docs/dashboards/bot-traffic) - See all bot visits
- [Bot Directory](/bots) - Full list of tracked bots
- [Getting Started](/docs/getting-started/overview) - Install the integration