# Cloudflare Workers Integration

Spyglasses integrates seamlessly with Cloudflare Workers to provide edge-based bot detection and AI referrer tracking. This integration is perfect for Webflow sites, e-commerce platforms, and any application that needs to understand and control bot traffic at the edge.

## Quick Start with Template

**New to Cloudflare Workers? Use our ready-to-deploy template:**

<a href="https://deploy.workers.cloudflare.com/?url=https://github.com/orchestra-code/spyglasses-cloudflare-worker-example" target="_blank" rel="noopener noreferrer" style={{
  display: "inline-block",
  backgroundColor: "#f38020",
  color: "white",
  padding: "0px 32px",
  borderRadius: "6px",
  textDecoration: "none",
  fontWeight: "600",
  fontSize: "16px",
  border: "none",
  cursor: "pointer",
  transition: "background-color 0.2s ease",
  marginBottom: "16px"
}}>
  🚀 Deploy to Cloudflare Workers
</a>

**📖 Complete example repository:** [github.com/orchestra-code/spyglasses-cloudflare-worker-example](https://github.com/orchestra-code/spyglasses-cloudflare-worker-example)

**✅ No complex configuration required** - all setup is done via Cloudflare Workers Routes

## Manual Installation

For custom implementations, follow these steps:

1. **Install the package**:
   ```bash
   npm install @spyglasses/cloudflare-worker
   ```

   with yarn:
   ```bash
   yarn add @spyglasses/cloudflare-worker
   ```

   with pnpm:
   ```bash
   pnpm add @spyglasses/cloudflare-worker
   ```

2. **Create your Worker**:
   Create a new `src/index.ts` file:
   ```typescript

   ```

3. **Configure your Worker**:
   Create a `wrangler.toml` file:
   ```toml
   name = "my-spyglasses-worker"
   main = "src/index.ts"
   compatibility_date = "2024-12-30"
   compatibility_flags = ["nodejs_compat"]

   [vars]
   SPYGLASSES_DEBUG = "false"
   ```

## Setting Up Workers Routes

Cloudflare Workers Routes determine which requests are sent to your Worker. This is the recommended way to configure your Worker's scope.

### Dashboard Configuration

1. **Navigate to Workers Routes**: In your Cloudflare dashboard, go to your domain and find **Workers Routes** in the sidebar.

2. **Add a Route**: Click "Add route" and configure:
   - **Route pattern**: `*your-domain.com/*` (matches all requests to your domain)
   - **Worker**: Select your Spyglasses Worker
   - **Zone**: Your domain zone

3. **Configure Request Limit Failure Mode** (Important for high-traffic sites):
   - **Recommended**: Set to **"Fail open (proceed)"** 
   - **Default**: "Fail closed (block)" - ⚠️ Not recommended
   - **Why this matters**: Cloudflare's free plan allows 100,000 Worker requests per day. When this limit is exceeded:
     - **Fail open**: Requests bypass the Worker and go directly to your site (recommended for most users)
     - **Fail closed**: Users receive error pages (can break your site)

![Route Configuration](/images/cloudflare-edit-route.webp)

![Workers Routes Interface](/images/cloudflare-workers-routes.webp)

### 📊 High-Traffic Site Considerations

If your site receives more than 100,000 requests per day and you want comprehensive AI traffic tracking, consider:

- **Cloudflare Paid Plans**: Higher request limits (10M+ requests/month)
- **Selective Routing**: Use specific route patterns (e.g., `/blog/*`) to reduce Worker usage
- **Path Exclusions**: Configure `excludePaths` to skip static assets

### Wrangler Configuration

You can also configure routes in your `wrangler.toml`:

```toml
[[routes]]
pattern = "*your-domain.com/*"
zone_name = "your-domain.com"
```

Or using zone ID:

```toml
[[routes]]
pattern = "*your-domain.com/*"
zone_id = "your-zone-id-here"
```

**Common Route Patterns:**
- `*example.com/*` - All requests to example.com
- `api.example.com/*` - Only API subdomain
- `example.com/blog/*` - Only blog paths
- `*.example.com/*` - All subdomains

For more information, see [Cloudflare's Workers Routes documentation](https://developers.cloudflare.com/workers/configuration/routing/routes/).

## Configuration

### Environment Variables

| Variable | Required | Description | Default |
|----------|----------|-------------|---------|
| `SPYGLASSES_API_KEY` | ❌ | API key for analytics and updates | - |
| `SPYGLASSES_DEBUG` | ❌ | Enable debug logging | `false` |
| `SPYGLASSES_COLLECTOR_ENDPOINT` | ❌ | Custom analytics endpoint | Spyglasses default |
| `SPYGLASSES_PATTERNS_ENDPOINT` | ❌ | Custom endpoint for loading AI and bot patterns | Spyglasses default |
| `SPYGLASSES_CACHE_TTL` | ❌ | Pattern cache TTL in seconds | `3600` |

### Setting Your API Key

For analytics and enhanced patterns, set your API key as a secret:

```bash
npx wrangler secret put SPYGLASSES_API_KEY
# Enter your API key when prompted
```

## Deployment

Deploy your Worker to Cloudflare:

```bash
npx wrangler deploy
```

## Advanced Configuration

For custom configuration, create your own worker:

```typescript

const config: SpyglassesWorkerConfig = {
  debug: true,
  excludePaths: [
    '/api/',
    '/admin/',
    '/_next/',
    /\.(css|js|png|jpg|gif|svg|ico)$/i, // Regex patterns supported
  ],
  platformType: 'my-platform',
  blockingTimeout: 1000, // Max time to wait for logging blocked requests
  awaitBlockedLogging: false, // Don't block response on logging
};

  async fetch(request: Request, env: any, ctx: ExecutionContext): Promise<Response> {
    const spyglassesWorker = createSpyglassesWorker({
      ...config,
      apiKey: env.SPYGLASSES_API_KEY,
      debug: env.SPYGLASSES_DEBUG === 'true',
    });

    return await spyglassesWorker.handleRequest(request, env, ctx);
  },
};
```

## Use Cases

### Webflow Sites

Perfect for Webflow sites that need bot protection. Use a route pattern like:
```toml
[[routes]]
pattern = "*your-site.webflow.io/*"
zone_name = "webflow.io"
```

### E-commerce Protection

Protect product pages from aggressive scrapers:

```typescript
const config: SpyglassesWorkerConfig = {
  excludePaths: [
    '/cart',
    '/checkout',
    '/account',
    '/admin',
  ],
  // Block AI model trainers but allow price comparison bots
};
```

### API Protection

For API-only protection, use a targeted route:
```toml
[[routes]]
pattern = "api.example.com/*"
zone_name = "example.com"
```

### Subdomain Routing

Route multiple subdomains through the same Worker:
```toml
[[routes]]
pattern = "*.example.com/*"
zone_name = "example.com"
```

## Understanding Detection Results

The worker adds custom headers to help you understand what's happening:

```
X-Spyglasses-Processed: true
X-Spyglasses-Blocked: true (if blocked)
X-Spyglasses-Reason: bot|ai_referrer (if blocked)
```

### Detection Types

- **🤖 Bot Detection**: Identifies crawlers, scrapers, and AI trainers
- **🧠 AI Referrers**: Tracks human visitors from AI platforms
- **✅ Legitimate Traffic**: Passes through unmodified

### Blocking Behavior

- **Blocked bots** receive a `403 Forbidden` response
- **AI referrers** are logged but never blocked (they're human visitors)
- **Regular visitors** pass through normally

## Debugging

Enable debug mode to see detailed logging:

```toml
[vars]
SPYGLASSES_DEBUG = "true"
```

Check your Worker logs:

```bash
npx wrangler tail
```

You'll see detailed information about:
- Which patterns matched
- Why requests were blocked or allowed
- Logging success/failure
- Performance metrics

## Performance

- **Sub-millisecond detection** using compiled regex patterns
- **Minimal memory footprint** with smart caching
- **Background logging** to avoid blocking responses
- **Edge distribution** for global low latency
- **Bundle size**: ~66KB total, ~13KB gzipped

## Verifying Installation

After deploying your Worker and setting up routes, you can verify the installation by:

1. **Check logs**: Use `npx wrangler tail` to see real-time logs
2. **Test with debug**: Enable debug mode to see detailed detection info
3. **Dashboard data**: Check your Spyglasses dashboard for incoming data
4. **Headers**: Look for `X-Spyglasses-Processed: true` in responses

## Troubleshooting

### Worker Not Triggering
- Verify your route pattern in Cloudflare dashboard under **Workers Routes**
- Check that the Worker is deployed and the route is correctly configured
- Ensure your domain is using Cloudflare nameservers

### Route Pattern Issues
- Use `*domain.com/*` to match all requests to your domain
- Use `domain.com/path/*` to match specific paths only
- Test route patterns carefully - they're case sensitive for paths

### No Data in Dashboard
- Verify your API key is set correctly as a secret
- Check Worker logs for any errors
- Ensure your Workers Route is correctly configured

### Performance Issues
- Enable debug mode to identify bottlenecks
- Check your exclude paths configuration
- Verify pattern caching is working (look for cache hit logs)

## Security Best Practices

1. **Use Secrets**: Store your API key as a secret, not a variable
   ```bash
   npx wrangler secret put SPYGLASSES_API_KEY
   ```

2. **Limit Scope**: Use specific route patterns to avoid processing unnecessary requests

3. **Monitor Logs**: Enable debug mode in staging to verify behavior

4. **Test Routes**: Verify your route patterns match only intended requests

## Cost Implications for using Cloudflare

Cloudflare Workers usage is included in most Cloudflare plans:
- **Free Plan**: 100,000 requests/day (set your worker to Fail Open to avoid overage)
- **Paid Plans**: 10M+ requests/month included

Need help? Contact support@spyglasses.io