Overview

AdBlock Compiler is a Deno-native rewrite of the original @adguard/hostlist-compiler with significant improvements for production use. It provides the same functionality with enhanced performance, Cloudflare Workers support, and a comprehensive API.

✨ Key Features

🚀 Quick Start

Installation

# Install from JSR (recommended)
deno add jsr:@jk-com/adblock-compiler

# Or run directly without installation
deno run jsr:@jk-com/adblock-compiler/cli

Basic Usage

import { compile } from '@jk-com/adblock-compiler';

// Compile from configuration
const rules = await compile({
  name: "My Filter List",
  sources: [
    {
      source: "https://example.com/filters.txt"
    }
  ],
  transformations: ["Deduplicate", "RemoveEmptyLines"]
});

console.log(`Compiled ${rules.length} rules`);

CLI Usage

# Compile from config file
deno run jsr:@jk-com/adblock-compiler/cli -c config.yaml

# Show version
deno run jsr:@jk-com/adblock-compiler/cli --version

🌐 Web Interface & API

AdBlock Compiler is deployed as a production service with a full-featured web interface and REST API.

API Endpoints

• GET /api - API information and examples
• POST /compile - Compile with JSON response
• POST /compile/stream - Compile with SSE progress
• POST /compile/batch - Compile multiple lists in parallel

Example API Request

curl -X POST https://adblock-compiler.jayson-knight.workers.dev/compile \
  -H "Content-Type: application/json" \
  -d '{
    "configuration": {
      "name": "My Filter List",
      "sources": [
        {"source": "https://example.com/filters.txt"}
      ],
      "transformations": ["Deduplicate", "RemoveEmptyLines"]
    }
  }'

📊 Performance Features

Caching with Compression

Compilation results are cached with gzip compression for 1 hour. Cache is automatically invalidated when configuration changes. Compression typically achieves 70-80% size reduction.

Request Deduplication

Identical concurrent requests are automatically deduplicated. Only one compilation executes, with all requests receiving the same result. Check the X-Request-Deduplication: HIT header.

Circuit Breaker

External source downloads include:

• Timeout: 30 seconds per request
• Retry Logic: Up to 3 retries with exponential backoff
• Retry Conditions: 5xx errors, 429 rate limits, timeouts
• No Retry: 4xx client errors (except 429)

Rate Limiting

• Limit: 10 requests per minute per IP address
• Response: HTTP 429 with Retry-After: 60 header
• Batch Limit: Maximum 10 compilations per batch request

🔧 Configuration

AdBlock Compiler supports the same configuration schema as the original hostlist-compiler, ensuring compatibility with existing configurations.

Configuration Example (YAML)

name: My Filter List
description: Custom ad-blocking filter
version: "1.0.0"

sources:
  - name: Local Rules
    source: data/local.txt
    type: adblock

  - name: EasyList
    source: https://easylist.to/easylist/easylist.txt
    transformations:
      - RemoveModifiers
      - Validate

transformations:
  - Deduplicate
  - RemoveEmptyLines
  - InsertFinalNewLine

exclusions:
  - "*.google.com"
  - "/analytics/"

Available Transformations

1. ConvertToAscii - Convert internationalized domains
2. TrimLines - Remove whitespace
3. RemoveComments - Strip comments
4. Compress - Convert hosts to adblock syntax
5. RemoveModifiers - Remove unsupported modifiers
6. InvertAllow - Convert to allowlist
7. Validate - Remove invalid rules
8. ValidateAllowIp - Validate but keep IPs
9. Deduplicate - Remove duplicates
10. RemoveEmptyLines - Clean empty lines
11. InsertFinalNewLine - Add final newline

📚 Documentation & Resources

🆚 Comparison with Original

AdBlock Compiler is a drop-in replacement for @adguard/hostlist-compiler with additional features:

Next Steps