API Management
Cache
Fastly CDN

Fastly CDN

Introduction

Fastly is a modern edge cloud platform built on top of Varnish Cache that provides Content Delivery Network (CDN), security, and edge computing services. Unlike traditional CDNs, Fastly provides real-time content updates (measured in milliseconds, not minutes) and powerful VCL customization capabilities for each service domain.

This documentation covers how Fastly operates as a CDN and how VCL configuration is managed and deployed for different domains like www.prabasiva.com and api.prabasiva.com.

Fastly CDN Architecture

Key Components

  1. Point of Presence (POP): Fastly has 80+ POPs globally, each running Varnish
  2. Shield: Optional intermediate cache layer between edge and origin
  3. Anycast Network: DNS routes clients to nearest POP automatically
  4. VCL Engine: Varnish-based request processing per domain
  5. Real-time Purging: Instant cache invalidation across all POPs

Request Flow Through Fastly CDN

Multi-Domain Service Architecture

Fastly supports multiple domains within a single service or separate services. Each domain can have different VCL logic, caching rules, and backend configurations.

VCL Management for Multiple Domains

Service Structure

Each Fastly service can have:

  • Multiple domains mapped to the service
  • Domain-specific VCL logic using conditionals
  • Shared VCL snippets for common logic
  • Backend definitions for each origin server
  • Version control for VCL configurations

Domain-Specific VCL Example

# vcl_recv - Route by domain
sub vcl_recv {
    # Domain: www.prabasiva.com
    if (req.http.host == "www.prabasiva.com") {
        set req.backend = www_origin;

        # Static assets - long cache
        if (req.url ~ "^/static/|\.(?:css|js|jpg|png|svg|woff2)$") {
            set req.http.X-TTL = "1 week";
        }

        # HTML pages - shorter cache
        else {
            set req.http.X-TTL = "5 minutes";
        }
    }

    # Domain: api.prabasiva.com
    else if (req.http.host == "api.prabasiva.com") {
        set req.backend = api_origin;

        # API endpoints - no cache for POST/PUT/DELETE
        if (req.method != "GET" && req.method != "HEAD") {
            return(pass);
        }

        # GET requests - short cache with stale-while-revalidate
        set req.http.X-TTL = "30 seconds";
        set req.http.X-Grace = "2 hours";
    }

    # Domain: blog.prabasiva.com
    else if (req.http.host == "blog.prabasiva.com") {
        set req.backend = blog_origin;

        # Blog posts - long cache
        set req.http.X-TTL = "1 hour";
    }

    return(hash);
}

# vcl_backend_response - Set TTL based on domain logic
sub vcl_backend_response {
    # Apply TTL from vcl_recv
    if (req.http.X-TTL) {
        set beresp.ttl = std.duration(req.http.X-TTL, 60s);
    }

    # Apply grace period
    if (req.http.X-Grace) {
        set beresp.grace = std.duration(req.http.X-Grace, 1h);
    }

    # Enable stale-while-revalidate for API domain
    if (req.http.host == "api.prabasiva.com") {
        set beresp.stale_while_revalidate = 3600s;
    }

    return(deliver);
}

# vcl_deliver - Add custom headers per domain
sub vcl_deliver {
    # Add cache status header
    if (obj.hits > 0) {
        set resp.http.X-Cache = "HIT";
        set resp.http.X-Cache-Hits = obj.hits;
    } else {
        set resp.http.X-Cache = "MISS";
    }

    # Domain-specific headers
    if (req.http.host == "www.prabasiva.com") {
        set resp.http.X-Domain = "www";
    } else if (req.http.host == "api.prabasiva.com") {
        set resp.http.X-Domain = "api";
        set resp.http.X-API-Version = "v1";
    }

    return(deliver);
}

VCL Deployment Workflow

Deployment Steps

1. Local Development & Testing

# Install Varnish locally for testing
brew install varnish
 
# Test VCL syntax
varnishd -C -f your-config.vcl
 
# Run local Varnish instance
varnishd -a localhost:8080 -f your-config.vcl
 
# Test with curl
curl -H "Host: www.prabasiva.com" http://localhost:8080/

2. Version Control

# VCL files in Git repository
git add vcl/www.prabasiva.com.vcl
git add vcl/api.prabasiva.com.vcl
git commit -m "Update cache TTL for API endpoints"
git push origin feature/update-api-cache

3. Fastly API Deployment

# Using Fastly CLI
fastly service-version clone --version=latest
 
# Upload VCL snippet
fastly vcl snippet create \
  --version=<VERSION> \
  --name="www_domain_logic" \
  --type=recv \
  --content="$(cat vcl/www.prabasiva.com.vcl)"
 
# Validate the version
fastly service-version validate --version=<VERSION>
 
# Activate the version
fastly service-version activate --version=<VERSION>

4. Using Fastly Web UI

  1. Navigate to your service in Fastly console
  2. Click "Edit configuration" to clone current version
  3. Go to "VCL snippets" or "Custom VCL"
  4. Upload or edit VCL code
  5. Click "Validate" to check syntax
  6. Click "Activate" to deploy

VCL Snippet Organization

Fastly allows organizing VCL into snippets for better maintainability:

Example Snippet Structure

# Snippet: www_domain_routing (Type: recv, Priority: 100)
if (req.http.host == "www.prabasiva.com") {
    set req.backend = www_origin;
    call www_caching_rules;
}

# Snippet: www_caching_rules (Type: recv, Priority: 101)
if (req.url ~ "^/static/") {
    set req.http.X-Cache-TTL = "604800";  # 1 week
}

# Snippet: api_domain_routing (Type: recv, Priority: 100)
if (req.http.host == "api.prabasiva.com") {
    set req.backend = api_origin;
    call api_caching_rules;
}

# Snippet: api_caching_rules (Type: recv, Priority: 101)
if (req.method != "GET" && req.method != "HEAD") {
    return(pass);
}

# Snippet: security_headers (Type: deliver, Priority: 100)
set resp.http.Strict-Transport-Security = "max-age=31536000";
set resp.http.X-Content-Type-Options = "nosniff";
set resp.http.X-Frame-Options = "DENY";

Fastly Service Configuration

Backend Definitions

# Backend: www.prabasiva.com
backend www_origin {
    .host = "www-origin.prabasiva.com";
    .port = "443";
    .ssl = true;
    .ssl_sni_hostname = "www-origin.prabasiva.com";
    .ssl_cert_hostname = "www-origin.prabasiva.com";
    .ssl_check_cert = always;

    .connect_timeout = 5s;
    .first_byte_timeout = 30s;
    .between_bytes_timeout = 10s;

    .max_connections = 200;

    # Health check
    .probe = {
        .request = "GET /health HTTP/1.1"
                   "Host: www-origin.prabasiva.com"
                   "Connection: close";
        .interval = 30s;
        .timeout = 5s;
        .window = 5;
        .threshold = 3;
    }
}

# Backend: api.prabasiva.com
backend api_origin {
    .host = "api-origin.prabasiva.com";
    .port = "443";
    .ssl = true;
    .ssl_sni_hostname = "api-origin.prabasiva.com";
    .ssl_cert_hostname = "api-origin.prabasiva.com";
    .ssl_check_cert = always;

    .connect_timeout = 3s;
    .first_byte_timeout = 15s;
    .between_bytes_timeout = 10s;

    .max_connections = 500;

    # Healthcheck for API
    .probe = {
        .request = "GET /api/health HTTP/1.1"
                   "Host: api-origin.prabasiva.com"
                   "Connection: close";
        .interval = 10s;
        .timeout = 3s;
        .window = 5;
        .threshold = 3;
    }
}

Shield Configuration

Shield is Fastly's origin protection feature that adds an intermediate cache layer.

Benefits of Shield

  1. Reduced Origin Load: Only shield fetches from origin
  2. Better Cache Hit Ratio: Consolidates cache across all POPs
  3. Origin Protection: Single point of contact with origin
  4. Cost Optimization: Fewer origin requests = lower bandwidth costs

Enabling Shield

sub vcl_recv {
    # Enable shield for all requests
    set req.backend = F_Shield_Ashburn_VA_US;

    # Or conditionally by domain
    if (req.http.host == "www.prabasiva.com") {
        set req.backend = F_Shield_Ashburn_VA_US;
    }
}

Real-Time Cache Purging

One of Fastly's key features is instant cache purging across all POPs.

Purge Methods

1. URL Purge

# Purge specific URL
curl -X PURGE \
  -H "Fastly-Key: YOUR_API_KEY" \
  https://www.prabasiva.com/blog/post-1
 
# Using Fastly API
curl -X POST \
  -H "Fastly-Key: YOUR_API_KEY" \
  https://api.fastly.com/purge/www.prabasiva.com/blog/post-1

2. Surrogate Key Purge

# Purge by surrogate key (soft purge)
curl -X POST \
  -H "Fastly-Key: YOUR_API_KEY" \
  https://api.fastly.com/service/SERVICE_ID/purge/blog-posts

3. Purge All

# Purge entire service (use with caution!)
curl -X POST \
  -H "Fastly-Key: YOUR_API_KEY" \
  https://api.fastly.com/service/SERVICE_ID/purge_all

Setting Surrogate Keys in VCL

sub vcl_backend_response {
    # Add surrogate keys for purging
    if (bereq.url ~ "^/blog/") {
        set beresp.http.Surrogate-Key = "blog-posts blog-" + bereq.url;
    }

    if (bereq.url ~ "^/api/") {
        set beresp.http.Surrogate-Key = "api-responses";
    }
}

Monitoring and Observability

Key Metrics to Monitor

  1. Cache Hit Ratio: Should be > 80% for static content
  2. Response Time: P95 should be < 100ms for cached content
  3. Error Rate: 5xx errors should be < 0.1%
  4. Origin Load: Should remain low with good cache hit ratio

Best Practices

1. VCL Organization

  • Separate snippets by domain for clarity
  • Use descriptive names for snippets and backends
  • Version control all VCL code in Git
  • Test locally with Varnish before deploying

2. Caching Strategy

  • Static assets: Cache for weeks/months
  • API responses: Short TTL with stale-while-revalidate
  • HTML pages: Medium TTL (5-15 minutes)
  • Personalized content: Use vary headers or don't cache

3. Security

  • Always use HTTPS for origin connections
  • Validate SSL certificates (.ssl_check_cert = always)
  • Implement rate limiting for API domains
  • Add security headers in vcl_deliver
  • Use ACLs for IP-based restrictions

4. Performance

  • Enable Shield to reduce origin load
  • Use surrogate keys for efficient purging
  • Set appropriate TTLs per content type
  • Implement stale-while-revalidate for better UX

5. Deployment

  • Always validate VCL before activating
  • Test on staging service first
  • Use versioning for easy rollback
  • Monitor after deployment for errors

Troubleshooting

Common Issues

1. Low Cache Hit Ratio

# Debug: Log cache status
sub vcl_deliver {
    if (obj.hits > 0) {
        set resp.http.X-Cache-Hits = obj.hits;
    }
    set resp.http.X-Cache-TTL = obj.ttl;
}

Solutions:

  • Check if requests have varying query parameters
  • Ensure proper cache key normalization
  • Verify TTL is set correctly

2. Origin Connection Errors

sub vcl_backend_error {
    # Log backend errors
    set beresp.http.X-Backend-Error = beresp.status;

    # Return cached stale content if available
    if (stale.exists) {
        return(deliver_stale);
    }
}

Solutions:

  • Verify backend health checks
  • Check origin server connectivity
  • Increase timeout values if needed

3. VCL Syntax Errors

# Validate VCL before deployment
fastly service-version validate --version=<VERSION>

Solutions:

  • Test locally with Varnish first
  • Use VCL linter
  • Check Fastly VCL documentation for supported features

Summary

Fastly CDN provides:

  1. Global edge network with 80+ POPs for low latency
  2. VCL customization for domain-specific caching logic
  3. Real-time purging for instant cache updates
  4. Shield layer for origin protection
  5. Instant deployment with version control
  6. Comprehensive monitoring and observability

Domain Management Strategy

  • www.prabasiva.com (opens in a new tab): Long cache for static assets, short for HTML
  • api.prabasiva.com: Short TTL with stale-while-revalidate
  • blog.prabasiva.com: Medium TTL with surrogate key purging

Key Takeaways

  • Each domain can have custom VCL logic within the same service
  • VCL snippets provide modular configuration
  • Shield reduces origin load and improves cache efficiency
  • Real-time purging enables instant content updates
  • Comprehensive monitoring ensures optimal performance

Additional Resources

Last Updated: 2025

Service MeshVarnish
© 2025 Praba Siva. Personal Documentation Site.