Restricting access to content

There are a number of cases where a business may want to block or limit access to their content — it could be due to an abusive user(s) making excessive requests that impose strain on customers’ origin servers, or it could be blocking due to regulatory/business requirements (e.g. video content that can’t be viewed outside certain geography due to licensing rules). 

In this article, I’ll outline three common methods you can use for achieving this with Fastly. 

They include:

1. Geo-blocking

2. IP address-based blocking

3. Adaptive blocking based on origin responses

Geo-blocking

Fastly exposes a number of geographic variables about users’ IP addresses, like country code, geographical region, city, etc. The full list of variables is available here.

For example, if you wanted to restrict access to users in the United States, you could use this example snippet of Varnish Configuration Language (VCL) code:

if ( client.geo.country_code != "US" ) {
  error 405 "Not allowed";
}

IP address-based blocking

IP address blocking is achieved by using Varnish Access Control List (ACL) objects. ACL objects allow you to specify a list of IP addresses or IP ranges which can be evaluated inside VCL. For example, to block access to 192.168.20.30 and 172.16.0.0/16, you would define the following ACL:

acl banned_ips {
  "192.168.20.30";
  "172.16.0.0"/16;
}

Please note the slightly different IP prefix notation. Once ACL is defined, you can refer to it inside VCL. For example, to block the above IPs, you would add the following statement at the top of your vcl_recv function:

if ( client.ip ~ banned_ips ) {
  error 405 "Not allowed.";
}

Workflow for the requests will look like this:

The drawback of this method is that it’s not very flexible, since any addition to the ACL object requires a new VCL upload.

Adaptive blocking based on origin responses

In many cases, the above two methods may not cut it for more complicated cases. This is especially true for businesses that deal with user-generated content. 

In one particular case, Fastly had a customer whose origins were taken down because a user(s) was generating hundreds of expensive API POST requests per second that caused their database to grind to a halt. 

Initially, they attempted to block users by IP, however they couldn't keep up as the abuser kept changing IPs. To address the issue, they kept a tab of the number of API requests per IP, then blocked the IP once a limit was exceeded. Although that was an improvement, each request still had to make a database call, which kept the databases quite loaded. They reached out to us, and we devised a method to block these types of requests on our edge. Setup is as follows:

1. Watch origin response codes on certain URL paths, e.g. POST /object/create/*.

2. If the origin responds with HTTP code 429, cache the client IP for one day.

3. If IP is blocked return the cached error response.

To achieve the above, we wrote the following VCL code:

sub vcl_recv {
  if (req.request == "POST" && req.url ~ "^/object/create" && req.restarts == 0) {
    return(lookup);
  }

### your code below ###
}

Default behavior for Fastly VCL is not to cache POST requests. However, in this particular case we will do a hash lookup for POST requests that start with /object/create. 

Hash lookup is defined inside the vcl_hash function. By default, Fastly VCL contains the following code:

set req.hash += req.url;
set req.hash += req.http.host;
set req.hash += "#####GENERATION#####";

This means we'll take the requested URL (e.g., /page.html), requested hostname (e.g., www.fastly.com), and generation (this is the Fastly internal variable used for purge all) to build a hash which points to a cached object. To cache user IPs, we'll need to modify the hash as follows:

sub vcl_hash {
  if (req.url ~ "^/object/create") {
    if (req.request == "POST") {
      set req.hash += client.ip;
    } elsif (req.request == "PURGE") {
      set req.hash += req.http.Purge-Banned-IP;
    }
    return(hash);
 }

  ### your code below ###
}

We'll only modify the hash for POST requests starting with /object/create and only use an IP to build the hash. We also need to address the issue of purging, since we are no longer purging URLs and need a way to purge individual IPs. We achieve that by using a custom header known as Purge-Banned-IP. If you were using curl, the call would look something like this:

curl -X PURGE -H “Purge-Banned-IP: 192.168.10.10” https://api.domain.com/object/create

This call would remove 192.168.10.10 from the banned IP list. The last bit we need to do is make sure we cache POST requests that return HTTP code 429. This is achieved through the following code snippet inside vcl_fetch:

sub vcl_fetch {
  if (req.request == "POST" && req.url ~ "^/object/create" && req.restarts == 0) {
    if (beresp.status == 429) {
      set obj.cacheable = true;
      set obj.ttl = 1d;
      return(deliver);
    }

    set beresp.ttl = 0s;
    return(pass);
  }

### your code below ###
}

Workflow for these requests will look like this: