Custom Fastly VCL snippets

Fastly and Magento Commerce (Cloud) support creating custom Varnish Configuration Language (VCL) snippets. For best results, we recommend creating Edge Dictionaries and Edge ACLs for your VCL snippets. You’re free to customize your Fastly VCL snippets any way you like to complete custom code. The following examples and instructions walk through creating edge dictionaries, edge ACLs, and VCL snippets.

To create and upload these VCL snippets, you use a terminal application. You do not need to SSH into a specific environment. This information includes a walk-through creating regular snippets a bash command and .vcl snippet files of JSON code. Don’t worry, we walk you through the process with examples.

You need the following information to create VCL snippets:

  • Fastly Service ID for Staging and Production to assign the snippets to a specific service or environment
  • Fastly API key used for the FASTLY_API_TOKEN in the commands

To get started, review the following sections:

Understand VCL snippet values

Your VCL snippets include the following values. You can use these key/value pairs in JSON snippets in .vcl files and in cURL commands.

Value Description
service_id The ID indicates the specific Staging or Production service/environment. We provide this value. You will use this value when setting up your bash script for custom VCL snippets for SERVICE_ID.
API_KEY The API Key for your Fastly account. We provide this value. You will add this value to your bash script.
version The version of the service you add snippets to for validating and activating. Fastly uses Editable Version # in their example values. In the bash script, we use SERVICE_ID_VERSION for this value.
type Specifies the location to place the generated snippet such as init (above subroutines) and within subroutines like recv. See Fastly VCL snippet object values for information on these values.
content The snippet of VCL code to run. We recommend keeing this code all in one line. The VCL snippet code, cURL commands, and bash script require the content code in one line.
priority

Determines the order VCL snippets call. Lower values run first, from 1 to 100. All Magento module uploaded snippets are 50. If you want an action to occur last or override Magento default VCL snippets, enter a higher number like 100. To have code occur immediately, enter a lower value like 5.

Any VCL snippet at priority 5 will run immediately, best for blacklists, whitelists, and redirects. Priority 100 is best for overriding default VCL snippet code and values, best for extending timeouts. If you do not set a priority with your cURL command, the default value set is 100.

dynamic Indicates if this is a dynamic snippet or regular snippet.
active Indicates if the snippet or version is activated and currently in use. Returns true or false. Make note of the version number for an active snippet. You will use this to clone the version.

The following is an example of a returned JSON for a customer VCL snippet:

{
  "id": "62Yd1WfiCBPENLloXfXmlO",
  "service_id": "{Service ID}",
  "version": "{Editable Version #}",
  "name": "apply_acl",
  "priority": "100",
  "dynamic": "1",
  "type": "hit",
  "content": "if ((client.ip ~ {ACLNAME}) && !req.http.Fastly-FF){ error 403; }",
  "created_at": "2016-08-15T09:37:10+00:00",
  "updated_at": "2016-08-15T09:37:10+00:00",
  "deleted_at": null
}

Prepare to create VCL snippets

Before creating custom VCL snippets, you need to set up and save two bash script files, one for each Staging and Production. Each file will include the specific Fastly Service ID per environment. When creating VCL snippets, you will create a series of VCL snippet files (.vcl) with JSON code. The bash scripts run through all .vcl files, creating and running CURL commands, to push custom VCL snippets to Staging or Production for a cloned version.

Anytime you want to add or edit existing VCLs, you will need to edit the files and change the version number to match the new cloned version.

To create the bash script files:

  1. Copy the following bash script.

       #!/bin/bash
       #############################################################################
       # Upload snippets from the current directory
       #
       # Requires you to specify following environment variables
       #  VERSION - The new cloned version of the VCL
       #  SERVICE_ID - Service ID for Staging or Production
       #  API_KEY - Fastly API Key for your account
       #############################################################################
       if [ "x$VERSION" == "x" -o  "x$SERVICE_ID" == "x" -o "x$API_KEY" == "x" -o "xSNIPPET_NAME_PREFIX" == "x" ]; then
       cat <<ENDOF
       This script upload snippets from the local directory to their appropriate snippet type e.g. recv.vcl
       to recv type, fetch.vcl to fetch type etc.
       You need to define these variables
            export VERSION=<SERVICE_ID_VERSION>
            export SERVICE_ID=<SERVICE_ID>
            export API_KEY=<API_KEY>
            export SNIPPET_NAME_PREFIX=<SNIPPET_NAME_PREFIX_NO_SPACES>
       ENDOF
       exit 1
       fi
       rawurlencode() {
       while IFS="" read -r string; do
           local strlen=${#string}
           local encoded=""
           local pos c o
           for (( pos=0 ; pos<strlen ; pos++ )); do
                c=${string:$pos:1}
                case "$c" in
                       [-_.~a-zA-Z0-9] ) o="${c}" ;;
                       * )               printf -v o '%%%02x' "'$c"
                esac
                encoded+="${o}"
           done
           echo -n "${encoded}"
           echo -n %0A
       done
       }
       # Find all VCLs in this directory
       for vcl in *.vcl
       do
       TYPE=${vcl%.vcl}
       curl -X POST -s https://app.fastly.com/service/${SERVICE_ID}/version/${VERSION}/snippet \
       -H 'Content-Type: application/json'
       -H "Fastly-Key:$API_KEY" \
       --data "name=${SNIPPET_NAME_PREFIX}_${TYPE}&type=$TYPE&dynamic=0&content=$(cat $vcl | rawurlencode)";
       done
    
  2. Save two versions of the file: staging_snippets.sh and production_snippets.sh.
  3. In staging_snippets.sh, edit the following values and save:

    • SERVICE_ID: Replace <SERVICE_ID> with the Service ID for your Staging environment.
    • API_KEY: Replace <API_KEY> with your Fastly API Key.
    • SNIPPET_NAME_PREFIX: Replace <SNIPPET_NAME_PREFIX_NO_SPACES> with a value of your choice. Do not use magentocloud as the prefix.
  4. In production_snippets.sh, edit the following values and save:

    • SERVICE_ID: Replace <SERVICE_ID> with the Service ID for your Staging environment.
    • API_KEY: Replace <API_KEY> with your Fastly API Key.
    • SNIPPET_NAME_PREFIX: Replace <SNIPPET_NAME_PREFIX_NO_SPACES> with a value of your choice. Do not use magentocloud as the prefix.
  5. Optional, save the files to their own directories.

For example if you want different custom VCL snippets for Staging then Production, you can save the staging_snippets.sh bash script and Staging .vcl files into the same directory. When you run that bash script, only those custom VCLs generate to Staging.

When modifying the bash scripts, make sure to carefully enter the correct Service ID. Fastly assigns different Service IDs per Staging and Production environments.

The custom VCL snippet process

To create custom VCLs, you should have the bash script prepared and saved to a directory. You can then continue with the following:

  1. Locate the active version for VCL snippets. You will use this version to clone.
  2. Clone the active VCL snippet version to generate a new version. All changes will save to this new version. Save this version number!
  3. Modify or add VCL snippets in .vcl files. We provide a number of examples to get you started.
  4. Run the bash scripts to add and update VCL snippets to Fastly. The bash script runs through all of the .vcl files, generating and running the CURL commands for you.
  5. Validate and activate the version number and all associated VCL snippets.

The following are best practices and recommendations:

  • The default VCL snippets you uploaded included a prepended name of magentomodule_ with a priority of 50. For your custom VCL snippets, do not use the magentomodule_ name. Also consider the priority of your custom snippets if they should override the default snippets.
  • This process allows you to create and update multiple VCL snippets by running the bash script. It generates and runs the cURL command for you using all VCL JSON snippet files (.vcl) in the directory.
  • If you want different values and settings or different VCLs per Staging and Production, create two directories: one for each environment. Save the specific bash script and VCL files to each one. Then move to those directories when done and run those scripts during step 4. If you keep everything in the same directory and run both bash scripts, the VCLs are added to both Staging and Production.
  • If you decide to delete a VCL, save the VCL file to another archive folder. If you want to add it back, you can move the file back into the folder with the bash script and perform the steps again for updating your custom snippets.
  • Don’t forget to always locate the active version, clone, and edit the bash script with the new version! Version is not part of your VCL snippet files.
  • If you want to override values and settings from the default Fastly VCL snippets, we recommend creating a new snippet with updated values and code with a higher priority value of 100. You should not try overwriting default VCLs. We provide an example for Custom extend Admin timeout VCL.

Locate the currently active VCL snippet version

To view an entire list of all VCL snippets by version, use the following command:

curl -X GET -s https://api.fastly.com/service/<Service ID>/version -H "Fastly-Key:FASTLY_API_TOKEN"

Look for the active key from the returned list. This is the version you will clone in the next section.

For more information on this Fastly API, see this get version command.

Clone the active VCL version and all snippets

Clone the version using the active version number. This creates a copy of all existing VCL snippets for that version with a new version number. After you clone the version, you can modify and add VCL snippets. Save the new version number as you will need it for the bash script.

curl -H "Fastly-Key: {FASTLY_API_TOKEN}" -H 'Content-Type: application/json' -H "Accept: application/json" -X PUT https://api.fastly.com/service/{Service ID}/version/{Current Active Version #}/clone

For more information on this Fastly API, see this clone command.

Create custom VCL snippets

We recommend the following process for creating and modifying snippets. The process allows you to push code for one or more snippets, save the code used, and formats the CURL commands just by running the bash scripts you prepared.

Create a VCL snippet as a .vcl file with the following JSON content and format:

{
  "name": "<name>",
  "priority": "100",
  "type": "<type>",
  "content": "<code all in one line>",
}

The values include:

  • name: Name for the VCL snippet
  • priority: Determines the order VCL snippets call. Lower values run first, from 1 to 100. All Magento module uploaded snippets are 50. If you want an action to occur last or override Magento default VCL snippets, enter a higher number like 100. To have code occur immediately, enter a lower value like 5.
  • type: Specifies the location to place the generated snippet such as init (above subroutines) and within subroutines like recv. See Fastly VCL snippet object values for information on these values.
  • content: The snippet of VCL code to run in one line, without line breaks.

For detailed examples and custom code, see the following:

Update and run the bash scripts

To add the VCL snippets to Fastly, you need to modify the bash scripts with the cloned version number and run the scripts.

Before running the bash scripts, make sure the .vcl files in the directory should be added to that environment. If you plan on having all of the same VCL settings for Staging and Production, you can keep all files in the same directory.

If you want different VCLs for Staging and Production, do the following:

  • Create a Staging directory with staging_snippets.sh and those .vcl files
  • Create a Production directory with production_snippets.sh and those .vcl files

To add custom snippets:

  1. Edit the bash scripts staging_snippets.sh and production_snippets.sh.
  2. Change the value for <SERVICE_ID_VERSION> with the cloned version number in the line: export VERSION=<SERVICE_ID_VERSION>
  3. Save the files.
  4. Run the bash script by entering the file name. The command generates and runs a CURL command for every .vcl file in the directory.
  • Enter production_snippets to run that file and generate VCL snippets to Fastly for your Production environment.
  • Enter staging_snippets to run that file and generate VCL snippets to Fastly for your Staging environment.
    1. As the script runs, VCL snippets should be generating and uploading to Fastly.

Validate and activate snippets for a version

When you add the VCL snippet(s) to the version, Fastly creates and assigns it to your service per that version number. Next, you should validate the entered VCL snippets with Fastly. Use the following command to validate all snippets for the version:

curl -H "Fastly-Key: {FASTLY_API_TOKEN}" -H 'Content-Type: application/json' -H "Accept: application/json" -X GET https://api.fastly.com/service/{Service ID}/version/{Editable Version #}/validate

Fastly should return: "status": "ok". If you received an OK, activate the version for that service.

Assuming no errors (if there are errors, fix them before proceeding), the last step is to activate the version with the following command. All VCL snippets associated with the version activate and the previous version snippets deactivate.

curl -H "Fastly-Key: {FASTLY_API_TOKEN}" -H 'Content-Type: application/json' -H "Accept: application/json" -X PUT https://api.fastly.com/service/{Service ID}/version/{Editable Version #}/activate

If your received errors back from Fastly, track down the errors and update the specific snippet with the following command. Make sure to use the same version number you are working to activate.

curl -X PUT -s https://api.fastly.com/service/<Service ID>/version/<Editable Version #>/snippet/<Snippet Name e.g my_regular_snippet> -H "Fastly-Key:FASTLY_API_TOKEN" -H 'Content-Type: application/x-www-form-urlencoded' --data $'content=if ( req.url ) {\n set req.http.my-snippet-test-header = \"affirmative\";\n}';

For more information on these Fastly APIs, see validate and activate commands.

Manage regular VCL snippets with curl

To review an individual snippet, enter the following API call in a terminal:

curl -X GET -s https://api.fastly.com/service/<Service ID>/version/<Editable Version #>/snippet/<Snippet Name e.g my_regular_snippet> -H "Fastly-Key:FASTLY_API_TOKEN"

To list all regular VCL snippets attached to a service, enter the following API call in a terminal:

curl -X GET -s https://api.fastly.com/service/<Service ID>/version -H "Fastly-Key:FASTLY_API_TOKEN"

If you want to override values and settings from the default Fastly VCL snippets, we recommend creating a new snippet with updated values and code with a priority of 100 (overrides the defaults).

To delete an individual VCL snippet using the API, get a list of snippets and enter a curl command with the speicific snippet information to delete. We recommend keeping a copy of the .vcl file in an archive directory if you need to recreate it later.

curl -X DELETE -s https://api.fastly.com/service/<Service ID>/version/<Editable Version #>/snippet/<Snippet Name e.g my_regular_snippet> -H "Fastly-Key:FASTLY_API_TOKEN"

Fastly resources

For Fastly resources on creating VCL snippets, you can review their docs:

Fastly supports two types of snippets. We recommend and document how to create and use regular snippets.

  • Regular snippets are versioned VCL snippets. The code and settings are locked per version to create, modify, and deploy with the Fastly service.
  • Dynamic snippets are snippets you can only create via API calls. These snippets do not have a version and deploy separately from your Fastly service.