Testing fiddles

The Tests option in a fiddle's request configuration allows you to write test assertions against the instrumentation data that is returned by Fiddle when you press RUN. Writing tests is a good way to express what parts of a fiddle are most important when sharing the solution with someone else, or to express a bug by demonstrating the gap between observed behavior and what you expected to see.

Each test should be written on a single line and be in the following format:

[LABEL] TARGET COMPARISON_TYPE REFERENCE_VALUE

For example:

[Cache TTL is positive] events.where(fnName=fetch)[0].attribs.ttl isMoreThan 0

The optional LABEL, which must be wrapped in square brackets if present, allows you to define a 'human readable' expression for the test. TARGET is the thing being tested, REFERENCE_VALUE is a literal value to compare it with, and COMPARISON_TYPE is the comparison you want to make.

You can write more than one test per request.

An example test in Fiddle

Test targets

Within a fiddle test case, TARGET should reference a property of the result data, based on the following schema:

NameTypeDescription
clientFetchObjThe request from the client to Fastly (and the response from Fastly).
├─ .reqStrHTTP request block, containing request method, path, HTTP version, header key/value pairs and request body.
├─ .respStrHTTP response header, contains response status line and response headers (not the body).
├─ .respTypeStrParsed Content-type response header value (mime type only).
├─ .isTextBoolWhether the response body can be treated as text.
├─ .isImageBoolWhether the response body can be treated as an image.
├─ .statusNumHTTP response status.
├─ .bodyPreviewStrUTF-8 text preview of the body (truncated at 1K).
├─ .bodyBytesReceivedNumAmount of data received.
├─ .bodyChunkCountNumNumber of chunks received.
├─ .completeBoolWhether the response is complete.
└─ .trailersStrHTTP response trailers.
originFetchesArrayOrigin fetches made during the request.
└─ [idx]ObjEach fetch is one object.
   ├─ .fetchIDStrUnique ID for this fetch.
   ├─ .traceIDStrID for the compute instance that triggered this fetch.
   ├─ .reqStrHTTP request block, containing request method, path, HTTP version, header key/value pairs and request body.
   ├─ .respStrHTTP response header, contains response status line and response headers (not the body).
   ├─ .remoteAddrStrResolved IP address of origin.
   ├─ .remotePortNumPort on origin server.
   ├─ .remoteHostStrHostname of origin server.
   └─ .elapsedTimeNumTotal time spent on origin fetch (ms).
eventsArrayFastly platform events related to the request, in order of execution.
└─ [idx]ObjEach array element is one event.
   ├─ .typeStrOne of vcl-sub, log, fetch or a Compute event type: ecp-start, ecp-end, geo, dict-open, dict-get
   ├─ .timeNumUnix timestamp in microseconds
   ├─ .popStrThree letter code identifying the Fastly POP location in which this event occurred, e.g., 'LON', 'IAD', 'SYD'.
   ├─ .nodeIDNumNumeric identifier of the individual server on which this event occurred.
   ├─ .reqIDStrIdentifier for this client request. In a fiddle that sends more than one client request, there may be multiple different reqIDs.
   ├─ .traceIDStrID for the execution flow that triggered this event. In the Compute platform, this represents the instance that is currently handling the request. In VCL, it represents one pass through a VCL state flow (i.e., it resets on a restart, ESI, or HTTP/2 server push)
   ├─ .seqIdxNumSequence number of this event with respect to other events with the same traceID and nodeID.
   ├─ .parentStrIf the event is part of a process that was initiated by a different process, this is the traceID of the parent compute process. Normally populated as a result of shielding
   ├─ .isAsyncBool(vcl-sub only) True if the event happened as part of an asynchronous task.
   ├─ .fnNameStr(vcl-sub only) For VCL events, the VCL subroutine that triggered the event. May be 'recv', 'hash', 'hit', 'miss', 'pass', 'fetch', 'deliver', 'error', or 'log'.
   ├─ .logsArray(vcl-sub only) Array of log messages logged from this event.
   │  └─ [idx]ObjEach array element is one log message.
   │     ├─ .contentStrThe content of the log message
   │     └─ .logNameStrThe name of the log destination, e.g., stdout or bigquery
   ├─ .attribsObjEvent-specific properties. These are displayed in the Fiddle UI for the event. For example: within miss events, a staleExists property is reported.
   └─ .prevAttribsObjEvent-specific properties where the value at the start of the subroutine was different to that recorded at the end.
logsArrayArray of strings, messages logged from all events.
insightsArrayInsight tags for this request. Insight tags identify recommendations or divergences from best practices. For example: [ "client-cc-missing", "invalid-header" ].

For example, to target the headers that are sent by Fastly to the backend in an origin request, use the .req property of the first item in the originFetches collection:

originFetches[0].req includes "My-Custom-Header"

In some cases targets are more complex. For example, the TTL set in vcl_fetch is reported as the .ttl property of the vcl-sub event object created by the vcl_fetch subroutine, but it's not possible to know ahead of time the position of that event within the events array. Instead, use a filter function to narrow down events to just the events associated with the vcl_fetch subroutine, and then pick the first one:

events.where(fnName=fetch)[0].attribs.ttl isMoreThan 0

HINT: If there is an .attribs property on an object, Fiddle will treat it as a special case and allow the target expression to directly access the child properties of attribs, to make targeting event properties easier. The example shown above can therefore also be written as:

events.where(fnName=fetch)[0].ttl isMoreThan 0

The where(fnName=fetch) is an example of using an aggregation function to transform the test data and make it easier to target the values that you want to test. The available functions are:

FunctionDetails
listBy(field)
ArrayArray
Takes an array of objects and makes an array of arrays, where in each sub-array, all objects share the same value of field. Items in the output array are ordered based on the order in which the values of the selected field first appear in the input.

e.g. events.listBy(vclflowkey)[1][0].fnName is "recv"
where(field=val)
ArrayArray
Takes an array of objects and filters it to leave only those where the property called field has value val.

e.g. events.where(fnName=recv)[0].url startsWith "/a/b"
groupBy(field)
ArrayObj
Takes an array of objects and splits it into multiple arrays where each one has the same value for the property field, and organizes the resulting data into an object where the field values are keys.

e.g. events.groupBy(fnName).recv[0].url startsWith "/a/b"
transpose()
ArrayObj
Takes an array of objects and makes an object of arrays.  Where multiple input objects share the same property name, that property becomes a top level property with an array containing all the values.

e.g. events.transpose().return notIncludes "error"
count()
ArrayNum
Takes an array and returns the length

e.g. originFetches.count() greaterThan 1
concat()
ArrayStr
Takes an array and returns a string representation of all array items joined together, delimited by newlines.

e.g. logs.concat() includes "Hello"

HINT: All timing properties reported by Fiddle are in microseconds - so 1 hour is reported as 3600000000 (3.6 billion microseconds). The most common timing property to target for testing is .ttl, which is provided on events where fnName is fetch.

Lastly, you need to assert something about this target data, which requires a comparison type and a reference value, together making up an assertion.

Assertions

We support the following options for COMPARISON_TYPE:

NameValue typeDescription
isAnyNon-strict equality (using ==)
isJSONNoneThe target is valid JSON (does not require a reference value, so just two parameters)
isTrueNoneThe target is true
isAtLeast, isAboveJSON numberTarget is numerically higher than (or at least) the reference value
isAtMost, isBelowJSON numberTarget is numerically lower than (or at most) the reference value
includesAnyThe target includes the reference value. Can be used to assert the inclusion of a value in an array, a substring in a string, or a subset of properties in an object.
matchesJS RegExpThe target matches a regular expression. Regex must be delimited with / and may be followed by modifiers, e.g. /abc/i
oneOfJSON arrayChecks that the value of the target is equal to at least one of the values in the reference array
startsWithJSON stringChecks that the value of the target starts with the reference string
endsWithJSON stringChecks that the value of the target ends with the reference string

All comparisons can be negated by prefixing them with not, e.g. notOneOf, isNotJSON, notMatches. The word 'is' will be ignored so can be used purely to improve readability.

The REFERENCE_VALUE must be expressed in a way that can be interpreted as the appropriate type:

  • JSON number: A plain number (3, 5.7, -42)
  • JSON string: A double quoted string ("foo"; contained double-quotes must be escaped as \")
  • JSON boolean: The word true or false
  • JSON array: A JSON-parseable array containing any combination of numbers, strings and booleans as defined above ([1, "foo"])
  • JS RegExp: Either a JavaScript regex literal (/(foo|bar)/ism) or the bare pattern, not quoted or delimited (foo|bar). In the latter case, the s, m and i flags will be applied to the pattern automatically.

Labels

Labelling your tests is optional but helps to illustrate your intent, as well as make test results easier to read. The LABEL portion of the test expression may contain any character other than square brackets ([]), and since test expressions must be one line, labels may also not include newline characters.

[Response is OK] clientFetch.status is 200

Labelled tests in the test results

Example tests

If you want to test...Try this example
HTTP response status on the client fetchclientFetch.status is 200
Request headers on the client fetchclientFetch.req notIncludes "Fastly-FF: "
Response headers on the client fetchclientFetch.resp includes "Age: "
HTTP response status of a backend fetchevents.where(fnName=fetch)[0].status isOneOf [301,302,307,308]
Request headers on a backend fetchoriginFetches[0].req includes "GET /"
Response headers on a backend fetchoriginFetches[0].resp matches /\w+@gmail\.com/
TTL set in a vcl_fetch subroutineevents.where(fnName=fetch)[0].ttl isLessThan 86400000000
URL of the request following a VCL restartevents.where(fnName=recv)[1].url is "/foo"
(After a restart, vcl_recv will run a second time, so the [1] targets the second occurrence
Number of origin fetchesoriginFetches.count() is 2
Response body from Fastly to clientclientFetch.bodyPreview contains "<meta"
(You will only get the first 1K of the response)
Value of fastly.info_state in deliverevents.where(fnName=deliver)[0].state is "HIT"
Number of times deliver is runevents.where(fnName=deliver).count() is 1

Debugging tests

When a test fails, Fiddle will display a representation of the observed, 'actual' value of the target, so you can see why it didn't match. That observed value will often be undefined because the target didn't match anything. In that case, it can be helpful to shorten your target expression, to reveal the shape of the data structure and allow you to see what you can target:

Test failure showing actual value of target

Testing best practices

When constructing tests for your fiddle, consider the following best practices:

  • Label your tests to better illustrate your intent
  • Try to avoid asserting the nature of the implementation of the solution. Instead, assert the achievement of the overall goal. It should be possible to change the way the solution is achieved without having to change the tests.
  • For solutions that act on the user's IP address, for example to block users from certain regions from accessing certain content, use the Client IP override in request options to ensure that the test results are reliably reproducible.
  • Fiddle data arrives asynchronously, but not all instrumentation delays are created equal! Tests that target clientFetch will be fastest. Those that target originFetches will be slightly slower, and events is slowest. Fiddle will keep repeating your tests until they pass or a timeout is reached, to allow asynchronous data to arrive. Speed up your fiddle by using clientFetch and originFetch assertions instead of testing events or logs.