Testing fiddles
The Tests option in a fiddle's request configuration allows you to write test assertions against the instrumentation data that is returned by Fiddle when you press RUN. Writing tests is a good way to express what parts of a fiddle are most important when sharing the solution with someone else, or to express a bug by demonstrating the gap between observed behavior and what you expected to see.
Each test should be written on a single line and be in the following format:
[LABEL] TARGET COMPARISON_TYPE REFERENCE_VALUE
For example:
[Cache TTL is positive] events.where(fnName=fetch)[0].attribs.ttl isMoreThan 0
The optional LABEL
, which must be wrapped in square brackets if present, allows you to define a 'human readable' expression for the test. TARGET
is the thing being tested, REFERENCE_VALUE
is a literal value to compare it with, and COMPARISON_TYPE
is the comparison you want to make.
You can write more than one test per request.
Test targets
Within a fiddle test case, TARGET
should reference a property of the result data, based on the following schema:
Name | Type | Description |
---|---|---|
clientFetch | Obj | The request from the client to Fastly (and the response from Fastly). |
├─ .req | Str | HTTP request block, containing request method, path, HTTP version, header key/value pairs and request body. |
├─ .resp | Str | HTTP response header, contains response status line and response headers (not the body). |
├─ .respType | Str | Parsed Content-type response header value (mime type only). |
├─ .isText | Bool | Whether the response body can be treated as text. |
├─ .isImage | Bool | Whether the response body can be treated as an image. |
├─ .status | Num | HTTP response status. |
├─ .bodyPreview | Str | UTF-8 text preview of the body (truncated at 1K). |
├─ .bodyBytesReceived | Num | Amount of data received. |
├─ .bodyChunkCount | Num | Number of chunks received. |
├─ .complete | Bool | Whether the response is complete. |
└─ .trailers | Str | HTTP response trailers. |
originFetches | Array | Origin fetches made during the request. |
└─ [idx] | Obj | Each fetch is one object. |
├─ .fetchID | Str | Unique ID for this fetch. |
├─ .traceID | Str | ID for the compute instance that triggered this fetch. |
├─ .req | Str | HTTP request block, containing request method, path, HTTP version, header key/value pairs and request body. |
├─ .resp | Str | HTTP response header, contains response status line and response headers (not the body). |
├─ .remoteAddr | Str | Resolved IP address of origin. |
├─ .remotePort | Num | Port on origin server. |
├─ .remoteHost | Str | Hostname of origin server. |
└─ .elapsedTime | Num | Total time spent on origin fetch (ms). |
events | Array | Fastly platform events related to the request, in order of execution. |
└─ [idx] | Obj | Each array element is one event. |
├─ .type | Str | One of vcl-sub , log , fetch or a Compute event type: ecp-start , ecp-end , geo , dict-open , dict-get |
├─ .time | Num | Unix timestamp in microseconds |
├─ .pop | Str | Three letter code identifying the Fastly POP location in which this event occurred, e.g., 'LON', 'IAD', 'SYD'. |
├─ .nodeID | Num | Numeric identifier of the individual server on which this event occurred. |
├─ .reqID | Str | Identifier for this client request. In a fiddle that sends more than one client request, there may be multiple different reqID s. |
├─ .traceID | Str | ID for the execution flow that triggered this event. In the Compute platform, this represents the instance that is currently handling the request. In VCL, it represents one pass through a VCL state flow (i.e., it resets on a restart , ESI, or HTTP/2 server push) |
├─ .seqIdx | Num | Sequence number of this event with respect to other events with the same traceID and nodeID . |
├─ .parent | Str | If the event is part of a process that was initiated by a different process, this is the traceID of the parent compute process. Normally populated as a result of shielding |
├─ .isAsync | Bool | (vcl-sub only) True if the event happened as part of an asynchronous task. |
├─ .fnName | Str | (vcl-sub only) For VCL events, the VCL subroutine that triggered the event. May be 'recv', 'hash', 'hit', 'miss', 'pass', 'fetch', 'deliver', 'error', or 'log'. |
├─ .logs | Array | (vcl-sub only) Array of log messages logged from this event. |
│ └─ [idx] | Obj | Each array element is one log message. |
│ ├─ .content | Str | The content of the log message |
│ └─ .logName | Str | The name of the log destination, e.g., stdout or bigquery |
├─ .attribs | Obj | Event-specific properties. These are displayed in the Fiddle UI for the event. For example: within miss events, a staleExists property is reported. |
└─ .prevAttribs | Obj | Event-specific properties where the value at the start of the subroutine was different to that recorded at the end. |
logs | Array | Array of strings, messages logged from all events. |
insights | Array | Insight tags for this request. Insight tags identify recommendations or divergences from best practices. For example: [ "client-cc-missing", "invalid-header" ] . |
For example, to target the headers that are sent by Fastly to the backend in an origin request, use the .req
property of the first item in the originFetches
collection:
originFetches[0].req includes "My-Custom-Header"
In some cases targets are more complex. For example, the TTL set in vcl_fetch
is reported as the .ttl
property of the vcl-sub
event object created by the vcl_fetch
subroutine, but it's not possible to know ahead of time the position of that event within the events
array. Instead, use a filter function to narrow down events
to just the events associated with the vcl_fetch
subroutine, and then pick the first one:
events.where(fnName=fetch)[0].attribs.ttl isMoreThan 0
HINT: If there is an .attribs
property on an object, Fiddle will treat it as a special case and allow the target expression to directly access the child properties of attribs
, to make targeting event properties easier. The example shown above can therefore also be written as:
events.where(fnName=fetch)[0].ttl isMoreThan 0
The where(fnName=fetch)
is an example of using an aggregation function to transform the test data and make it easier to target the values that you want to test. The available functions are:
Function | Details |
---|---|
listBy(field) Array ➔ Array | Takes an array of objects and makes an array of arrays, where in each sub-array, all objects share the same value of field . Items in the output array are ordered based on the order in which the values of the selected field first appear in the input.e.g. events.listBy(vclflowkey)[1][0].fnName is "recv" |
where(field=val) Array ➔ Array | Takes an array of objects and filters it to leave only those where the property called field has value val .e.g. events.where(fnName=recv)[0].url startsWith "/a/b" |
groupBy(field) Array ➔ Obj | Takes an array of objects and splits it into multiple arrays where each one has the same value for the property field , and organizes the resulting data into an object where the field values are keys.e.g. events.groupBy(fnName).recv[0].url startsWith "/a/b" |
transpose() Array ➔ Obj | Takes an array of objects and makes an object of arrays. Where multiple input objects share the same property name, that property becomes a top level property with an array containing all the values. e.g. events.transpose().return notIncludes "error" |
count() Array ➔ Num | Takes an array and returns the length e.g. originFetches.count() greaterThan 1 |
concat() Array ➔ Str | Takes an array and returns a string representation of all array items joined together, delimited by newlines. e.g. logs.concat() includes "Hello" |
HINT: All timing properties reported by Fiddle are in microseconds - so 1 hour is reported as 3600000000
(3.6 billion microseconds). The most common timing property to target for testing is .ttl
, which is provided on events where fnName
is fetch
.
Lastly, you need to assert something about this target data, which requires a comparison type and a reference value, together making up an assertion.
Assertions
We support the following options for COMPARISON_TYPE
:
Name | Value type | Description |
---|---|---|
is | Any | Non-strict equality (using == ) |
isJSON | None | The target is valid JSON (does not require a reference value, so just two parameters) |
isTrue | None | The target is true |
isAtLeast , isAbove | JSON number | Target is numerically higher than (or at least) the reference value |
isAtMost , isBelow | JSON number | Target is numerically lower than (or at most) the reference value |
includes | Any | The target includes the reference value. Can be used to assert the inclusion of a value in an array, a substring in a string, or a subset of properties in an object. |
matches | JS RegExp | The target matches a regular expression. Regex must be delimited with / and may be followed by modifiers, e.g. /abc/i |
oneOf | JSON array | Checks that the value of the target is equal to at least one of the values in the reference array |
startsWith | JSON string | Checks that the value of the target starts with the reference string |
endsWith | JSON string | Checks that the value of the target ends with the reference string |
All comparisons can be negated by prefixing them with not
, e.g. notOneOf
, isNotJSON
, notMatches
. The word 'is' will be ignored so can be used purely to improve readability.
The REFERENCE_VALUE
must be expressed in a way that can be interpreted as the appropriate type:
- JSON number: A plain number (
3
,5.7
,-42
) - JSON string: A double quoted string (
"foo"
; contained double-quotes must be escaped as\"
) - JSON boolean: The word
true
orfalse
- JSON array: A JSON-parseable array containing any combination of numbers, strings and booleans as defined above (
[1, "foo"]
) - JS RegExp: Either a JavaScript regex literal (
/(foo|bar)/ism
) or the bare pattern, not quoted or delimited (foo|bar
). In the latter case, thes
,m
andi
flags will be applied to the pattern automatically.
Labels
Labelling your tests is optional but helps to illustrate your intent, as well as make test results easier to read. The LABEL
portion of the test expression may contain any character other than square brackets ([]
), and since test expressions must be one line, labels may also not include newline characters.
[Response is OK] clientFetch.status is 200
Example tests
If you want to test... | Try this example |
---|---|
HTTP response status on the client fetch | clientFetch.status is 200 |
Request headers on the client fetch | clientFetch.req notIncludes "Fastly-FF: " |
Response headers on the client fetch | clientFetch.resp includes "Age: " |
HTTP response status of a backend fetch | events.where(fnName=fetch)[0].status isOneOf [301,302,307,308] |
Request headers on a backend fetch | originFetches[0].req includes "GET /" |
Response headers on a backend fetch | originFetches[0].resp matches /\w+@gmail\.com/ |
TTL set in a vcl_fetch subroutine | events.where(fnName=fetch)[0].ttl isLessThan 86400000000 |
URL of the request following a VCL restart | events.where(fnName=recv)[1].url is "/foo" (After a restart , vcl_recv will run a second time, so the [1] targets the second occurrence |
Number of origin fetches | originFetches.count() is 2 |
Response body from Fastly to client | clientFetch.bodyPreview contains "<meta" (You will only get the first 1K of the response) |
Value of fastly.info_state in deliver | events.where(fnName=deliver)[0].state is "HIT" |
Number of times deliver is run | events.where(fnName=deliver).count() is 1 |
Debugging tests
When a test fails, Fiddle will display a representation of the observed, 'actual' value of the target, so you can see why it didn't match. That observed value will often be undefined
because the target didn't match anything. In that case, it can be helpful to shorten your target expression, to reveal the shape of the data structure and allow you to see what you can target:
Testing best practices
When constructing tests for your fiddle, consider the following best practices:
- Label your tests to better illustrate your intent
- Try to avoid asserting the nature of the implementation of the solution. Instead, assert the achievement of the overall goal. It should be possible to change the way the solution is achieved without having to change the tests.
- For solutions that act on the user's IP address, for example to block users from certain regions from accessing certain content, use the Client IP override in request options to ensure that the test results are reliably reproducible.
- Fiddle data arrives asynchronously, but not all instrumentation delays are created equal! Tests that target
clientFetch
will be fastest. Those that targetoriginFetches
will be slightly slower, andevents
is slowest. Fiddle will keep repeating your tests until they pass or a timeout is reached, to allow asynchronous data to arrive. Speed up your fiddle by usingclientFetch
andoriginFetch
assertions instead of testingevents
orlogs
.