November 29, 2021
On November 11th 2021, Fastly Engineering received alerts related to segmentation faults on Compute@Edge. A Fastly investigation into CVE-2021-43790, a bug in Lucet, a dependency of Compute@Edge, is disclosed in a recent Bytecode Alliance security advisory. Fastly investigations have not identified additional impact outside of the single case disclosed in this advisory.
It's our goal in this Fastly Security Advisory to illustrate our knowledge about the bug discovered and the actions we have taken to prevent further possible impact to our customers.
May 21, 2021
The bug identified in the Cranelift x64 backend performs a sign-extend instead of a zero-extend on a value loaded from the stack, when the register allocator reloads a spilled integer value narrower than 64 bits. This interacts poorly with another optimization: the instruction selector elides a 32-to-64-bit zero-extend operator when we know that an instruction producing a 32-bit value actually zeros the upper 32 bits of its destination register. Hence, the x64 compiler relies on these zeroed bits, but the type of the value is still i32, and the spill/reload reconstitutes those bits as the sign extension of the i32’s MSB.
September 3, 2020
On July 29th at 00:00 UTC, Fastly was notified by a customer (customer X) that a single log line intended for a different customer (customer Y) was received by customer X’s log system. Fastly promptly began to investigate and determined that when a complex series of conditions occur, a log line may be misrouted to an incorrect logging service. We were able to trace the root cause to an error in logic introduced by Fastly to improve performance in April 2012. This single report from one customer is the only instance that Fastly is aware of, where all necessary conditions aligned simultaneously in eight years.
August 5, 2020
Fastly was notified of the issue on May 21, 2020 13:30 UTC. Fastly immediately launched an investigation, identifying which origin servers responded with a test port number in the redirect response, in order to understand the vulnerability and possible solutions. After the investigation, Fastly first notified potentially affected customers on July 15, 2020 at 04:30 UTC.
The vulnerability is a variant of a previously reported vulnerability, and ultimately the result of constructing cacheable origin responses based on user-defined data. The issue occurs when an attacker issues an HTTPS request and specifies within the Host header a port number that is not actually being used for any services. It is possible to cache a resource in such a way as to deny future requests from being serviced properly.
November 14, 2019
On November 11, 2019, at 21:57 UTC, Fastly deployed a new build of its HTTP/2 termination software to two Fastly cache servers in the Minneapolis-St.Paul (STP) data center. This build contained a processing flaw involving connection re-use between internal Fastly systems (unrelated to HTTP/2 multiplexing), and caused some incoming HTTP/2 requests for Fastly customers’ services to potentially be routed incorrectly to a group of up to 20 different Fastly customers’ services and origins. This led to some client request data being delivered to, and a response returned by, an incorrect customer origin. The customers whose origins erroneously received these requests may have logged the incorrectly-routed request data.
Fastly was first notified by a customer of a client error on November 12, 2019, at 23:07 UTC. On November 13, 2019, at 00:50 UTC, all customer traffic was diverted away from the affected data center. Fastly immediately commenced an investigation, and on November 14, 2019, at 00:31 UTC, we validated the presence of incorrectly routed request data in a customer’s logs.
We estimate this flaw affected 0.00016% of our global request traffic during the 27-hour period. It is unlikely that affected client requests came from outside of North America.
Because Fastly does not store customer log data, we are not able to say with certainty if an affected request was incorrectly routed.
August 13, 2018
On Thursday, August 9th, research was published at Black Hat USA 2018 on cache poisoning attacks against websites deployed behind caching infrastructure. These attacks could allow an attacker to inject arbitrary content into a victim’s cache.
Fastly service configurations that do not take into consideration the interaction between headers that backends use to select content may be vulnerable. This risk can be fully mitigated via a VCL patch or by modifying backend configurations.
August 6, 2018
On August 6, 2018, a vulnerability in the Linux kernel TCP implementation, called SegmentSmack, was publicly disclosed. This vulnerability allowed a remote attacker to cause a denial-of-service attack on a target server by simply establishing a TCP connection to the server and sending specific segments over the connection.
Fastly has worked with the security community in advance of this disclosure to address this vulnerability in our edge networks. They pose no threat to Fastly customers.
January 9, 2018
On Wednesday, January 3rd, research was published on a class of security vulnerabilities affecting specific processors. These vulnerabilities could allow a user who can execute code on a system to gain unauthorized access to information across security boundaries.
Fastly has completed initial analysis of these vulnerabilities and does not believe they pose an immediate threat to Fastly customers.
January 8, 2018
From August 31st through November 4th, Fastly deployed a version of Varnish which contained a security bug that, in a limited and non-standard set of configurations, disclosed request bodies to other customer origins. In these cases, a request body sent to an affected Fastly customer's service would have been included in a malformed request to a different customer's origin, which may have been logged in that origin web server's access logs.
Fastly performed a comprehensive assessment to identify customers most likely to be affected by this issue. These customers have been contacted directly by Fastly Customer Engineering.
September 12, 2017
During the investigation of a customer report, Fastly became aware of and addressed a security vulnerability (CVE-2017-13761) in the Fastly CDN module intended to be integrated into Magento2. This is open source code which Fastly releases to enable easy integration with our partner’s products. All versions prior to 1.2.26 are affected and customers are encouraged to upgrade.
Fastly has reached out directly to customers currently using affected versions of the module.
November 16, 2016
On Monday, November 14, 2016, security researchers published a paper “Measuring the Security Harm of TLS Crypto Shortcuts.” Among other findings across the TLS implementation of several sites, the paper identified Fastly as not frequently rotating TLS session tickets, limiting the effectiveness of forward secrecy.
While Fastly was not directly contacted by the researchers, Fastly had previously been made aware of the issue, and this vulnerability was addressed on Friday, November 11. No customer action is required to benefit from the fix.
October 21, 2016
On October 21st, 2016, Dyn, a major managed DNS provider, experienced a Distributed Denial of Service attack, which led to outages affecting several major websites, including Fastly infrastructure (such as the Fastly Control Panel and API) and Fastly customers.
Fastly worked with our additional managed DNS providers to ensure availability during the incident. This mitigated impact on Fastly customers.