The Dark Side of Request Routing

Some security issues are obvious or relatively easy to uncover using simple patterns, others that are harder to detect even with the use of advanced techniques. And then there are subtle issues that are caused due to a combination of factors and can remain in the code base for years without being discovered.

One such security issue stems from the following factors:

1. Developers can programmatically map HTTP requests
2. Some backend servers automatically map HTTP HEAD requests to GET requests

The next few sections will dive into how these features can be used to bypass security mechanisms, such as CSRF tokens. Finally, we will see which backend web frameworks are potentially vulnerable to this type of attack.

This research was inspired by a GitHub vulnerability that was discovered using such a technique.

Background

HTTP Request Methods

The HTTP protocol has a set of request methods that allow/describe the action to be performed on a resource. In this post we will talk about the following HTTP methods also known as HTTP verbs:

• GET – fetches specific data in the message-body
• HEAD - similar to get GET, but only fetches the HTTP headers (no message-body)
• POST, PUT, DELETE, & PATCH - state changing verbs for altering data

Web APIs and HTTP Requests

The functionality provided by Web APIs can be executed by using a HTTP request method pointed at a URL endpoint along with URL Parameters. Differentiations in the URL, parameters and request method all change which code is executed on the backend. This is to referred to as request mapping.

Automatic mapping of HEAD to GET

An interesting case of request mapping is the HEAD request. A feature was added to backend web frameworks that is often hidden and undiscussed: HEAD requests are automatically mapped to GET requests. This appears to be a win-win scenario for everyone as developers do not have to write the feature but users still have access to it. But there is one downside: a large portion of developers do not know about this transformation that is occurring.

Many developers write logic based upon the HTTP method of a request. After reading about the CSRF bypass vulnerability found on GitHub via a HEAD request, I became curious about the real-world implications of these types of attacks.

“If developers don’t know about HEAD requests being auto-mapped to GET requests, they might write logic that is vulnerable to some type of security bypass"

Eventually I thought if developers don’t know about HEAD requests being auto-mapped to GET requests, they might write logic that is vulnerable to some type of security bypass and if were possible to abuse this obscure logic to bypass security features in other frameworks.

Testing Outline

I ran the following tests to discover how poor request routing could potentially be exploited:

• Test all HTTP verbs being sent to a service. Use following verbs for each test configuration:
  • GET, POST, PUT, PATCH, DELETE, CONNECT, OPTIONS, HEAD, TRACE, ?? (invalid)
• Test a plethora of backend web frameworks. The following frameworks were tested:
  • Django, Flask, Ruby on Rails, Springboot, Laravel, Express and ASP.net

NOTE: Backend web servers tend to have multiple different configurations. For testing, we included non-standard setups that are still available within the framework.

Impact

What does the poor Request Routing actually mean for security? The most common attack vector is bypassing CSRF token checks. Cross-Site Request Forgery (CSRF) is an attack that forces a user to make unintended state changing actions on a website upon loading a separate website. In order to prevent this attack, random tokens are added to state-changing requests; these are called CSRF Tokens. Most web frameworks automatically check for CSRF tokens on all requests that change state (POST, PUT, PATCH, DELETE, etc.). For more information on CSRF, please refer to https://github.com/mdulin2/head_req_server.

If the logic for a POST request (or any state changing method) can be triggered on an endpoint, without actually making a POST request, this would bypass the CSRF token check while making the state changing request. This is a direct compromise in the security of the application, as it defeats the CSRF protection of that API. The following example, in Flask, demonstrates this vulnerability:

@app.route("/unsafe", methods=["GET", "POST"]) 
def log_in_unsafe():
if(request.method == "GET"): 
    print("GET") 
    return "GET" 
  else: 
    print("POST") 
    return "POST" 

In the Flask code shown above, the developer made an assumption that only GET and POST requests would be handled by this routing. However, in Flask, HEAD requests are mapped to GET, which would then call the same code as the GET and POST. In the case where the request method is HEAD, it would not map the logic for the GET test. Hence, it would invoke the logic for the POST request. The main dangerous consequence of this is that HEAD requests do not get checked for CSRF tokens. Because of this, an attacker could use a HEAD request in order to bypass the CSRF token check to trigger the POST request functionality.

Findings

The likelihood of a configuration being exploited can be broken down into two categories: Directly Mapped APIs and Accept All Methods. These will be described below.

Directly Mapped

Several frameworks allow for multiple request methods to map to the same endpoint. In this situation, it is very common for functionality to be chosen based upon the request method.

When the developer can specify which request methods are allowed on an endpoint and this is not exactly followed, it can lead to logic bugs. Flask, Springboot and Ruby on Rails were found to be vulnerable to this type of attack. An example of a vulnerable configuration can be seen in the Flask example shown above. This configuration is the most likely to be vulnerably, as developers assume that the request checks are safe.

Accept All Methods

Several frameworks have an endpoint that will accept all HTTP methods. When developers create logic, based upon these request methods, mistakes can be made that lead to logic bugs. A developer could write code similar to the following:

def index(request): 
  if(request.method == 'GET'): 
    print("do GET stuff") 

  # State changing request 
  else: 
    print("Do other stuff") 
  return HttpResponse(str(request.method))

In the above Django example, the developer is assuming that only a GET request will hit this endpoint and bypass the CSRF check. The developer is assuming that all other requests would then need to be verified by CSRF tokens. By making a non-GET request that bypasses the CSRF check, this can trigger an unintended code path.

What is considered a safe request? A safe request is any request method that is guaranteed to not change state. According to RFC 7231, GET, HEAD, OPTIONS and TRACE are safe requests that do not require CSRF tokens validation. The results section has a table showing the results of each request method for each framework and how it handles CSRF token checks. It should be noted that this is assuming that the developer is using the proper verbs for all actions.

It should be noted that this situation does not rely on HEAD requests being mapped to GET requests. However, it does rely on the fact that HEAD requests do not get checked for CSRF tokens.

Results

Framework Findings

The table below goes into the findings, and is organized in the following way: the second column (directly mapped) describes a framework where each request method must be specified. The third column (Accept All Endpoints) describes a framework that has the option to accept all HTTP methods on a single endpoint. The fourth column (Map Head to Get) describes a framework that maps HTTP HEAD requests into HTTP GET requests. The following table shows the findings from the research, with the checkmarks demonstrating a vulnerable configuration:

The ASP.net framework was the only framework found not to be vulnerable to this issue at all, while Ruby on Rails was found to be the only framework vulnerable to this attack in multiple configurations.

It should be noted that this table is not exhaustive, several of the frameworks support multiple ways to do routing. Developers can also overwrite the default configurations to make the framework perform other actions. This table just shows a yes/no result of whether the frameworks have this functionality, in any capacity. For more into this, please visit the in-depth notes at https://github.com/mdulin2/head_req_server.

CSRF Token Checks per Request

The following table demonstrates which request methods do not validate CSRF tokens for each framework:

Most of the frameworks were as strict or stricter than the specification. However, Flask was the lone framework that allow for the CONNECT verb and invalid verbs (?) to bypass CSRF token checks, if explicitly stated.

Conclusion

Writing logic based upon the request method can be difficult, particularly when the framework adds an unexpected request mapping.

Subtle bugs can be in code bases for years without being discovered. This article had two main insights: developers write code to manually handle requests based upon the request method and some frameworks automatically map HEAD request to GET requests. Using the first observation incorrectly or the two features together can lead to vulnerabilities in the routing mechanisms of a website.

References

• https://blog.teddykatz.com/2019/11/05/github-oauth-bypass.html
• https://owasp.org/www-community/attacks/csrf