HTTP 404 for Missing API Resources

Should an API return HTTP 404 status when the specified resource cannot be found? Of course, that’s exactly what (Not Found) means. As RFC2616 states pretty clearly, 404 Not Found means:

The server has not found anything matching the Request-URI.

However, if you think APIs are like web pages, you might be perplexed by such an interpretation. Maybe 404 feels like an error because it connotes that something went wrong with the web app. Oh no, the page wasn’t found! The world is coming to an end! Run for the hills! Save the women and children!

But APIs aren’t web apps, are they? They aren’t pages at all. APIs are cacheable, uniformly-addressable, resource-oriented interfaces. Given the address for a resource, if it’s not available, the API should return the HTTP 404 status in most cases.

Unfortunately, there’s so much momentum around the idea of 404 being an error condition that it’s really hard to get developers to think differently about how the web is supposed to work. Monitoring tools, log indexers and server health checks don’t help much for changing minds. So many of these tools dutifully treat 404 as an error condition that well-meaning API developers can’t see outside of the little boxes into which they’ve been coaxed.

One of the most troubling manifestations of this page-oriented bias is Microsoft Azure’s App Service server health check. Deploying a pure Web API to an Azure App Service, you’ll find that you’re subject to all sorts of page-specific constructs related to health. Most notably, if you return HTTP 404 from a server too often, the health checks will mark the instance as failing. Obviously, if pages are missing, something’s clearly wrong with that one naughty instance out of ten. The sheer narrow-mindedness of that idea just boggles my mind given that all the servers run the same code and connect to the same resources by definition.

If this incorrect behavior meant that Azure simply recycled instances more often than it needed to, that wouldn’t be too high a price to pay for their pedantic interpretation of the HTTP specification. Unfortunately, we don’t stop paying the price there. The standard load balancer for Azure App Service seems to be completely perplexed when specific server instances are marked as unhealthy. Performance degrades quickly as more and more instances are quarantined and taken out of rotation while new ones have to be created and spun up. What a mess!

If you’re using a managed API gateway like Azure API Management (APIM), there’s a fairly elegant way to deal with this particular problem. Have your API return 200-series statuses for these conditions, keeping the health checks ignorant and happy, then translate the outbound HTTP statuses to the ones you really want to convey at the edge. Here’s some outbound policy you can add to your API to pick up a special response header named RealHttpStatus and return that instead.

The policy begins by fetching the special response header called RealHttpStatus if it exists. The API should inject this header whenever it means to return something that might be misinterpreted or mishandled by the application server’s management tools. Next, the policy removes the special header so clients won’t see how our chicanery was perpetrated. Lastly, if the integer value of the special header is 404, the actual status returned by APIM will be 404, regardless of what the back end actually provided via the actual HTTP status code.

Of course, this policy’s <choose> element can be extended to include as many other interesting HTTP status types as you might require. Your API can go on respecting HTTP for all its beauty and prescience while keeping those fossilized ne’er-do-wells completely in the dark about your villainous plans.

Create SAS Tokens for Azure API Management with an Azure Function

Shared Access Signature (SAS) tokens are required to call Azure API Management’s original REST API. We can generate these manually on the Azure portal for testing. However, in production, if you want to invoke the APIM REST APIs programmatically, you’ll need to generate these tokens with a bit of code. There’s a snippet available in the APIM documentation that shows how to do this but it’s (possibly) got a flaw that I’ll address below. Moreover, with Azure Functions available these days, it makes sense to expose this token generator as a service. Here’s the code for an Azure Function to do just that:

This Azure Function requires two web application settings named APIM_SAS_ID and APIM_SAS_KEY to be used in the hashing process. You can fetch those from the APIM publisher portal (and wherever they may be on the main Azure portal once APIM is fully integrated there). The token that gets generated from this code will be good for ten minutes. You can add more time if you like by modifying the line of code that calls DateTime.AddMinutes(). Currently, APIM SAS tokens can be generated to last up to thirty days although it’s not good practice to make them last that long.

The problem that I found with the snippet of code that was shown in the APIM documentation is that the inclusion of the seconds in the expiration time caused it to fail validation no matter how the middle (EX) portion of the SAS token was formulated. Perhaps I was doing something else wrong but I found that by setting the seconds to zero in the expiration date, I was able to generate SAS tokens that are honored by the APIM REST API. Here’s a GET operation that fetches the value of a property in APIM named SOME_APIM_PROP using the SharedAccessSignature Authorization schema:

With this Azure Function in place (and the credentials to access it), I can generate SAS tokens for APIM any time I like using a simple, clean HTTP interface. Azure Functions are great architectural building blocks for any modern, API-centric design. If you agree or disagree with that assertion, let me know by reaching me on Twitter @KevinHazzard. Enjoy.

Extract JWT Claims in Azure API Management Policy

JSON Web Tokens (JWT) are easy to validate in Azure API Management (APIM) using policy statements. This makes integration with Azure Active Directory and other OpenID providers nearly foolproof. For example, one might add the following directive to the <inbound> policy for an API to ensure that the caller has attached a bearer token with acceptable audience, issuer and application ID values in the signed JWT:

That’s nice. A little bit of markup and all that nasty security plumbing is handled outside the API. But what if we want to pass some individual claims named inside the token on to the API backend? Unfortunately, Azure APIM doesn’t have that built into JWT token validation policy. Ideally, we’d be able to extract claims during validation into variables and pass them in HTTP headers before the request is forwarded to the backing API. Until that feature is added, here’s how you can do that:

In this code, I’ve added some script inside the <set-header> policy statement to fetch the Authorization header from the request, check that it’s a Bearer type token, attempt to parse it (which checks the token’s signature), then finally extracts the value of one specific claim. Most of that work already happens inside <validate-jwt> policy, as you can imagine. Until there’s an easier way to extract JWT claims individually, the solution shown here works nicely. Enjoy.

If you agree with me that this feature should be built right into the <validate-jwt> policy, please upvote the feature request I wrote on the APIM feedback site.