Nothing lasts forever, including even the best designed APIs.
Let’s imagine you are a developer who has taken over ownership of a Catalog microservice. You’ve been asked to deprecate the /v1/catalog
endpoint in favor of the new /v2/catalog
endpoint. How do you go about this?
Whatever the reason for removal – a new version or a planned end-of-life – the first step in a graceful API deprecation is to observe:
Before you can deprecate the endpoint, you need to first check if the endpoint is actually being used.
For internal endpoints, a great way to start is to search the codebase for calls to the API. However, once you believe all calls have been removed, you will still want to use observability tooling to verify that all usage of the API has indeed stopped. It's possible that you may still be getting traffic from an older version of a service that is still running.
Note that after you remove all API calls from the codebase, company protocol may dictate that you wait several releases before turning off the endpoint. Most established companies have standards for backwards compatibility of their microservice APIs (even internal ones). For example, a company might have a policy requiring 3 releases to pass between deprecation of an API and removal, in the event that there’s a rollback.
Your company’s specific method for determining endpoint usage may vary. Some applications export metrics that they explicitly define on their services (e.g. Prometheus). Some applications are set up to log every inbound HTTP request (e.g. Apache logging).
Another option is to use Pixie, an open source observability tool for Kubernetes applications. Pixie automatically traces request traffic of numerous protocols (HTTP, MySQL, gRPC, and more) using eBPF. But no matter how you gather the data, you’ll need to answer the same questions.
Let’s check for HTTP traffic to the /v1/catalog
endpoint to see if there are any clients of this endpoint.
Output of a PxL script showing all HTTP/2 traffic sent to a specific service.
Now you have an answer: the /v1/catalog
endpoint is actually being used.
Taking a look at the different request paths, you can see that the endpoint contains a wildcard parameter. In this case, it appears we have a /v1/catalog/{uuid}/details
endpoint that takes an uuid
query parameter that will change depending on the product the API client would like to get details about.
Clustering by logical endpoint provies a better high-level view of the usage of the API.
For example, these two calls:
/v1/catalog/d3588631-ad8e-49df-bbd6-3167f7efb291/details/v1/catalog/d3234332-s5fe-s30s-gsh6-323434sdf634/details
Should be clustered together into the logical endpoint:
/v1/catalog/*/details
Let’s cluster the requests to the Catalog service by logical endpoint. Pixie takes a statistical approach to this, but you can also try to manually build patterns with regexes.
Output of PxL script showing all endpoints for a specific service, with high-level latency, error and throughput statistics.
This high-level view of the Catalog service traffic confirms that there are two versions of the /catalog
endpoint receiving traffic and that only the /v1
version has the /details
endpoint.
Unfortunately, your endpoint is still receiving traffic. How do you determine the source so that they can be notified about the deprecation?
Let’s inspect the request headers for clues. Pixie automatically traces full requests, including body and request headers. Service meshes can also capture this type of information in Kubernetes.
Output of a PxL script showing all HTTP/2 traffic to a specific endpoint (with the request headers expanded in JSON form).
Here, you can see that the request headers include a Referer
and API-Key
field. Aggregating these values gives us a list of API clients to notify:
Output of a PxL script listing unique values for the request header Referer
and API-Key
fields.
Can’t find any information identifying the API client in the request headers?
Here are some other places to check:
Any API clients you identify should be notified of the impending deprecation. If certain clients fail to migrate to the new API, this sort of identifying information could be used to implement a progressive shutdown that affects clients differently. For example, free-tier clients could have their deprecated API request responses slightly delayed, while paying clients could continue using the deprecated API without penalty.
Now that you know how your API is being used, you can create a deprecation plan.
Developers don't appreciate surprise deprecations, so it’s best to notify in multiple ways, including:
Once you’ve done your best to migrate remaining clients off the deprecated API, it’s time to turn off the endpoint. Tech debt eliminated!
Interested in a Tutorial? Learn how to run the scripts included in this post.
Questions? Find us on Slack or Twitter at @pixie_run.
Terms of Service|Privacy Policy
We are a Cloud Native Computing Foundation sandbox project.
Pixie was originally created and contributed by New Relic, Inc.
Copyright © 2018 - The Pixie Authors. All Rights Reserved. | Content distributed under CC BY 4.0.
The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our Trademark Usage Page.
Pixie was originally created and contributed by New Relic, Inc.