Versioning APIs

I have blogged in the past about microservices, and the advantages that architectural style can bring. These small, focussed and, most importantly, autonomous services commonly expose their functionality via a REST interface.

Inevitably there will come a time when you need to change that interface. Yes, in an ideal world, you will come up with a perfect API first time round and it will never need to change, but requirements change or new users come on board, and we must adapt. Read on to find some approaches to dealing with changing interfaces without breaking clients.

Backwards compatibility

Changes interfaces is usually only a problem when they break backwards compatibility. If calls to the API that worked with the old version will no longer work with the new API, you are forcing existing clients to upgrade.

Again, one of the key drivers of microservices is autonomy; being independent from other services – and from the client. If your clients need to be updated in synch with your server, you have lost the ability to upgrade independently.

For example, suppose you expose an endpoint such as http://acme.com/api/customers, and users can request or submit a customer using a customer id, such as http://acme.com/api/customers/1234, for a customer resource such as this:

{
     "id": "1234",
     "firstName": "John",
     "lastName": "Smith",
     "balance": 0, }

But at some point, it is decided that balance shouldn’t be a part of customer, that it is expensive to lookup, and that most clients don’t actually use the value anyway. So, the representation of customer is changed to:

{
     "id": "1234",
     "firstName": "John",
     "lastName": "Smith",
}

Even small changes like this can cause headaches since even client’s that didn’t use the balance field in any meaningful way may now break.

So, what are the best ways to deal with such non-backwards-compatible changes?

 

Don’t change the interface!

Martin Fowler described describes creating services that are a Tolerant Reader. That is, interfaces should be designed to be flexible and forgiving by obeying Postel’s Law (aka the robustness principle): “Be conservative in what you do, be liberal in what you accept.” (Note I think “what you do” can be better read as “what you send”).

This allows the interface to remain as is, even when the details of what are being passed changes.

What if you simply have to introduce a “breaking” change? How do you best deal with it?

 

Force clients to upgrade

One option is to simply force the service and all clients to upgrade in lockstep. That is, you upgrade the service, and force all users to start using that new service immediately, or get errors. This may be acceptable (and controllable) in internal corporate networks, but less palatable when your clients are out of your control.

How about less harsh options?

 

Consumer driven contracts

Consumer driven contracts are a way for services to evolve while reducing the risk of breaking clients. Each consumer records their expectation of the service in a contract file, which they then publish to the service. Since the service is now aware of what each client expects, changes that could break those contracts as easily identifiable. This is in contrast to making a change to the service, publishing it, and only then hearing from clients that you have “broken” them.

The two main ways to implement consumer driven contract testing seem to be Pact and Pacto.

 

Semantic Versioning

Semantic versioning allows you to look at the version number of a service and know how significant the changes are, and hence if you can integrate with it.

Semantic versioning comes in the form

major.minor.patch

Major: A major change that likely breaks backwards compatibility

Minor: A minor change, such as a new feature, that maintains backwards compatibility

Patch: Bug fix only. No changes to interfaces and backwards compatibility maintained

 

 

Supporting multiple versions

Coexisting endpoints

Coexisting endpoints involves supporting multiple versions in the same app.

In v100 of Loan Service below, we have a single end point used by both Client1 and Client2.

In v109 of the Loan Service, we introduce a new v2 of the endpoint. But rather than deploying a new version of the service that supports only the new v2 interface, and hence breaking the clients that can only talk to the old version, we instead release a new version of the app that supports both the old v1 and the new v2.

This should be a temporary stepping stone, since supporting both versions can be laborious, and restrictive e.g. it may stop you removing other code or infrastructure (such as db tables) that are only used by the old version. However, this stepping stone allows you to get new features out asap, available for new clients or existing clients that can upgrade to take advantage of the features if they wish. However it allows other existing clients a grace period to upgrade but still function fine in the meantime.

An alternative approach, shown by v112 of Loan Service is where v2 is still supported but simply translates the call into one compatible with the new v2 interface. This isolates the ‘old’ code that needs to be maintained.

Finally, as shown in v120 of Loan service, the old end points should be deleted as soon as all clients have been migrated.

This is an example of the expand and contract pattern, and is adapted from an example in the awesome “Building Microservices” book by Sam Newman.

 

Co-existing servers

Another approach to dealing with changing interfaces is to run multiple versions of the app in parallel.

Again, adapted from an example in “Building Microservices” book by Sam Newman.

Issues:

• Bug fixes need to be applied to both versions of the codebase
• Routing to the correct version of the app may involve updates to middleware, routers, load balancers
• Are both versions of the app persisting to the same database? If so, are they persisting data that is compatible to both versions of the app?

 

This is a common approach when using Blue Green Deployments, but using it for longer term migration strategies can be problematic.

Specifying the version

When you support multiple versions, whether via the expand-contract option of multiple endpoints in the same app, or via multiple instances of the same app, you need to allow the client to signal which version they wish to use.

There are 2 main ways of doing this

1. Versioned URI

http://acme.com/api/v3/customer/123

2. Request Headers

For example, using the Accept-Version header (which is what Restify uses)

There are many discussions on the web about approaches to handle REST API versioning, including on stackoverflow, and this article on Lexical Scope.

 

Sources and references

“Building Microservices” book by Sam Newman.

Consumer Driven Contracts by Ian Robinson

Pact (github.com)

Pacto (thoughtworks.com)

 

Tags: api, cdc, consumerDrivenContracts, HTTP, json, microservices, pact, REST, RESTful, SOA