Contract-Testing

Achieve System-wide Contract Compliance

Contract Testing holds great promise, especially for systems. It assumes every upstream service (Provider) has a contract, and like in any contract (e.g. a legal contract), it specifies all the methods a downstream service (Consumer) can interact with the Provider, as well as the response of the Provider to any Consumer message or request.

With this assumption in mind, we define a software regression as an outcome where a Provider or a Consumer fails to meet one or more of the terms specified in the service contract.

Assuming every service has a contract, system-wide contract compliance is achieved when all services adhere to their contracts. When compliance is met, the software regressions are significantly reduced.

You can use the following principles to ensure System-wide Contract Compliance whether you utilize 5, 500, or 5,000 services.

Providers & Consumers

A Provider is an upstream service that sends data to a Consumer that is a downstream service, whether synchronously or asynchronously.

Contract Management

Contract Testing is achieved by managing a few types of contracts across Providers, Consumers and versions. Both Contract Testing and Contract Virtualization can be derived from these contracts, helping both Providers ensure contract compliance and Consumers to prepare themselves for integration with the Providers.

Contract-Management

Service Contract

At the heart of Contract Testing lies the Service Contract. A Service Contract (Contract) usually belongs to a Provider and lists all the methods a Consumer can interact with the Provider. Interactions result in data returned or actions performed.

To make Contract Testing useful, every service should have an up-to-date Contract. The service owner should publish this Contract at all times for two main reasons:

  • So Consumers can derive their Contract variants and build their code with the Consumer context in mind.
  • So Providers can avoid software regressions by ensuring Contract compliance.

In the case of synchronous communication, the Contract is likely to list the service endpoints used to interact with the Provider, while in the asynchronous case, it will likely include the message schema.

While there is no clear guidance of how to represent a Contract, OpenAPI specifications (with its friendly name, Swagger) is a popular choice.

Contract-Sample

There are two types of Service Contracts:

Provider and Consumer Contracts

Provider Contract

The Provider Contract includes a comprehensive list of methods guiding Consumers as to how to interact with the Provider, with no specific Consumer in context.

The Provider Contract is published by the Provider with the promise to adhere to the Contract. Consumers can derive a variant of the Provider Contract for their own usage and context.

Consumer Contract

A Consumer Contract represents the agreement a certain Consumer has with a Provider. The Consumer Contract is a variant of the Provider Contract for use by a specific Consumer, and therefore includes only a subset of the Provider Contract.

The Consumer Contract usually provides a better understanding of how the Consumer can interact with the Provider as well as mock the Provider for testing purposes.

Contract Testing

Contract Test-suite

A Contract Test-suite includes a collection of test-cases, providing comprehensive test-coverage to the Contract, usually covering all functional aspects of the Contract methods as well as performance and load.

There are two types of Contract Test-suites:

Consumer Contract Test-suites

A Consumer Contract Test-suite (Consumer Test-suite) can be used to ensure the Contract compliance of a Provider in the context of a certain Consumer.

For example, a Consumer can ask a Provider to use the Consumer Test-suite to make sure there are no software regressions.

A Provider can run a certain Consumer Test-suite in case they want to ensure no potential issues with a certain Consumer, especially when changes are expected.

Provider Contract Test-suite

A Provider Contract Test-Suite (Provider Test-Suite) includes a comprehensive collection of test-cases with no specific Consumer in mind.

Provider Test-Suites can be used to ensure a Provider’s compliance to its Contract.

Contract Virtualization

While testing helps Providers ensure Contract Compliance, Virtualization can help Consumers prepare for integration with Providers. Usually the problem is far more complex than having a one-to-one relationship.

Individual services are usually part of an ecosystem of services that communicate with each other. Transactions and flows usually involve numerous services. Service Dependencies usually inhibit testing services in isolation before integration.

Contract Mock

A Contract Mock (Mock) mocks a Provider. It is a lightweight service trained to respond as the Provider, usually during a predefined test.

Mocks help developers of a Consumer test the Consumer’s interface with a Provider without actually integrating with the Provider, by using a Mock of the Consumer Contract.

In the Monolith days, having a Mock would be enough. That’s not the case in a Microservices environment, where you have numerous services with some level of dependency.

Test Environment

As services often have dependencies with other services, it's hard to test services outside of integration, making Test Environments a requirement when you’d like to test services in isolation and before integration.

By having a Test Environment, developers of a Consumer can test their integration with a Provider without having to actually integrate with the Provider, an action that is likely to reduce the risk of regression during integration.

Test Environments can be expensive to build and cumbersome to maintain.

There are two types of of test environments:

Ephemeral Environment

An Ephemeral Environment is usually a hosted, complete environment with data and real services. While devops can build and manage a few environments, it becomes challenging when more environments are required, as when the number of services grows.

Virtual Environments

A Virtual Environment uses Mocks instead of real services and Mock data instead of real data (e.g. a DB). Virtual Environments are lightweight enough to run on developer environments (e.g. Laptops) and require no hosting. When compared to Ephemeral Environments, they are more limited in scope, however they can scale across a growing number of services and developers.

Communication Patterns

Services communicate using two primary patterns:

Kafka Vs Rest Vs gRPC

Synchronous Communication

Until recently, synchronous communication was the most common communication pattern, leveraging protocols such as REST, and more recently, , which is more performance-oriented.

Rest and gRPC

Synchronous communication includes request/response transactions, where the response immediately follows the request. An upstream service will usually wait for incoming requests using an agreed-upon communication protocol (e.g REST or ), and will then return a response including either data or the status of an action.

Asynchronous Communication

Lately, there has been a strong trend toward asynchronous communication, which further decouples the services.

Kafka

Asynchronous communication includes reading and writing messages to a message bus. A Provider will usually post (write) a message to a message bus (e.g. Kafka) that will be consumed (read) by Consumers.

Why Contract Testing

Traditional testing is losing relevance, impacting developer productivity, software reliability and speed to market.

Developers spend more than 25% of their time building tests and maintaining test environments. Yet, testing is the No. 1 reason releases are delayed, especially with the onset of new technologies like Kubernetes and Kafka adding even more complexity.

End-to-end testing was built for web and mobile applications, and does not scale well when organizations embrace cloud-native and the number of services grow.

Contract Testing holds great promise, especially related to cloud-native systems. Contract testing scales across a growing number of services requiring about the same level of effort whether you have 5, 500 or 5,000 services.

Contract Testing can help you avoid software regressions by enforcing system-wide contract compliance by making sure every service adheres to its service contract.