top of page

28 December 2023

9 Min. Read

Contract Testing

Top 5 Best API contract testing tools in 2024

Consumer-driven Contract Testing with PACT A Step-by-Step Guide.jpg

Key Highlights

In this blog, we cover following key highlights:

  1. Learn about why contract testing suite is a better choice than E2E test suite

  2. Get insights on why Nubank moved entirely from E2E to Contract tests

  3. See how contract testing works along with all the advantages of it

  4. Get to know about various contract testing tools

This guide introduces a proactive testing strategy, optimizing microservices benefits while avoiding interdependency pitfalls.

“If these trends continue, we’ll take more and more time in our continuous integration and deployment pipelines, and eventually find ourselves stuck in a corner. Folks will commit bigger and bigger batches of changes that will take longer to be deployed, while still being open to the risk of bugs in production”.


-The pain of using end-to-end tests cases nicely presented by an engineering folk at Nubank


However, the issue doesn't stop there; it's a widespread challenge shared by all companies employing the practice of implementing end-to-end tests to identify issues within the boundaries of microservices architecture.


Graph

Considering the number of services and engineers they had, they got to know that it would take an infinite time to run! And hence, they switched to contract testing.


What is Contract Testing?

Imagine you're planning a big family dinner. You're responsible for the turkey, your sister is bringing the dessert, and your cousin is in charge of the side dishes. To ensure everything goes smoothly, you all agree on specific details beforehand: the dinner starts at 6 PM, the turkey should be large enough to feed 10 people, the dessert must be nut-free due to allergies, and the side dishes should include vegetarian options. This agreement is essential to make sure everyone enjoys the meal and nobody is left hungry or with food they can't eat.


Now, apply this scenario to software development. In this world, different parts of a software application (like services or components) are like family members preparing different parts of the meal. Contract Testing is the agreement or 'contract' between these parts. It ensures that when one part of the software (say, the 'turkey') communicates with another (like the 'dessert'), they both understand and fulfill each other's expectations. For instance, if one service expects data in a certain format, the other service needs to send it in that format.


Just like how your family dinner could turn into a disaster if someone doesn't stick to the agreed plan (imagine a turkey too small or a dessert with nuts!), software can malfunction if one part doesn’t meet the agreed-upon expectations.


Contract Testing is like doing a practice run before the big dinner, making sure everyone knows their part and sticks to the plan. This helps in catching any issues early, preventing a big mess during the actual event (or in software terms, when the application is live and being used by real users).


What is consumer-driven contract testing?

Consumer-Driven Contract Testing is the widely accepted approach of performing contract testing.


Basically there are two parties involved in a contract, one asking for the data(consumer) and the other one providing the data(provider).


Here, the consumer of the service dictates the terms of the contract. It tells the provider what it expects in terms of data format and structure. The provider then ensures that it can meet these expectations. This approach has several benefits:


  1. Flexibility: Consumers define their requirements, leading to more flexibility and less risk of miscommunication.

  2. Independence: Teams can work independently on their services, as long as they adhere to the agreed contracts.

  3. Reduced Risk of Breakdowns: By ensuring that the provider meets the consumer's expectations, the risk of breakdowns in communication between services is significantly reduced.


How Does Contract Testing Work?

  1. Defining the Contract: This is the core of Contract Testing. A contract is a detailed agreement on how two services will interact. It includes the expected requests and responses, specifying the data format, structure, and types of fields. For example, a contract might state that a request must include a customer ID as an integer and the response must include a status message.


  2. Creating Contract Tests: Once the contract is defined, each service involved writes tests to verify they can meet their part of the contract. For the 'consumer' service (the one making requests), tests check if it sends requests correctly as per the contract. For the 'provider' service (the one responding to requests), tests ensure it can handle incoming requests and provide responses that match the contract.


The Process in Practice

Let's consider a practical example involving two services: a 'User Service' and an 'Order Service' in an e-commerce platform.


  1. Step 1: Contract Creation

    • The teams responsible for the User Service and the Order Service come together to define the contract.

    • They agree that when the Order Service requests user details, it will send a user ID, and the User Service will respond with the user's name and address.


  2. Step 2: Implementing Contract Tests

    • The Order Service team writes tests to ensure their service sends the correct request format with a valid user ID.

    • The User Service team writes tests to verify their service can handle these requests and respond with the correct user information.


  3. Step 3: Continuous Testing and Integration

    • As development progresses, these tests are run frequently.

    • If the User Service team changes their API and the response no longer matches the contract, the tests for the Order Service will fail. This signals a breach in the contract, alerting teams to resolve the issue.


  4. Step 4: Versioning and Evolution

    • Contracts can evolve. When a service needs to change its API, it must also update the contract.

    • This ensures ongoing communication and agreement between teams, preventing unexpected failures in production due to mismatched expectations.


What are the Benefits of Contract Testing?

  • Problems are caught during development, not after deployment.

  • Teams can work on their services without constant coordination, as long as they adhere to the contracts.

  • Ensures that as long as the contract is respected, services will interact seamlessly in production.

  • Contract testing streamlines the integration and examination of microservices, making the process smoother.

  • The upkeep of the system is simplified and becomes less burdensome.

  • Contract testing allows for focused attention on individual modules. For instance, to assess module A's contract, there's no need for full integration with other modules; it can be evaluated on its own.

Interested in learning
how HyperTest can help you?

Get a Personalised Demo within 24 Hrs.

Contract Testing Use Cases

Contract testing stands out as an effective technique for verifying the dependability and interoperability of microservices and APIs. Nonetheless, it's important to note that it doesn't suit every testing need. Below, we outline several typical scenarios where contract testing is particularly beneficial:


Use Case 1: User Authentication in a Social Media App


Scenario:

In a social media application, there are two microservices: User Service and Post Service. The User Service handles user authentication, while the Post Service manages the creation and display of posts.


Contract:

The contract specifies that when the Post Service receives a user ID, it sends a request to the User Service to authenticate the user and receive basic user profile data.


Example Contract (JSON format):

{
  "request": {
    "path": "/authenticateUser",
    "method": "POST",
    "body": {
      "userId": "string"
    }
  },
  "response": {
    "status": 200,
    "body": {
      "userId": "string",
      "userName": "string",
      "isAuthenticated": "boolean"
    }
  }
}

Testing the Contract:

  • User Service: Tests to ensure it can process the authentication request and return user data in the correct format.


  • Post Service: Tests to verify it sends the correct user ID format and handles the received user data appropriately.


Use Case 2: Inventory Check in an Online Retail System


Scenario:

An online retail system involves a Product Catalog Service and an Inventory Service. The Catalog Service lists products, while the Inventory Service manages stock levels.


Contract:

The contract outlines that the Product Catalog Service will request stock information from the Inventory Service for specific products.


Example Contract (JSON format):

{
  "request": {
    "path": "/getStockInfo",
    "method": "GET",
    "queryParameters": {
      "productId": "string"
    }
  },
  "response": {
    "status": 200,
    "body": {
      "productId": "string",
      "stockLevel": "integer"
    }
  }
}

Testing the Contract:

  • Inventory Service: Tests to confirm it can handle stock information requests and provide data in the specified format.


  • Product Catalog Service: Tests to check if it correctly queries the Inventory Service and interprets the stock level data.


Use Case 3: Payment Processing in a Booking System


Scenario:

In a hotel booking system, there's a Booking Service and a Payment Service. The Booking Service handles reservation details, and the Payment Service processes payments.


Contract:

The contract states that the Booking Service will send payment details to the Payment Service and expect a confirmation response.


Example Contract (JSON format):

{
  "request": {
    "path": "/processPayment",
    "method": "POST",
    "body": {
      "bookingId": "string",
      "amount": "float"
    }
  },
  "response": {
    "status": 200,
    "body": {
      "paymentId": "string",
      "status": "string"
    }
  }
}

Testing the Contract:

  • Payment Service: Tests to ensure it can process the payment details and return a confirmation with the correct format.


  • Booking Service: Tests to verify it sends accurate payment information and handles the payment confirmation appropriately.


In these use cases, Contract Testing ensures that the microservices can reliably communicate with each other, adhering to the predefined contracts, which is vital for the smooth functioning of complex, distributed systems.


API Contract Testing Tools in 2024

API Contract Testing Tools are essential in modern software development, especially when dealing with microservices architectures. These tools help ensure that APIs behave as expected and adhere to their defined contracts.


We have covered both the free tools and the paid tools in the API Contract Testing category. The top 10 best performing API Contract Testing Tools to consider for 2024 are:


  1. HyperTest

  2. PACT

  3. Swagger/OpenAPI

  4. Spring Cloud Contract

  5. Dredd


1. HyperTest - API Contract Testing Tool

  • HyperTest is a modern tool specifically designed for API contract testing. It offers robust capabilities for ensuring that APIs meet their specified contracts.


Key Features:

  • Automated Contract Validation: HyperTest automates the process of validating APIs against their defined contracts, ensuring compliance and consistency.


  • Easy Integration: Designed to easily integrate with existing development workflows and CI/CD pipelines, making it convenient for continuous testing.


  • Comprehensive Reporting: Provides detailed reports on test results, making it easier to identify and address any issues or deviations from the contract.


  • Language and Framework Agnostic: Can be used with a variety of programming languages and frameworks, offering flexibility for diverse development environments.


2. Pact - API Contract Testing Tool

  • Pact is a popular open-source tool for contract testing. It focuses on the interactions between consumer and provider by defining and verifying HTTP requests and responses.


Key Features:

  • Consumer-Driven Contracts: Pact allows the consumer to define the expected behavior of the provider, which can then be verified by the provider.


  • Mock Service: It provides a mock service for the consumer to interact with during testing, ensuring that the consumer's requests match the contract.


  • Integration with CI/CD: Pact integrates seamlessly with continuous integration/continuous deployment pipelines, enhancing the development workflow.


  • Language Support: Offers wide language support including Ruby, JVM languages (Java, Kotlin, Scala), .NET, JavaScript, Swift, and more.


3. Swagger/OpenAPI - API Contract Testing Tool

  • The OpenAPI Specification (formerly known as Swagger) is more of a framework than a tool but is integral in defining RESTful APIs and is often used in contract testing.


Key Features:

  • API Design and Documentation: Enables designing APIs and generating documentation that can be used as a contract for testing.


  • Tool Ecosystem: Many tools support OpenAPI, offering functionalities for generating server stubs, client libraries, and performing contract testing.


  • Community and Support: Large community and widespread industry support.


4. Spring Cloud Contract - API Contract Testing Tool

  • Designed for Spring applications, this tool is used for implementing Consumer-Driven Contract (CDC) testing.


Key Features:

  • Integration with Spring: Perfect for applications built with the Spring framework.


  • Stub Runner: Automatically generates stubs for the consumer, which can be used for tests.


  • Supports Messaging: Apart from HTTP, it also supports contract testing for asynchronous messaging.


5. Dredd - API Contract Testing Tool

  • Dredd is a language-agnostic HTTP API testing tool that validates whether an API implementation adheres to its documentation.


Key Features:

  • Support for API Blueprint and OpenAPI: Works with API Blueprint and OpenAPI specifications.


  • Hooks: Offers hooks in several languages to set up preconditions or clean up after tests.


  • Continuous Integration: Easy integration with CI tools and services.


Conclusion

Each of these tools has its strengths and fits different needs in the API development lifecycle. The choice of tool often depends on the specific requirements of the project, such as the programming language used, integration capabilities, and the complexity of the API interactions. By employing these tools effectively, teams can ensure more reliable and robust API communication within their applications.


Here is a detailed comparison chart of two widely used contract testing tools, get to know more technical details about them.

Frequently Asked Questions (FAQs)

1. Which tool is used for contract-driven testing?

Pact is a popular tool for contract-driven testing, ensuring seamless integration in distributed systems. It allows teams to define and manage contracts, validating interactions between services to prevent issues caused by changes. Pact supports various languages and frameworks, making it versatile for diverse technology stacks.

2. Is contract testing same as API testing?

No, contract testing and API testing differ. API testing examines the functionality and performance of an API, while contract testing focuses on verifying agreements or contracts between services to ensure they communicate correctly. Contract testing validates interactions and expectations, enhancing compatibility in distributed systems.

3. What is the basic of contract testing?

Contract testing ensures seamless integration in distributed systems. It relies on predefined agreements or contracts specifying expected behaviors between software components. These contracts serve as benchmarks for validation, preventing issues caused by changes during development.
bottom of page