PACT Contract Testing: A Step-by-Step Guide

In our previous contract testing article, we covered the basics of what contract testing is and how it works. Now, in this blog post, we'll introduce you to a popular tool for contract testing—PACT Contract testing.

What is PACT contract testing?

Let's understand why PACT contract testing became essential through a real team retrospective about a production failure.

Q: Why did our user profile feature break in production when the Auth service team said they only made a "minor update"?

A: We were consuming their /user endpoint expecting the response to always include a phone field, but they changed it to optional without telling us.

Q: But didn't we have unit tests covering the user profile logic?

A: Yes, but our unit tests were mocking the Auth service response with the old structure. Our mocks had the phone field as required, so our tests passed even though the real service changed.

Q: Why didn't integration tests catch this?

A: We only run full integration tests in staging once a week because they're slow and flaky. By then, the Auth team had already deployed to production and moved on to other features.

Q: How could we have prevented this?

A: If we had a contract between our services - something that both teams agreed upon and tested against - this wouldn't have happened. That's exactly what PACT contract testing solves.

PACT is a code-based tool used for testing interactions between service consumers and providers in a microservices architecture. Essentially, it helps developers ensure that services (like APIs or microservices) can communicate with each other correctly by validating each side against a set of agreed-upon rules or "contracts".

Here's what PACT does in a nutshell:

• It allows developers to define the expectations of an interaction between services in a format that can be shared and understood by both sides.

• PACT provides a framework to write these contracts and tests for both the consuming service (the one making the requests) and the providing service (the one responding to the requests).

PACT has a lot of manual effort involved in generating the test cases, move beyond that and adopt in a fast-performing approach that auto generates test cases based on your application's network traffic. Curious to know more?

• When the consumer and provider tests are run, PACT checks whether both sides adhere to the contract. If either side does not meet the contract, the tests fail, indicating an issue in the integration.

• By automating these checks, PACT helps teams catch potential integration issues early and often, which is particularly useful in CI/CD environments.

So, PACT focuses on preventing breaking changes in the interactions between services, which is critical for maintaining a reliable and robust system when multiple teams are working on different services in parallel.

Importance of PACT Contract Testing

➡️PACT reduces the complexity of the environment that is needed to verify integrations, as well as isolates changes to the specific interaction between services. This prevents cascading failures and simplifies debugging.

➡️Decoupling for Independence: PACT enables microservices to thrive on decoupled, independent development, testing, and deployment, ensuring adherence to contracts and reducing compatibility risks during the migration from monoliths to microservices.

➡️Swift Issue Detection: PACT's early identification of compatibility problems during development means faster feedback, with precise, interaction-focused tests that expedite feedback and streamline change signoffs.

➡️Enhanced Collaboration and Confidence: Clear, shared service interaction contracts reduce misunderstandings, fostering collaboration and developer confidence in releasing changes without breaking existing contracts.

➡️Living Documentation: Pact contracts serve as dynamic, clear-cut documentation, simplifying developers' comprehension of integration points.

➡️Reduced Service Outages: Pact contract tests swiftly highlight provider service changes that break consumer expectations, facilitating quick identification and resolution of disruptive modifications.

How does Pact implement contract testing?

Pact implements contract testing through a process that involves both the consumer and the provider of a service, following these steps:

➡️Consumer Testing:

• The consumer of a service (e.g., a client application) writes a test for the expected interaction with the provider's service.

  
  

• While writing this test, Pact stubs out the actual provider service and records the expectations of the consumer—what kind of request it will make and what kind of response it expects—into a Pact file, which is a JSON file acting as the contract.

  
  

• The consumer test is run with the Pact mock service, which ensures the consumer can handle the expected response from the provider.

➡️Pact File Generation:

• When the consumer tests pass, the Pact file (contract) is generated. This file includes the defined requests and the expected responses.

➡️Provider Verification:

• The provider then takes this Pact file and runs it against their service to verify that the service can meet the contract's expectations.

  
  

• The provider's tests take each request recorded in the Pact file and compare it against the actual response the service gives. If they match, the provider is considered to be in compliance with the contract.

  
  

➡️Publishing Results:

• Results of the provider verification can be published to a Pact Broker, which is a repository for Pact files. This allows for versioning of contracts and tracking of the verifications.

  
  

• Both the consumer and the provider use the Pact Broker to publish and retrieve Pact files. It helps to ensure that both parties in the service interaction are always testing against the latest contract.

  
  

➡️Continuous Integration:

• Pact is often integrated into the CI/CD pipeline. Whenever changes are made to the consumer or provider, the corresponding contract tests are automatically run.

  
  

• This helps in identifying any breaches in the contract immediately when a change is made, ensuring that any integration issues are caught and addressed early in the development lifecycle.

  
  

➡️Version Control:

• Pact supports semantic versioning of contracts, which helps in managing the compatibility of interactions between different versions of the consumer and provider services.

By automating the creation and verification of these contracts, Pact helps maintain a reliable system of independent services by ensuring they can communicate as expected, reducing the likelihood of integration issues in a microservices architecture.

How to perform Pact Contract Testing?

Now we all know that Pact is a code-first tool for testing HTTP and message integrations using contract tests. Instead of testing the internal details of each service, PACT contract testing focus on the "contract" or the agreement between services on how their APIs should behave.

Step 1: Define the Consumer Test

In the consumer service, you would write a test that defines the expected interaction with the provider's API.

Step 2: Run the Consumer Test

When this test is executed, the pact context manager starts the mock service, and the defined interaction is registered with it. Then, the test makes a request to the mock service, which checks that the request matches the registered interaction. If it does, it responds with the predefined response.

Step 3: Generate the Contract (Pact File)

If all assertions pass and the test completes successfully, Pact will generate a .json file representing the contract. This file is then used by the provider to verify that their API meets the expectations defined by the consumer.

{
    "consumer": {
        "name": "ConsumerService"
    },
    "provider": {
        "name": "ProviderService"
    },
    "interactions": [
        {
            "description": "a request for user id 1",
            "providerState": "a user with id 1 exists",
            "request": {
                "method": "GET",
                "path": "/user/1"
            },
            "response": {
                "status": 200,
                "body": {
                    "id": 1,
                    "name": "John Doe",
                    "email": "john.doe@example.com"
                }
            }
        }
    ],
    "metadata": {
        "pactSpecification": {
            "version": "2.0.0"
        }
    }
}

Step 4: Verify the Provider with the Pact File

The provider's test suite would use this .json Pact file to ensure their service can handle the requests and send the expected responses. The provider doesn't necessarily need to know the internals of the consumer; it just needs to satisfy the contract as outlined in the Pact file.

The Verifier uses the pact file to make requests to the actual provider service and checks that the responses match the contract. If they do, the provider has met the contract, and you can be confident that the provider and consumer can communicate correctly.

Problems with PACT

PACT contract testing has become very popular among teams off late given its simplicity and effectiveness. But it comes with its own set of challenges, making adoption at scale a challenge.

There are some obvious challenges in getting started and also the manual intervention in contract maintenance doesn’t make it the perfect fit for testing microservices alone.

👉Complex setup and high maintenance

👉CI/CD Pipeline Integration Challenges

👉High Learning Curve

👉Consumer Complexity

👉Test Data Management

Let’s get started with all of them, one-by-one.

1. Lots of Manual Effort Still Needed

Pact contracts need to be maintained and updated as services evolve. Ensuring that contracts accurately reflect real interactions can become challenging, especially in rapidly changing environments. Ensuring that contracts accurately reflect the expected interactions can become complex, especially when multiple consumers are involved.

Any time teams (especially producers) miss updating contracts, consumers start testing against incorrect behaviors which is when critical bugs start leaking into production.

➡Initial Contract Creation

Writing the first version of a contract involves a detailed understanding of both the consumer's expectations and the provider's capabilities. Developers must manually define the interactions in test code.

# Defining a contract in a consumer test
@pact.given('user exists')
@pact.upon_receiving('a request for a user')
@pact.with_request(method='GET', path='/user/1')
@pact.will_respond_with(status=200, body={'id': 1, 'name': 'John Doe'})
def test_get_user():
    # Test logic here

This change must be communicated and agreed upon by all consumers of the API, adding coordination overhead.

➡ Maintaining Contract Tests

The test suite for both the consumer and the provider will grow as new features are added. This increased test suite size can make maintenance more difficult.

Each function represents a new contract or a part of a contract that must be maintained.

# Over time, you may end up with multiple contract tests
def test_get_user():
    # ...

def test_update_user():
    # ...

def test_delete_user():
    # ...

2. Testing Asynchronous Patterns

Pact supports non-HTTP communications, like message queues or event-driven systems, but this support varies by language and can be less mature than HTTP testing.

// A JavaScript example for message provider verification
let messagePact = new MessageProviderPact({
  messageProviders: {
    'a user created message': () => Promise.resolve({ /*...message contents...*/ }),
  },
  // ...
});

This requires additional understanding of how Pact handles asynchronous message contracts, which might not be as straightforward as HTTP.

3. Consumer Complexity

In cases where multiple consumers interact with a single provider, managing and coordinating contracts for all consumers can become intricate.

➡ Dependency Chains

Consumer A might depend on Consumer B, which in turn depends on the Provider. Changes made by Provider could potentially impact both Consumer A and the Consumer B. This chain of dependencies complicates the contract management process.

💡 Let’s understand this with an example:

Given Services:

- Provider: User Management API.
- Consumer B: Profile Management Service, depends on the Provider.
- Consumer A: Front-end Application, depends on Consumer B.

Dependency Chain:
- `Consumer A` depends on `Consumer B`, which in turn depends on the `Provider`.

Change Scenario:

- The `Provider` adds a new mandatory field `birthdate` to its user data response.
- `Consumer B` updates its contract to incorporate `birthdate` and exposes it through its endpoint.
- `Consumer A` now has a failing contract because it doesn't expect `birthdate` in the data it receives from `Consumer B`.

Impact:

- `Consumer A` needs to update its contract and UI to handle the new field.
- `Provider` needs to coordinate changes with both the `Consumer B` and `Consumer A` to maintain contract compatibility.
- The `Provider` must be aware of how its changes affect downstream services to avoid breaking their contracts.
</aside>

➡ Coordination Between Teams

When multiple teams are involved, coordination becomes crucial. Any change to a contract by one team must be communicated to and accepted by all other teams that are consumers of that API.

# Communication overhead example
# Team A sends a message to Team B:
"We've updated the contract for the /user endpoint, please review the changes."

This communication often happens outside of Pact, such as via team meetings, emails, or chat systems.

4. Test Data Management

Test data management in Pact involves ensuring that the data used during contract testing accurately represents real-world scenarios while maintaining consistency, integrity, and privacy. This can be a significant challenge, particularly in complex microservices ecosystems. The problems that might arise would be:

➡ Data Generation

Creating meaningful and representative test data for all possible scenarios can be challenging. Services might need specific data states to test different interactions thoroughly.

➡ Data Synchronization

PACT tests should use data that accurately reflects the behavior of the system. This means that the test data needs to be synchronized and consistent across different services to ensure realistic interactions. Mismatched or inconsistent data can lead to false positives or negatives during testing.

➡ Partial Mocking Limitations

Because Pact uses a mock service to simulate provider responses, it's possible to get false positives if the provider's actual behavior differs from the mocked behavior. This can happen if the provider's implementation changes without corresponding changes to the contract.

How we've fixed the biggest problem with the Pact workflow?

PACT driven integration testing has becoming very popular among teams off late given its simplicity and effectiveness. But some obvious challenges in getting started and contract maintenance still does not make it the perfect solution for integration testing.

So, we at HyperTest have built an innovative approach that overcomes these shortcomings, making contract testing easy to implement and scalable.

In this approach, HyperTest builds contract tests for multiple services autonomously by monitoring actual flows from production traffic.

Principally there are two modes i.e. record mode which records real world scenarios 24x7 and replay/test mode that then replays these scenarios to test the service with an external system, without actually making it live and running.

Let's explore how these two modes work:

Record mode: Automatic tests generation based on real-world scenarios

The HyperTest SDK is placed directly above a service or SUT. It observes and documents all incoming requests for traffic that the service receives. This includes recording the entire sequence of steps that the SUT takes in order to generate a response. The incoming requests represent the paths users take, and HyperTest captures them exactly as they occur.

This ensures that no scenarios are overlooked, resulting in a comprehensive coverage of all possible test cases. In this mode HyperTest records:

👉The incoming request to the SUT

👉The outgoing requests from the SUT to downstream services and databases. Also, the response of these external systems

👉The response of the SUT which is stored (say X’)

• Replay/Test mode: Replay of recorded test scenarios with mocked dependencies

During the (replay) Test mode, integrations between components are verified by replaying the exact transaction (request) recorded during the record mode. The service then makes external requests to downstream systems, databases, or queues whose response are already mocked. HyperTest uses the mocked response to complete these calls, then compares the response of the SUT in the record mode to the test mode. If the response changes, HyperTest reports a regression.

Advantages of HyperTest over PACT

This simple approach of HyperTest takes care of all the problems with PACT. Here is how:

👉Auto-generate service contracts with no maintenance required

HyperTest observes actual calls (requests and responses) and builds contracts in minutes. If requests (consumer) or responses (producer) change breaking the contracts, respective service owners can approve changed contracts with a click for all producers or consumers rather than rewriting in PACT files. This updation of contracts (if needed) happens with every commit.

Respective consumer - provider teams are notified on slack needing no separate communication. This instant feedback on changing behavior of external systems helps developers make rapid changes to their code before it breaks in production.

👉Test Data Management

It is solved by design. HyperTest records real transactions with the real data.

For example:

✅When testing for login, it has several real flows captured with user trying to login.

✅When it tests login, it will replay the same flow (with transactional data) and check if the same user is able to login to verify the right behavior of the application.

HyperTest's approach to aligning test data with real transactions and dynamically updating mocks for external systems plays a vital role in achieving zero bugs in production.

👉Dependency Management

HyperTest autonomously identifies relationships between different services and catches integration issues before they hit production.

Through a comprehensive dependency graph, teams can effortlessly collaborate on one-to-one or one-to-many consumer-provider relationships.

• Notification on Disruption: 

  
  

  Let's developer of a service know in advance when the contract between his and other services has changed.

• Quick Remediation: 

  This notification enables quick awareness and immediate corrective action.

• Collaboration on Slack: 

  This failure is pushed to a shared channel where all developers can collaborate

👉CI/CD integration for early issue detection and rollback prevention

HyperTest identifies issues early in SDLC for developers to quickly test changes or new features and ensure they integrate seamlessly with rest of the system.

✅Early Issue Detection

✅Immediate Feedback

Automatically run tests using your CI/CD pipeline when a new merge request is ready. The results can be observed on platforms like GitHub, GitLab or Bitbucket and sign-off knowing your change will not break your build in production.

👉Build Confidence

Knowing that their changes have undergone rigorous testing and integration checks, devs can sign off with confidence, assured that their code will not break the production build.

This confidence significantly reduces the likelihood of introducing bugs that could disrupt the live system.

Conclusion

While PACT undoubtedly excels in microservices contract testing, its reliance on manual intervention remains a significant drawback in today's agile environment. This limitation could potentially hinder your competitiveness.

HyperTest, comes as a better solution for testing microservices. Offering seamless collaboration between teams without the burden of manual contract creation and maintenance, it addresses the challenges of the fast-paced development landscape. Already trusted by teams at Nykaa, PayU, Urban Company, Fyers, and more, HyperTest provides a pragmatic approach to microservices testing.

To help you make an informed decision, we've compiled a quick comparison between HyperTest and PACT. Take the time to review and consider your options.

If you're ready to address your microservices testing challenges comprehensively, book a demo with us. Happy testing until then! 🙂

Check out our other contract testing resources for a smooth adoption of this highly agile and proactive practice in your development flow:

• Tailored Approach to Test Microservices

• Comparing Pact Contract Testing and HyperTest

• Checklist for Implementing Contract Testing