Testing Pyramid and Microservices

The testing pyramid is a concept used to describe the strategy for automated software testing. It's particularly relevant in the context of microservices due to the complex nature of these architectures.

It provides a structured approach to ensure that individual services and the entire system function as intended. Given the decentralized and dynamic nature of microservices, the emphasis on automated and comprehensive testing at all levels - unit, integration, and end-to-end - is more critical than ever.

The Layers of the Testing Pyramid in Microservices

a. Unit Testing (Bottom Layer):

In microservices, unit testing involves testing the smallest parts of an application independently, such as functions or methods. It ensures that each component of a microservice functions correctly in isolation, which is crucial in a distributed system where each service must reliably perform its specific tasks.

Developers write these tests during the coding phase, using mock objects to simulate interactions with other components.

b. Integration Testing (Middle Layer):

This layer tests the interaction between different components within a microservice and between different microservices. Since microservices often rely on APIs for communication, integration testing is vital to ensure that services interact seamlessly and data flows correctly across system boundaries.

Tests can include API contract testing, database integration testing, and testing of client-service interactions.

c. End-to-End Testing (Top Layer):

This involves testing the entire application from start to finish, ensuring that the whole system meets the business requirements. It’s crucial for verifying the system's overall behavior, especially in complex microservices architectures where multiple services must work together harmoniously.

Automated end-to-end tests simulate real user scenarios and are typically run in an environment that mimics production.

The Problem with Testing Pyramid

The testing pyramid provides a foundational structure, but its application in microservices requires adjustments. Since the distributed and independently deployable nature of this multi-repo systems presents challenges while adopting the testing pyramid.

👉The Problem with End to End tests

• Extremely difficult to write, maintain and update: An E2E test that actually invokes the inter service communication like a real user would catch this issue. But cost of catching this issue with a test that could involve many services would be very high, given the time and effort spent creating it.

• The inter-service communication in microservices architectures introduces complexity, making it difficult to trace issues.

• Ensuring that test data is consistent across different services and test stages.

  
  

👉The Problem with Unit tests

• The issue of mocks: Mocks are not trustworthy, specially those that devs write themselves. Static mocks that are not updated to account for changing responses could still miss the error.

• Replicating the production environment for testing can be challenging due to the distributed nature of microservices.

  
  

For microservices, the interdependencies between services mean integration testing becomes significantly more critical. Ensuring that independently developed services interact correctly requires a proportionally larger emphasis on integration testing than what the traditional pyramid suggests. So a balanced approach with a stronger emphasis on integration and contract testing, while streamlining unit and end-to-end testing, is essential to address the specific needs of microservices architectures.

Why Testing Microservices is a Challenge?

This bring us to the main topic of our article, why testing microservices is a challenge in itself. We have now understood where the testing pyramid approach lacks and how it needs some adjustments to fit into the microservices system.

Testing multi-repo system need a completely different mindset and strategy. This testing strategy should align with the philosophy of running a multi-repo system i.e. test services at the same pace at which are they are developed or updated.

Multi-repo systems have a complex web of interconnected communications between various micro-services.

1. Complex Service Interactions: Microservices operate in a distributed environment where services communicate over the network. Testing these interactions is challenging because it requires a comprehensive understanding of the service dependencies and communication protocols. Ensuring that each service correctly interprets and responds to requests from other services is critical for system reliability.

   
   

2. Diverse Technology Stacks: Microservices often use different technology stacks, which can include various programming languages, databases, and third-party services. This diversity makes it difficult to establish a standardized testing approach.

   
   

3. Isolation vs. Integration Testing: Balancing between isolated service tests (testing a service in a vacuum) and integration tests (testing the interactions between services) is a key challenge. Isolation testing doesn’t capture the complexities of real-world interactions, while integration testing can be complex and time-consuming to set up and maintain.

   
   

4. Dynamic and Scalable Environments: Microservices are designed to be scalable and are often deployed in dynamic environments like Kubernetes. This means that the number of instances of a service can change rapidly, complicating the testing process.

   
   

5. Data Consistency and State Management: Each microservice may manage its own data, leading to challenges in maintaining data consistency and state across the system. Testing must account for various data states and ensure that transactions are handled correctly, especially in distributed scenarios where services might fail or become temporarily unavailable.

   
   

6. Configuration and Environment Management: Microservices often rely on external configuration and environment variables. Testing must ensure that services behave correctly across different environments (development, staging, production) and that configuration changes do not lead to unexpected behaviors.

The Right Approach To Test Microservices

We are now presenting an approach that is tailor-made to fit your microservices architecture. As we’ve discussed above, a strategy that tests integrations and the contracts between the services is an ideal solution to testing microservices.

Let’s take an example to understand better:

Let's consider a simplified scenario involving an application with two interconnected services: a Billing service and a User service. The Billing service is responsible for creating invoices for payments and, to do so, it regularly requests user details from the User service.

Here's how the interaction works: When the Billing service needs to generate an invoice, it sends a request to the User service. The User service then executes a method called <GetUser> and sends back all the necessary user details to the Billing service.

Imagine a situation where the User service makes a seemingly minor change, such as renaming an identifier from User to Users. While this change appears small, it can have significant consequences. Since the Billing service expects the identifier to be User, this alteration disrupts the established data exchange pattern. The Billing service, not recognizing the new identifier Users, can no longer process the response correctly.

This issue exemplifies a "breaking change" in the API contract. The API contract is the set of rules and expectations about the data shared between services. Any modification in this contract by the provider service (in this case, the User service) can adversely affect the dependent service (here, the Billing service).

In the worst-case scenario, if the Billing service is deployed in a live production environment without being adapted to handle the new response format from the User service, it could fail entirely. This failure would not only disrupt the service but also potentially cause a negative user experience, as the Billing service could crash or malfunction while users are interacting with it.

Testing Microservices the HyperTest Way

Integration tests that tests contracts [+data]:

Testing Each Service Individually for Contracts:

In our example the consumer service can be saved from failure using simple contracts tests that mock all dependencies like downstreams and db for the consumer.

• Verifying (testing) integrations between consumer and provider by mocking each other i.e. mocking the response of the provider when testing the consumer, and similarly when testing the provider mocking of the outgoing requests from the consumer.

• But changing request / response schema makes the mocks of either of the services update real-time, making their contract tests valid and reliable for every run.

• This service level isolation helps test every service without needing others up and running at the same time.

  
  

Service level contract tests are much simple to maintain than E2E and unit tests, but test maintenance is still there and this approach is not completely without effort.

Build Integration Tests for Every Service using Network Traffic

If teams find it difficult to build tests that generate response from a service with pre-defined inputs, there is a simple way to test services one at a time using HyperTest Record and Replay mode.

We at HyperTest have developed just this and this approach will change the way you test your microservices, reducing all the efforts and testing time you spend on ideating and writing tests for your services, only to see them fail in production.

• If teams want to test integration between services, HyperTest sits on top of each service and monitors all the incoming traffic for the service under test [SUT].

• Like in our example, HyperTest will capture all the incoming requests, responses and downstream data for the service under test (SUT). This is Record mode of HyperTest.

• This happens 24x7 and helps HyperTest builds context of the possible API requests or inputs that can be made to the service under test i.e. user service.

• HyperTest then tests the SUT by replaying all the requests it captured using its CLI in the Test Mode.

• These requests that are replayed have their downstream and database calls mocked (captured during the record mode). The response so generated for the SUT (X'') is then compared with the response captured in the Record Mode (X').

Once these responses are compared, any deviation is reported as regression. A HyperTest SDK sitting on the down stream updates the mocks of the SUT, with its changing response eliminating the problem of static mocks that misses failures.

HyperTest updates all mocks for the SUT regularly by monitoring the changing response of the down streams / dependent services

Advantages of Testing Microservices this way

1. Automated Service-Level Test Creation: Service level tests are easy to build and maintain. HyperTest builds or generates these tests in a completely automatically using application traffic.

   
   

2. Dynamic Response Adaptation: Any change in the response of the provider service updates the mocks of the consumer keeping its tests reliable and functional all the time.

   
   

3. Confidence in Production Deployment: With HyperTest, developers gain the assurance that their service will function as expected in the production environment. This confidence comes from the comprehensive and automated testing that HyperTest provides, significantly reducing the risk of failures post-deployment.

   
   

4. True Shift-Left Testing: HyperTest embodies the principle of shift-left testing by building integration tests directly from network data. It further reinforces this approach by automatically testing new builds with every merge request, ensuring that any issues are detected and addressed early in the development process.

   
   

5. Ease of Execution: Executing these tests is straightforward. The contract tests, inclusive of data, can be seamlessly integrated and triggered within the CI/CD pipeline, streamlining the testing process.

HyperTest has already been instrumental in enhancing the testing processes for companies like Nykaa, Shiprocket, Porter, and Urban Company, proving its efficacy in diverse environments.

Witness firsthand how HyperTest can bring efficiency and reliability to your development and testing workflows.

Schedule your demo now to see HyperTest in action and join the ranks of these successful companies.