The Most Powerful Regression Technique And Why It Works


With APIs carrying the majority of the functional and business logic for applications, teams use a variety of open source and in-house tools for testing APIs but struggle to catch every possible error.


There is a way to catch every error, every critical regression in your APIs without writing a single line of code.


Why do existing regression techniques fail?


The hardest thing about writing API or backend tests is accurately defining the expected behavior.


With 80%+ of the web or mobile traffic powered by APIs, all new features in applications involve a corresponding update or change in relevant APIs


These changes would be of two types, desired i.e. ones that are intended, and undesired i.e. the ones that might break the application as side-effects and result in bugs. It is hardest to find these side-effects or regression issues because unless one asserts every single validation across all the APIs, new changes will break some unasserted validation, causing an unknown bug. To ensure the expected behavior of applications remains intact forever means anticipating and testing every new change, which becomes harder to impossible as the number of APIs increases and becomes more complex.


THE SOLUTION

API changes that can cause application failures would because of either

  1. Contract or schema changes

  2. Data validation issues or simply

  3. Status code failures


The best test strategy is the one that reports all changes across all updated APIs in the new build. However, as applications grow and involve more APIs, covering and testing all new changes becomes increasingly difficult. The simplest way to catch deviance from expected behavior in APIs is to compare them with the version that is stable or currently live with users

The existing version of the API or application that is currently live with users is the source of truth. Any deviance from how the application currently works (expected) is going to become a bug or problem (unexpected)


Comparing responses across the 2 versions for the same user-flow is the surest way to ensure no breaking change has happened, and the deviance in response only possibility of breaking change


16 views0 comments