Despite their best efforts, QA teams struggle to keep applications bug-free because they spend their entire time writing and maintaining tests that fail to cover every user-flow in the product. Result, workflows outside these tests break, generating new bugs all the time.
Bu the record & replay approach used by HyperTest can monitor every user interaction to automatically build integration tests without asking teams to write a single line of code, catching every possible API regression i.e. logical, functional, and performance failures. This is how HyperTest makes it happen.
1. Record Application Traffic
This is the first and easiest step. HyperTest records all the incoming http traffic of any application by integrating with any ingress controller like load balancers or reverse-proxies.
Check the detailed Set-up doc here.
This traffic is mirrored by HyperTest asynchronously and is completely agent-less. This mirrored traffic are the user-flows that are recorded as-is by HyperTest. They are separated into distinct user sessions (or test cases) using configurable identifiers like authentication cookies, headers etc. Recording a user session that can be used as a test case is very simple.
HyperTest records hundreds or thousands of such different flows to build a powerful regression suite in minutes.
Each recorded session in HyperTest is a user scenario or flow. It captures the exact sequence of steps executed by the user (or a tester) when he interacts with the application. It hence removes the need to physically write a test script for several such scenarios that need to be tested, because of all them can be captured by just recording from the network traffic.
All such sessions are intelligently segregated to have unique flows captured for every user or user type.
So the recorded session is the sequence of all API calls called on the application exactly in the order that an actual user would have called.
But at the time of monitoring only request payload i.e. headers, query parameters and body, are recorded and not the API responses.
In summary to make HyperTest work in the record mode:
Integrate HyperTest with your load balance or reverse proxy
Set-up HyperTest binary on any VM that props up the dashboard
Configure HyperTest to segregate user sessions on some unique user identifier
Record mode is on
2. Replay Traffic to test a new build
The integration tests are ready, segregated by flows and ready to test a new build. These are then replayed (or run as a test suite) on two versions of your application at the same time.
One of the version is the production commit (master branch), and the other the new build or test version both deployed in the same environment. Both the versions are connected to the same database and downstream dependencies.
When QA teams write their own test scripts, they write assertions that validate response codes, schema and data. Since these assertions are hand-written, teams could possibly miss asserting every business rule. Even if they somehow put in all assertions, they need to be updated everytime the application changes.
HyperTest eliminates the requirement to think and physically put in all assertions by comparing the responses of the version under test with the last releases master that is considered the source of truth, during the replay mode.
These responses are generated from both versions at the same time, both talking to the same data base and dependencies. When compared, any deviations between the 2 versions is a possible breaking change in the version under test.
Also all these assertions are put in place by HyperTest in real-time at the point these tests are executed removing the need to think and physically put in any assertions. Also as soon as the test version is merged to master, the reference branch changes and all the test cases are instantaneously updated.
3. Handling (write) requests that cannot be replayed
Replay of traffic breaks in case of non-idempotent requests i.e. write or POST because the state of the application has already changed for it to be tested with the same payload.
Consider testing sign-up. If a user signs-up with an email user@email.com, on replay the same email id cannot be used because the application would respond with message ‘the user already exists’. This kind of non-idempotency in requests is handled by HyperTest by modifying the request payload before executing the flow. Here in the example supplying the sign-up request with a new email id to test registration of a new user
HyperTest will bring the application to the right state when executing non-idempotent flows. Consider 2 more examples:
1️⃣ 2-factor authentication with one time password (OTP)
A user logs into your application using an OTP. The user service generates an OTP when an existing user tries to login, which the user enters at the login step to get in.
HyperTest will monitor the user logging in for the first time from the application to create the flow. But when this flow is tested during the replay stage HyperTest will do the following modifications
LOGIN: After running the login request, the application generates an OTP that HyperTest will fetch from the database and save it in session variables (temporary memory)
ENTER OTP: In the next step, where the OTP has to be entered HyperTest will use the saved OTP from session variables
HyperTest will then fetch the new access token from the response of the last call and save it again to session variables
This access token is hence passed to all subsequent request after to be the flow authenticated
2️⃣ Order creation flow
Most applications work around creating an order for any transaction of their users. Users can then do multiple other operations on these orders i.e. adding discounts, loyalty points and then eventually completing a payment.
A sample testable scenario for a flow like this is handled by HyperTest by fetching the order ID of the user from the response of the request that creates the order and then passing it to all subsequent requests hence maintaining the state of the app
ORDER CREATION: The user places the order that creates an order ID. HyperTest scans the response, fetches this ID and stores in temporarily in session variables
APPLY DISCOUNT: In the next step of the flow the user wants to apply a discount on the order. To maintain the state HyperTest passes the same order ID in the request payload of the subsequent discount request
CHECKOUT: The checkout API would expect again the order ID on which the payment is expected. HyperTest passes the same order ID stored in the session variables for the checkout API to make payment on the right order.
HyperTest maintains the state of the application across the entire flow to keep it testable.
Record and Replay approach of HyperTest is the fastest way for teams to:
Get to very high coverage within days not weeks or months
Test new features or changes early in the development cycle
Eliminate the need to write or maintain a regression suite