What does it do?

End to end automation including UI and API

Complete backend testing: APIs, database calls, message queues & inter-service contracts

Who are its users?

Will not test database calls, async flows or message queues.

Only Developers

What will it not do?

SDETs, test and QA engineers

Front-end testing: will not test UI or test across browsers or devices

How to start?

Katalon Studio. Provides a comprehensive IDE for writing, recording, and executing tests

10-line SDK in the source code of repo. Records traffic and builds tests which can be replayed later using CLI

How does it work?

Recording and Scripting: Users can record their actions on web or mobile applications, generating automated test scripts. They can also write custom scripts using Groovy

Record and Replay: Monitors application traffic using its SDK to generate backend tests automatically. 100% autonomous.

Where does it run tests?

Katalon Cloud. Needs dedicated and test isolated environments with SUT and dependencies available

Environment Agnostic. No dedicated or isolated environments needed. These tests can be run locally 100% local or before push-commit in CI

Maintenance

Manual. Write and update tests using Katalon Studio manually

100% autonomous or no-code. Automatically generates tests by recording actual user flows. Auto updates all test cases and assertions as APIs change

Quality of Tests

Poor. Depends on quality of assertions which are manually written

High. Quality programmatically generated assertions that cover schema & data to never miss errors

Test Coverage

Unknown. No way to measure test coverage that can result in poor coverage and untested scenarios

Measurable. Reports code coverage i.e. actual lines tested lines of code leaving behind no untested scenario

Test Data Management

Yes. users need to use data sheets or custom scripts to seed and manipulate data for tests. Can turn out to be non-trivial

No. HyperTest uses data from traffic for tests and keeps it reusable. Handles both read & write requests

Test Execution Speed

High. API test: Depends on response time of API and run environment End to end test: Take longer to run

Negligible. Runs as fast as unit tests. Does not need dedicated environments but are still able to test e2e backend flows

Can it test databases?

No

Yes

Can it test message queues?

No

Yes