💡 I am a senior SWE at a large tech company. I started on a legacy tool that has multiple repos (frontend, backend, and some other services) and has no automated interface testing. We do have a QA team that runs through scenarios, but that's error prone and expensive. In reality, a lot of our defects are encountered by users in prod (sad).
I have had mixed experience with e2e testing in the past: challenging to maintain with a lot of moving parts, sometimes flakey, challenging to coordinate between repos moving at different speeds.Relatable, right?
That’s the story of every other SWE!
E2E tests are good only when they’re working correctly and navigating the actual failure instead of any other broken parts.
But this is not a guide to tell the reality-check of E2E tests. INSTEAD, this guide is here to tell you everything about End-to-end tests. We’re not going to discuss only about the positive and how-to sides of this top segment of the testing pyramid.
This is more like a raw version of everything E2E, which talks both about the + and - of it.
End-to-End testing: The most debatable part of the pyramid
E2E tests are the ones that tests your whole app in one go. Interesting?
Superficially, yes! But when you dig deep and gain more knowledge on this, you’ll also find yourself debating about keeping it or leaving it within your team.
A formal introduction on E2E tests⬇️:
End-to-end testing is a method of testing the entire software application from start to finish. The goal is to validate the system as a whole and ensure that all integrated components work together as expected. E2E testing simulates real user scenarios to identify any issues with the system's interaction and data integrity across different layers.
Basically, E2E tests can test your system under two conditions:
all your services, databases and other dependent components need to be kept up and running: Simulating a live scenario
mocking or stubbing any external dependencies as per your convenience to allow for controlled and repeatable testing
Why is end-to-end testing important?
1. Covers the Entire Application:
End-to-end testing checks the entire flow of an application from start to finish. It ensures all parts of the system work together as expected, from the user interface down to the database and network communications.
2. Detects System-Level Issues:
E2E helps identify issues that might not be caught during unit testing or integration testing, such as problems with data integrity, software interactions, and overall system behavior.
3. Mimics Real User Scenarios:
It simulates real user experiences to ensure the application behaves correctly in real-world usage scenarios. This helps catch unexpected errors and improves user satisfaction.
The Benefits of End-to-End testing
In an ideal scenario when E2E test is running smoothly and finding the right defects and bugs, it has potential to offer tremendous help:
E2E can often be the most straightforward or apparent way to add testing to an existing codebase that’s missing tests.
When it's working, it gives a lot of confidence when you have your most important use cases covered.
E2E tests for basic sanity checks (i.e. just that a site loads and the page isn’t blank) is very useful and are always good to have.
Key Components of End-to-End Testing
💡 Yes, E2E can be flaky, yes they can be annoying to keep up to date, yes their behavior can be harder to define, yes they can be harder to precisely repro. However, they can test behavior which you can't test otherwise.
Steps in End-to-End Testing
💡 Writing a couple of e2e (UI) tests is ok though. The key is to not overdo it. E2E tests are really complex to maintain.Requirement Analysis: Understand the application requirements and user workflows.
Test Planning: Define the scope, objectives, and approach for E2E testing.
Test Design: Create detailed test scenarios and test cases.
Test Environment Setup: Prepare the test environment to mimic production.
Test Execution: Run the test scenarios using automated tools.
Test Reporting: Document the results and identify any defects.
Defect Retesting: Fix and retest any identified defects.
Regression Testing: Ensure new changes do not affect existing functionality.
Example of End-to-End Testing
Consider an application with a user authentication system. An E2E test scenario might include:
User Signup: Navigate to the signup page, fill out the form, and submit.
Form Submission: Submit the signup form with user details.
Authentication: Verify the authentication process using the provided credentials.
Account Creation: Ensure the account is created and stored in the database.
Login Service: Log in with the newly created account and verify access.
Types of End-to-End Testing
There are two types of End-to-End Testing: Vertical E2E testing and horizontal E2E testing. Each type serves a different purpose and approach. Let’s have a quick look at both:
⏩Vertical E2E Testing
Vertical E2E testing focuses on testing a complete transaction or process within a single application. This type of testing ensures that all the layers of the application work together correctly. It covers the user interface (UI), backend services, databases, and any integrated systems.
Example:
Consider a banking application where a user transfers money. Vertical E2E testing would cover:
User logs into the application.
User navigates to the transfer money section.
User enters transfer details and submits the request.
The system processes the transfer.
The transfer details are updated in the user’s account.
⏩Horizontal E2E Testing
Horizontal E2E testing spans multiple applications or systems to ensure that they work together as expected. This type of testing is important for integrated systems where different applications interact with each other.
Example:
Consider an e-commerce platform with multiple integrated systems. Horizontal E2E testing would cover:
User adds a product to the shopping cart on the website.
The cart service communicates with the inventory service to check stock availability.
The payment service processes the user's payment.
The order service confirms the order and updates the inventory.
The shipping service arranges for the product delivery.
Best Practices for End-to-End Testing
Implementing E2E testing effectively requires following best practices to ensure thorough and reliable tests. Here are some key practices to consider:
E2E tests hitting API endpoints tend to be more useful than hitting a website. This is because it tends to break less, be more reliable, and easier to maintain.
Focus on the most important user journeys. Prioritize scenarios that are critical to the business and have a high impact on users.
E2E tests on an existing codebase often requires ALOT of test setup, and that can be very fragile. If E2E testing takes a lot of work to get setup, then chances it will become easily broken, as people develop. This will become a constant burden on development time.
If your E2E tests aren’t automated and lots of manual steps to run them. Then they won’t get used, and development will be painful. Ideally you’d want to be able to run your tests with 1 or 2 direct commands with everything automated.
Set up a test environment that mimics production but is isolated from it. This prevents tests from affecting live data and services.
If your E2E tests have any unreliability, then they will be ignored by developers on the build system. If they aren’t actively worked on, they will eventually get disabled.
Test data should be as close to real-world data as possible. This helps in identifying issues that users might face in a production environment.
💡 Eliminate the problem of test-data preparation while performing E2E tests cases, ask us howRegularly update and maintain test scripts to reflect changes in the application. This ensures that the tests remain relevant and effective.
If your E2E tests take longer to write and run than unit tests, than they will become unmaintainable.
By following these best practices, you can ensure that your E2E testing is thorough, efficient, and effective in identifying and resolving issues before they reach your users.
Challenges with End to End Testing
Well, here’s the part finally why I narrowed down to write on this topic.
Since I started on a negative side about E2E testing and then continued with all the positives and how-to things, I assume you might be confused by this time?
Whether E2E testing is a good practice to invest in or should the fast moving teams of today should leave it as it is?
Here’s the breakdown of some challenges that are worth to talk about before you make your decision to go ahead with E2E testing.
Extremely difficult to write, maintain and update.
While End-to-End (E2E) tests mimicking real user interaction can expose integration issues between services, the cost of creating and maintaining such tests, especially for complex systems with numerous services, can be very high due to the time and effort involved.
imprecise because they've such a broad scope
needs the entire system up & running, making it slower and difficult to identify the error initiation point
E2E testing might be overkill for this minor issue{user}→{users}. It requires all services to be operational, and even then, there's a chance it might not pinpoint the exact cause. It could potentially flag unrelated, less critical problems instead.
Tools for End-to-End Testing
⏩HyperTest
Before we start, we don’t do E2E testing! But we are capable of providing the same outcomes as you expect from an E2E test suite.
We perform integration testing that covers all the possible end-to-end scenario’s in your application.
HyperTest captures real interactions between code and external components using actual application traffic, then converted into integration tests.
TESTS INTEGRATION SCENARIOS
It verifies data and contracts across all database, 3rd party API calls and events.
SMART TESTING
HyperTest mocks external components and auto-refreshes mocks when dependencies change behavior.
RUN WITH CI OR LOCAL
These tests can be run locally or with CI pipeline much like unit tests.
⏩Selenium
A popular open-source tool for automating web browsers. It supports multiple programming languages and browser environments.
⏩Cypress
A modern E2E testing tool built for the web. It provides fast, reliable testing for anything that runs in a browser.
⏩Katalon Studio
An all-in-one automation solution for web, API, mobile, and desktop applications. It simplifies E2E testing with a user-friendly interface.
⏩Testim
An AI-powered E2E testing tool that helps create, execute, and maintain automated tests.
Conclusion
While E2E tests offer comprehensive system checks, they're not ideal for pinpointing specific issues. They can be unreliable (flaky), resource-intensive, and time-consuming. Therefore, focus on creating a minimal set of E2E tests.
Their true value lies in exposing gaps in your existing testing strategy. Ideally, any legitimate E2E failures should be replicated by more focused unit or integration tests. Try HyperTest for that.
Here's a quick guideline:
Minimize: Create only the essential E2E tests.
Maximize frequency: Run them often for early error detection.
Refine over time: Convert E2E tests to more targeted unit/integration tests or monitoring checks whenever possible.
Related to Integration Testing
