Four Tips for End-to-End Testing

I'm a big fan of end-to-end (E2E) tests. They give me confidence that my system works as a whole..

E2E tests have two goals:

1. Make sure a particular feature works well with all of its connected pieces.
2. Group features together, so one can understand what is going on in the system.

Even though we often use the same tools for writing unit and E2E tests, writing them is different. Unit tests only verify one thing and E2E tests verify multiple things (some implicitly).

A lot of developers find it hard to write and to debug E2E tests.

Here are a couple of tips to help you with writing and maintaining E2E tests.

1) Use data-test to match elements

Use data-test attributes instead of CSS selectors to find elements on pages.

// instead of this
element.find('.description button.expand-button').simulate('click');
// write this
element.find('[data-test="test"]').simulate('click');

Tests often break because the developer didn't know a particular property was used for testing. Having an explicit data-test communicates "this is needed for testing".

I have written about this in more detail before.

2) Verify after every action

In unit tests, you should follow the Four-Phase Test:

setup
action
verify
teardown

Here is an E2E example of that:

setup - create an unpublished blog post

action - visit post edit page
action - select new "published_at" date from date picker
action - submit the form
action - go to index page

verify - post's "published_at" is changed
verify - the post is shown on page

However, in E2E, if we only verify in the end, we might miss an error that happens somewhere in the middle. For example, a validation error might occur. A better approach is to do the following:

setup - create an unpublished blog post

action - visit post edit page
verify - the post is shown on the edit page

action - select new "published_at" date from date picker
verify - the correct date selected from the date picker

action - submit the form
verify - no errors and success message is shown
verify - post's "published_at" is changed

action - go to index page
verify - the post is shown on page

In this way, if the date picker fails, we will know immediately instead of having to hunt down why the unpublished post is not shown on the index page.

Many assertions are already made by your testing library of choice - like click_link "foo" failing if it can't find what it's looking for. But there might be gaps when you have custom components, and you could make your own checks for those.

3) Write helpers for custom operations

Often in E2E, you have to perform a group of related operations. It is useful to have shared helpers for those.

For example, if you have a fancy calendar picker. Every time you use it, you will have to write the following:

- click data-test="calendar-picker"
- enter the correct year in a text input
- pick a correct month from month picker
- click on the correct day
- click the "ok" button to save

It makes a lot more sense to have a single helper function, so when you change "ok" with "done", you can do it in one place.

Helpers.selectInCalendarPicker(date)

Here are some other examples for grouping operations:

Helpers.loginAs(user)
Helpers.enterCommentWith(text)
Helpers.submitCommentWith(text)
Helpers.closeModal()
Helpers.enterInForm(values)

You can hide your verify checks in those helpers.

This is essential to have a maintainable E2E test suite. Ideally, a good test should read like a story and its implementation details would be hidden.

4) Wait for Ajax

Often you have an Ajax call, followed by an assertion. Those calls take time and can lead to timeouts.

I always have a waitForAjax helper in my projects:

await Helper.waitForAjax();

Here is a sample implementation with Apollo and Cypbara:

import { ApolloLink, concat } from 'apollo-link';
import { environment } from '~/config';

let recordRequestsInFlight = (networkLink) => networkLink;

if (environment.isBrowser && environment.isTest) {
  recordRequestsInFlight = (networkLink) => {
    const recorderLink = new ApolloLink((operation, forward) => {
      window.test.apolloRequestsInFlight += 1;
      
      return forward(operation).map((result) => {
        window.test.apolloRequestsInFlight -= 1;
        return result;
      });
    });
    return concat(recorderLink, networkLink);
  };
}

// Usage
new ApolloClient({
 link: recordRequestsInFlight(
   new HttpLink(/* ... */),
 ),
 // ...
});

# this waits until there are no upcoming XHR request
def wait_for_ajax
  Timeout.timeout(Capybara.default_max_wait_time) do
    loop while page.evaluate_script('window.test.hasInFlightRequests()')
  end
rescue Timeout::Error
  # NOTE(rstankov): Some times this fails on CI while code is working
  nil
end

Code in Gist

Conclusion

Those tips are valid independently if you test with Cypbara, Cypress or XCTest or similar. Their goal is to make your tests more reliable and easy to write, so you can spend more time developing your features instead of hunting for brittle tests.

If you have any questions or comments, you can ping me on Twitter.