Drupal Takes a Step Forward in Accessibility with Automated Testing Integration
On Friday, April 7, 2023, Drupal core maintainer Lauri Eskola committed changes that were worked on in issue https://www.drupal.org/project/drupal/issues/3293469 to Drupal core (version 10.1.x).
Even in Drupal’s accessibility community, many folks will have missed the significance of this change. With this change, many accessibility errors will be caught before they can be committed to core. Drupal will be one of the first open source CMS to include axe testing as part of its CI/CD pipeline.
The Origins of Automated Accessibility Testing in Core
The history of working to implement automated accessibility checks for Core goes back to 2017, shortly after a few open source accessibility tools were released. Deque’s axe-core had been released two years earlier, and it was starting to get embedded in a variety of other systems. In 2018, NightwatchJS was integrated into Drupal core, shortly after that PreviousNext produced a prototype for an axe integration with it so that axe-core could become part of CI/CD pipeline.
As often happens with issues in open source projects, this issue was explored for a while, before a specific implementation was split off. Other open source projects were explored for both the accessibility engine and scanning. It was decided to focus on an accessibility engine that was part of Accessibility Conformance Testing (ACT). It was decided that axe provided the best tool to build from.
A list of key pages was suggested to test against, and Daniel Mundra, Associate Director of Drupal at CivicActions, started the process of incorporating axe into the existing Drupal Nightwatch testing.
A child issue, Automated A11y tests in Nightwatch, was started in 2022, and with a clear focus, the team was able to implement it. With close collaboration between CivicActions’ Daniel Mundra and Acquia’s Ben Mullins (also a Drupal Accessibility Core Maintainer) a solid patch was built and tested. Within a year it was committed to core.
What does it mean for Drupal core?
Before every commit:
- A test environment is set up using a custom install profile.
- Nightwatch runs axe tests on nodes, articles, & taxonomy pages. Additional pages will likely be added in the future.
- These pages are tested for anonymous users using the default theme:
- Home page: /
- Login: /user/login
- Search: /search/node
- These pages are tested for authenticated admin user using the admin theme
- User Edit: /user/1/edit
- Create Article: /node/1/edit
- Create Page: /node/add/page?destination=/admin/content
- Content Page: /admin/content
- Structure Page: /admin/structure
- Add content type: /admin/structure/types/add
- Add vocabulary: /admin/structure/taxonomy/add
- Structure | Block: /admin/structure/block
- And when errors happen they are displayed in the console output of the test runs.
If the tests do not pass, the change will not be accepted by Drupal. The accessibility regression will have to be addressed before the change can be committed.
Any accessibility issue that would be caught by manually running Axe on a page will also be caught by these tests. It will be important to continually evaluate if the pages being checked sufficiently reflect the use cases that needed to be accounted for when running accessibility tests. As new use cases are identified, they can be added to the test suite.
When adding the testing we already discovered a few accessibility problems on some of the pages. We have created follow-up issues to tackle them and currently disable those specific checks for the specific pages. For example in https://www.drupal.org/project/drupal/issues/3318394, we point out the problems with color-contrast, duplicate-id-active, and region that were seen on the Block UI page. Other issues are:
What does it mean for Drupal modules?
As mentioned above with the addition of Nightwatch.js in core, contributed modules have also looked at and have incorporated their own Nightwatch tests. Doing some research, we found the modules Autodrop, Performance Budget, and Quicklink include such tests for their functionality. There are definitely more modules that use it or have thought about using it specially if they have user interface features they want to test. Well in the near future when the contributed modules target core 10.1.x they can add Nightwatch based accessibility tests to their module.
This is an exciting prospect to consider. With accessibility tests, contributed modules can ensure their features are accessible and remain accessible as they build them just like they ensure features work as expected with unit and functional tests.
This will be new territory and might need additional support for the contributed modules (example issue for Enable Nightwatch Testing for Contrib), conversations on approaches to adding Nightwatch tests, and other aspects we haven’t thought about. We encourage you to follow the Nightwatch core accessibility tests and build out some for your own contributed modules. Share your success and struggles in the community, especially in the Drupal #accessibility Slack channel.
Acknowledgements
Rikki Bochow, for her post showing how accessibility testing can be done with Drupal.
Matt Glaman, for his multiple posts on how to run Nightwatch tests with ddev which helped make it easier to create these accessibility tests.
Ben Mullins, for his leadership, developments of the tests, keeping up with the discussions, moving the issue along, and turning the conceptual conversation into a working merge request.
Ivan Berdinsky, for his development of the tests and creation of the install profile.
Thank you to the reviewers Théodore Biadala, Jess, Stephen Mustgrave, and Lauri Eskola for your feedback and discussions.
Resources to check out
Watch: Nightwatch testing training to help get Olivero stable for Drupal 9.2!
Talking Drupal #382 - Automated A11y Testing with Nightwatch
Automate your Drupal accessibility testing with aXe and NightwatchJS | PreviousNext.