Accessibility confidence testing

What is confidence testing?

Confidence testing (also known as “smoke testing”) is a preliminary form of testing that checks for basic failures.

Caveats

Automated accessibility confidence testing is not comprehensive. Accessibility testing automation is often cited to cover approximately 30-50% of accessibility scenarios and requirements, and is thus not a replacement for manual accessibility testing by third parties.

Manual accessibility confidence testing is also usually not comprehensive. It’s best to perform a thorough audit beyond the scope of initial confidence tests.

Confidence testing doesn’t guarantee that everything is right, it only tells you that something is probably wrong. But that’s still very useful information! So, what is accessibility confidence testing good at?

Benefits

For that 30-50% of accessibility errors we can use a tool to scan for, confidence testing works great! Confidence testing allows us to improve the baseline quality of our software, documentation, and websites. It helps us develop expertise as we fix the issues we find. It also helps us reduce regressions in the same way that most tests do, by checking for them in future versions of our code.

Manual accessibility confidence testing

Manual confidence testing uses a tool or process to test for errors. These tools are usually browser plugins or websites that can scan a particular web page and check it for common errors. Manual testing works well for things that don’t change often, or for in-progress work, such as: web pages that almost never change or static websites that have consistent semantic markup, but many occasionally change content.

Here are a few examples of manual accessibility testing tools:

• Lighthouse
• Axe devTools
• Web Accessibility Evaluation Tools List

Automated accessibility confidence testing

Automated confidence testing involves tools that run against web pages automatically, usually in a Continuous Integration (CI) pipeline. Setting them up requires a bit more initial investment, but the benefits are often worth the effort. Automated accessibility tests work well for things that change structure or functionality fairly often and things that already have a CI build and deploy process.

Here are a few examples of automated accessibility testing tools:

• jest-axe
• cypress-audit
• BrowserStack Automate

Test your websites

Implementing a recurring scan of web pages, even with manual testing tools, can help to improve accessibility across the board. Give some of these tools a try on websites and applications that you work on, and see what they find!