Accessibility Testing Mobile Apps

What is it?#

Testing the app works well for users with accessibility needs on both Android and iOS devices. We would recommend that at least two rounds of an internal accessibility audit is carried out for both iOS and Android platforms.

This should enable us to identify the bulk of the accessibility issues that people who make use of text scaling/display scaling and screen readers could encounter.

Why do it?#

In 2011, nearly 1 in 5 people (17.9%) in England and Wales reported a disability that limited their daily activities. People living in deprived areas and working in routine occupations were more likely to be disabled, showing the inequality that exists across England and Wales.

With so many people reporting a need for some accessibility adjustments in their everyday lives it is highly likely that we will see a number of these people use the apps that we work on.

It is also of note that, in recent years more directives have been passed (and more proposed in future) in order to ensure that web and mobile services are accessible by all, like, the EU’s 2016 Web and Mobile Accessibility Directive as well as the UK’s 2010 Equality Act requires businesses to provide access to everyday services.

Both of these require digital services to meet a WCAG 2.0 AA standard. Ignoring directives like these can lead to litigation, as seen by this large list of such cases and in recent years this has to beyonce.com, Netflix, Nike being litigated.

Finally, accessibility improvements benefit everyone, not just those with accessibility needs.

How to do it#

We recommend running at least two rounds of accessibility testing throughout a private beta phase. Use a checklist based on more established internet accessibility standards such as the A11y checklist and the WCAG checklist, aim to meet WCAG 2.1 AA standards.

Aim to identify issues associated with text scaling/display sizes, and separately test to identify issues associated with using screen readers. We recommend running these tests separately as the problems and solutions are very different.

Try to focus on one discreet journey at a time so that work can be broken out into one or more sprints. Identify which bugs are affecting reused components to have a better understanding whether a fix could benefit multiple journeys.

Begin with the most important journeys like registration and login to ensure users can easily and successfully access the rest of the the app. Once you know users can get into the main app, then identify the most pressing accessibility issues, i.e. focus or keyboard traps, missing content descriptions, inaccessible elements, broken or unreadable UI etc. Once the most grievous issues have been corrected, try using an impact effort matrix to prioritise the remaining issues.

You can try mapping problems on a Miro board that shows all the screens from a discreet journey. Review the problems so that the team fully understand them and their implications and classify them using a red, amber green system. Relevant tickets can then be written and prioritised into sprints.

Once a ticket has been completed and merged into production we should then re-test. Any new accessibility UI (hints, content description, grouping etc.) will be added to designs.

You may also want to think about adding an accessibility statement to the app. This may mention our accessibility rating, accessibility features, known issues, workarounds, (legally binding) dates when we expect to have fixes in place.

When to do it#

If the project is already being developed then we should test when we can get production builds on physical devices.

When the bugs for the first discreet journey have been resolved, work can begin on testing the next section or journey within the app. This process will repeat until the entire app has been tested.

Expect there to be fewer problems identified as bugs are fixed and passed on to other areas of the app.

When the app has been fully tested and a public beta build is ready we should repeat the testing process again. As before we should expect there to be even fewer bugs to focus on.

Improving the process#

Ideally we should consider accessibility implications and requirements when defining and designing work. It is important to consider accessibility and be vigilant when developing a test for accessibility when new features are added and the UI is changed.

Accessibility is the responsibility of the whole team. Ingraining as much as we can into the whole process and considering accessibility will significantly reduce the effort required to make the app accessible. This should further reduce the bugs that are identified in future audits and eventually remove the need for internal auditing.

Possible further testing#

When we think we have addressed most of the bugs we will want to think about getting an external audit. Given that we have 2 apps we will need to find a budget for 2 tests. Previously when I have used the Digital Accessibility Centre for a website audit it was around £3000.

We may consider using:

Some of these like the Royal National Institute for the Blind can also provide observed user testing which would give us more confidence and the insight that only real user testing can provide.

We would then need some time to address any fixes. This would give us the confidence to really start advertising our accessibility credentials.