We Analyzed 10,000,000 Pages and Here’s Where Most Fail with ADA and WCAG 2.1 Compliance

It’s no secret that the topic of web accessibility is on a meteoric rise. Thousands of lawsuits and tens of thousands of demand letters have been served to small business owners (but not just them, see Dominos’ supreme court ruling), and there was a 143% year-over-year increase in ADA lawsuits in 2022. 

So, as the topic’s popularity increases exponentially, we, as industry leaders, are morally obligated to contribute as much data as we can, to help drive the field forward. 

During December 2019, accessiBe scanned, analyzed, and remediated tens of thousands of websites. Some of these websites were from clients of ours and others were websites we scan, analyze, and remediate as part of training our AI.

All in all, we hit 10,000,000 analyzed pages, and we want to spread some knowledge and share some of that data!

Here’s how the research breaks down:

~85% of websites were hosted in the U.S. & Canada, 15% in Europe and Asia
~65% of websites were top-level domains (com, net, org, etc.) the others were second and third-level (io, co, app, XYZ, etc.)
~80% of websites had fewer than 1,000 pages
~19% of websites had 1,000 – 100,000 pages
~1% of websites had over 100,000 pages (We stopped counting a website's pages after it reached 100,000)

Please note: we gave sites an “easier time” and didn’t always include all WCAG failures. If a fail was somewhat minor, and it did not repeat itself, we didn’t include it in the results.

For example, if a “Shopping Cart” icon did not include an aria label/text and a proper role, we failed it. But if it appeared 3 times and failed once in the footer, we passed it. 

We were looking for actual, practical accessibility compliance that makes websites usable with the keyboard, a screen reader, other assistive technologies or tools, and of course, usable for people with various disabilities. We were not after an imaginary 100% success rate.

Also, unlike common automated testing tools (WAVE, aXe, Tenon, and others), we actually emulate a browser and use AI to determine element roles. This means, that if a form or an image is not compliant, but is also hidden and never gets into view, we don’t fail it.

Here we go:

Below, you’ll find a chart concluding data collected from 10,000,000 web pages. The data breaks down some of the most important elements for achieving compliance. 

The bar graph below shows the percentage of all pages scanned that failed to comply with the requirements for menus, images, pop-ups, and so on, you get it!

Important note: Before we dive deeper and explain each category, please note that we will not always explain why a certain category or element needs to apply this or that behavior or attribute, only the requirements themselves. This is so the article will have an end :)

Let’s dig deeper and break this down into categories.

WCAG 2.1 AA - Compliance Level of Menus:

How many websites failed:? 98% - almost none passed, even websites that did work on their menu’s accessibility manually.

What is required by WCAG: 

1. A “NAV” tag or a “role” attribute equal to “navigation/menu/menubar” (depending on the menu type) must be present on the top element that contains all the links and menu items (role=”navigation/menu/menubar”).
2. Users can use the Tab key to navigate to the next element, and Shift+Tab to navigate to the previous element, and the focused element must be easily identifiable using a focus ring (outline).
3. Users can navigate across the menu bar itself using the left and right keyboard arrows. When reaching the end of the menu, and pressing the forward arrow key, the navigation should loop back to the first item. (not a set-in-stone WCAG requirement but is a best practice).
4. Users can open dropdowns using the Enter and the arrow-down keys. Dropdowns should also be opened by focusing on the menu item.
5. Users can navigate within dropdowns using the up-and-down arrows, and the focus must never escape and loop within the dropdown unless it is intentionally closed. (not a set-in-stone WCAG requirement but is a best practice).
6. Users can close the dropdown using the Esc key, and the keyboard focus must go back to the root menu item of this dropdown. (not a set-in-stone WCAG requirement but is a best practice).

Important note relating to the requirements: We know some of the requirements for menus seem like nit-picking, and this is true for some areas of the WCAG, but specifically with menus, they are incredibly important. 

Think about a user with a mobility disability and their clicking stick in their mouth, trying to navigate dozens or hundreds of links and drop-downs one by one with the Tab key, without being able to skip using the arrows, or close dropdowns with Esc. It’ll take the user a few good and long minutes just to find what they are looking for.

Our findings: We found that almost none of the websites implemented all of the requirements relating to menus, and that’s a big problem. 

Some enabled navigation using the Tab key, some even took it a step further and implemented Tab navigation in dropdowns, but almost none allowed users to close dropdowns with Esc or implemented proper arrow-key navigation. Also, proper role attributes were rarely found.

We did find out that popular templates of some of the popular CMSs such as WordPress, Shopify, Joomla, Magento, BigCommerce, Squarespace, and others, sometimes included built-in accessibility features relating to menus, but not nearly enough.

Here's how it breaks down:

WCAG 2.1 AA - Compliance Level of images:

How many websites failed? 52% (48% passed).

What is required by WCAG: All images must have an Alt attribute (some people call it a tag although it’s really an attribute), that properly describes the objects in the image, and if the image contains texts (like typical banners) then the embedded text must also be present in the alt attribute.

Our findings: This result took us by surprise. Images in 48% of web pages, almost half, had proper Alt attributes! We expected a much lower result. We don’t know if this was done for SEO or for accessibility, but that’s good either way. 

Some websites tried to be “smart” and made the file name, the alt attribute. Others were even “smarter” and removed the “–” and ”_” symbols from the file name to create what looks like a normal sentence. Unfortunately, this doesn’t pass. Nearby titles, auto-generated strings, and other manipulations also don’t pass.

To test for “proper” Alt attributes, we used both our OCR and IRIS technologies to match words included in the provided Alt, with objects and words actually present in the images.

WCAG 2.1 AA - Compliance Level of Popups:

How many websites failed? 89% - Very few passed, even websites using “accessible” popup plugins and services.

What is required by WCAG:

1. When a popup appears, the keyboard focus must lift off the main page and land back down within the first clickable element of the popup.
2. Navigating using the Tab key within the popup must loop back to start when reaching the last element and trying to move forward.
3. Users must be able to close popups with the Esc key, and the focus must go back to the element that was focused on prior to the popup.
4. Popups must include a “role” attribute equal to “dialog”.
5. Popups must include an “aria-modal” attribute equal to “true” 

Our findings: most websites use some form of a popup plugin, either a raw jQuery that’s embedded within the template like jQuery UI Dialog and Fancy Box, a CMS plugin like OptinMonster, or a third-party service like Zoom Analytics. Some of these services implement some accessibility requirements, but most have none. Either way, this means that even those who do think of accessibility usually make do with just the basics.

The big problem with popups is that once they don’t comply, they break the ENTIRE compliance level of the site. When a popup jumps, it literally blocks the entire screen, making keyboard users unable to do anything about it. Screen reader users won’t even know a popup appeared. Complying with popup requirements is THAT important.

Please note: although a good number of popups implemented Esc closing, a very small number actually returned the focus to the previously focused element properly.

Here’s how it breaks down:

WCAG 2.1 AA - Compliance Level of Forms:

How many websites failed? 71% failed - 29% passed due to a fairly good job some CMS plugins do in this regard.

What is required by WCAG:

1. All fields must include a “LABEL” tag that is connected to the field by the “id” and the “for” attributes, or an “aria-label” attribute.
2. Required fields must include both visual cues (Asterix*, text, or other), and the “aria-required” attribute equals true.
3. Fields must include the “aria-invalid” attribute to inform screen-readers whether the field is currently valid or invalid. This attribute must change dynamically according to the validations. E.g. an empty required “name” field must include aria-invalid=”true” to indicate that it’s invalid, but change to aria-invalid=”false” once the user fills it up.
4. When a form is submitted and errors are present, the keyboard focus must be taken to the first invalid field, and the user must receive an explanation (both visual and to the screen reader) of what the issue is with this field.
5. When a form is submitted successfully, a blind user with a screen reader should be informed of that using an alert element or via other means.

Our findings: we found that often in ready-made, closed systems like Shopify, Volusion, and BigCommerce, either the platform or its popular form plugins do a fairly good job of ensuring forms are accessible.

The situation is very different though with most other open-source platforms such as WordPress, Magento, and Joomla. These systems are side-open to changes and overrides, and although this is good for website managers, this is bad for accessibility. Website managers usually don’t apply the necessary compliance requirements manually.

Additionally, we also found that even if forms were made accessible, and most fields were properly described, very few actually reported to a screen reader a successful submission. Some did report a failed submission, but mostly because forms often use the built-in HTML5 validations (that are compliant by default) without even knowing they did it.

WCAG 2.1 AA - Compliance Level of Buttons:

How many websites failed? 83% (17% passed)

What is required by WCAG:

1. Must contain text, title, or an aria label.
2. Must be an actual “BUTTON” tag or alternatively, a “role” attribute that equals to “button” present.
3. Buttons must include text/aria-label/title.

Our findings: While most buttons do conform to clauses 1 and 3, the vast majority are not being tagged properly!

Most websites use “SPAN”, “DIV” or “A” (link) HTML tags, for creating buttons (without using role=button as a fallback). Developers change how these tags look by default using CSS, and their behavior using JavaScript click events.

Doing this prevents the “fake” buttons from being properly read by screen readers, and this also means they are not clickable using the keyboard by default.

In our research, almost every website included such “fake” buttons. Standard “BUTTON” tags were used almost only with forms, very rarely as standalone buttons, and very few used the “role” attribute as a fallback for screen readers.

WCAG 2.1 AA - Compliance Level of Links:

How many websites failed: 22% (78% passed with a big caveat)

What is required by WCAG:

1. Must contain text, title, or an aria-label.
2. Must be logically ordered within the document (a “read more” must come after the title and the paragraph of a section, for example).
3. Links must be reachable by keyboard navigation using the Tab key.
4. Links must provide a visual indicator if they are opened in a new window, and also announce that to a screen reader using a hidden text or title.
5. Links must be noticeable on-page and look different than regular text.

Our findings: In our research, most websites did pass the links test. We are not sure if we’re surprised by this or not, because almost all the requirements are a default browser behavior (or just make tons of sense), so there isn’t too much work needed to be done for link compliance.

Having said that: almost none of the links actually indicated is opened. We ended up passing websites anyway because this is not a HUGE compliance-breaking issue, and also, if we did, we’d have to fail almost all of them, despite being usable.

Additionally, there were a good number of links that did not contain text, titles, or aria-labels, but just an icon, like social media links. In our case, we did not address these as links but as icons and our research includes a designated category for them.

WCAG 2.1 AA - Compliance Level of Icons:

How many websites failed: 76% (24% passed).

What is required by WCAG: Icons don’t really have much of a guideline, because there isn’t such a thing as an “ICON” tag or element. Icons are usually being used with links or buttons, to describe some kind of action. So, the requirements for making icons accessible are pretty much the same as making links and buttons compliant.

Background: You must be asking yourself “If there aren’t specific requirements, why did you include an entire category for icons?” Well, this is because icons appear in almost 100% of sites, and even though they don’t have their own functionality, developers often use them differently than buttons and links.

To explain this, we first need to establish what icons are most often used for: Icons usually exist inside of a button or a link, and aim to describe that button or link, to people that see, and can recognize these “signs”. 

For example, many E-Commerce websites use a shopping cart icon without any text explaining that clicking it, triggers open the shopping cart. Other uses can be social media logos without text, favorites/wishlists (heart icon), search buttons (magnifying glass), product ratings (star icons), and many other instances.

Our findings: most icons are being used as “SVG” elements, font icons (like Font Awesome or IcoMoon), or background images (as Sprite or standalone).

While this is perfectly fine for people who see, neither of these practices is helpful for people using screen readers (blind people). To make these icons accessible, developers will have to include a hidden text or an aria-label attribute, describing what this icon is. For example, a Facebook icon will have to have a hidden text or an aria label equal to “Facebook”.

In our research, we found that most icons are not being described at all, especially font icons. Having said that, 24% of the web pages did pass, due to some popular icon libraries implementing descriptions built into to their SVG elements. Other font-icon CMS plugins also do something similar but on a different type of element (usually an “I” tag).

Other important findings

There are other important accessibility requirements websites must include to be compliant, but don’t belong in any of the above categories. 

Note: we did not include ALL compliance requirements, but only the most noticeable ones, that have an impact on the compliance level of most or all websites.

• HTML lang attribute: The “HTML” tag must include a “lang” attribute equal to the language of the document. For example, “EN” for English and “FR” for French.
• Skip Links: The top of every web page needs to include visually hidden buttons (that appear on focus) and allow users to skip to different sections within the page.
• Incorrect use of ARIA: Many websites include faulty usage of ARIA attributes, usually faulty "aria-labelledby" and "aria-describedby" that are being automatically populated by various form plugins.
• Tabbing in hidden elements: Web pages often include hidden elements that appear in certain conditions such as popups, sidebars, and dropdowns. These elements must not be navigable if not visually active on the screen.
• Weak/non-existent focus ring: Every clickable element must include a visual, noticeable focus ring (outline) when being focused during Tab navigation.

Final Takeaways

Phew. We are finally done! 

This is what we learned about the WCAG 2.1 compliance level of the web, after analyzing 10,000,000 pages. Truth be told? The internet is (or was, prior to us), has been in a very, very bad compliance shape since 2019.  

As seen from the results, oftentimes plugin and template developers do try to conform to certain requirements, but these are often not even close to sufficient. 

Moreover, at the end of the day, these plugins and templates are being used by website owners who usually don’t know anything about coding, best practices, or accessibility. They simply have a system and click the buttons to achieve what they want as quickly as possible without thinking about what is and what isn’t accessible. Thus website owners end up ruining their own compliance very quickly with each update they make.

Accessibility is not something that is in most business owners’ priorities. Marketing, hiring, customer service, and sales are (obviously). It’s our job, as developers and designers, to conceive and achieve technological breakthroughs to make websites compliant despite the topic not being a priority to business owners because it never will be! This is on us.