Companies are rushing to add generated AI capabilities to their products. Some promise to produce front-end components for you. Is that even possible, given the nature of accessibility and the nature of generative AI? And is it desirable?

The short answer is no, to both questions. The risk: that our rush to technlogical solutions comes at the expense of users.

To find out why, let's consider: how is the process of building an accessible component different between humans and machines? And what are the ethics of our tendency to reach for technological solutions?

The human approach

Let's look at the differences in process first. A human who writes accessible front-end code, writes (mostly) HTML elements and attributes based on:

• their understanding of specs and how they work together (including HTML and WAI-ARIA)
• what they intend to convey
• what they know about how assistive technologies interpet the code they write
• knowledge of browser and assistive technology support
• looking up the syntax and applying it correctly

(Leaving aside all the useful templating languages and orchestration libraries)

So they translate what they or their designer counterparts want to exist into something that works according to those intentions in a browser. Intentions are a key word here. Conveying author intentions accurately and understanding user needs is essential to accessibility.

They are likely also involved in writing CSS for things like colours, typography and spacing, which can all affect whether websites have barriers for users. And add JS for interactive stuff, managing state(s) and more.

The machine approach

A tool that generates code using language models basically predicts lines of code based on statistical likelihood, a bit like an autocomplete. If the output happens to be high quality, that's, in principal, coincidental. A systems' success rate can be (and is usually) increased by training models specifically with very good examples. In some cases, systems get very close to high quality, because they have enormous amounts of training data. For accessibility, this data is hard to get by—most of the web has accessibility problems: what we can see in the automated tests of the WebAIM Million is just the tip of the proverbial iceberg.

While humans map intentions to interactive content and apply their understanding in the process, LLMs don't have intentions or understanding. They just output blobs of text that matches some input best. I think this is fascinating, impressive and often akin to magic. And the output can look (and sometimes be) production ready and high quality. But it's unsurprising that the output can also contain problems. And as reasonable web developers, we've got to look at the problems we create.

To make this more concrete, let's look at v0, Vercel's LLM-based code generator product that the Vercel CEO announced as:

v0.dev produces the kind of production-grade code that we'd want to ship in our own @vercel products.

(From: tweet by Guillermo Rauch, 12:15 AM · Sep 15, 2023)

I mention this specifically, because I think claims like “production-ready” are an overestimation of the technology and an undervaluation of the need for humans. Which has real-world effects on people.

When I read “production-grade”, I read “accessible”. I had a brief look at the first six components in the ”featured“ section of v0, I found WCAG violations and accessibility barriers in each.

Examples of barriers in each

• in math learning app example: buttons marked up as links, progress indication that was only visual with no text alternative, heading marked up as div
• In kanban board example: list of items not marked up as list, column headers with low contrast, overlapping text on zoom
• in accessibility helper example: overrides existing shortcuts, icons not marked as decorative
• in terminal UI: buttons not marked up as buttons
• in pricing table: icons not marked up as decorative, button with insufficient contrast
• in music player example: various buttons not marked up as buttons, some buttons not available with just keyboard, buttons without accessible names

This isn't a full conformance audit, I just listed the first few things that stood out. I don't mean this as an attack, I just want to show exactly how common accessibility issues in LLM output are.

You might say it's not all terrible, and that's true. I also found lots of markup that makes things accessible, for instance headings that are useful to navigate in various tools, good contrasts and useful + valid ARIA. But that same level of accessibility often exists on websites that didn't involve LLMs. Lots of websites have fairly useful headings, good contrast on many elements and valid ARIA. It's the bits where those things aren't in place where web UIs create barriers for people with disabilities. It's the nuance that matters.

The self-confidence issue

In Can generative AI help write accessible code?, Léonie Watson looks at the output of three other generative AI tools (ChatGPT, Bard and Fix My Code). Like me, she found things that weren't terrible, things that were actually helpful and things that constituted accessibility issues. But Léonie points out a different problem: these tools thend to present themselves as authoritative. Regardless of whether they are. She explains:

Other than the generic statements about the need to check its responses, none of these generative AI tools gives any hint that their answers may not be correct or provides any recommended resources for checking".

In contrast, most good blog posts and resources about accessible coding have a lot of nuance in them. They usually can't recommend one authoritative solution that is guaranteed to work at all times (what definition of “work” would they use?). And that reflects making accessible interfaces in general. It involves rabbit holes. There are generally multiple ways and multiple least bad outcomes to balance between.

Ok, but can LLMs at least be partially useful?

Maybe the problem of authoritativeness could be solved. We could tune these tools to output responses that don't present as mansplainy know-alls. But that still leaves us with other problems: inaccessible suggestions, lack of intention and understanding and lack of innovation.

Falsehoods and hallucinations

LLMs give inaccessible suggestions, as demonstrated in the examples I shared above and in the examples in Léonie's post. If these falsehoods are a consequence of training data, that could be improved with different training data (emphasis on “in theory”). But it's also due to “hallucinations”, a problem inherent to the tech that research shows is inevitable. They make wrong stuff up. Output may be nonsensical. At the expense of users. That can't possibly be an improvement to the status quo: even without “AI” there are plenty of accessibility tips on the web with specific bugs or issues, automating the addition of falsehoods and hallucinations to the mix seems absurd.

Lack of intentions

LLM tools don't have intentions, and intentions are necessary for (most) “accessible coding”. In his post Why doesn't AI work for producing accessible code, Alastair Campbell explains accessibility is not an average. That makes it incompatible with statistical methods to make suggestions.

Lack of innovation

While there are lots of open source component libraries, many UI patterns and their implications haven't been invented yet. Their assumptions dearly need testing. Relying on LLMs for suggestions means relying on (remixed) existing knowledge, so it's unsuitable for making new patterns accessible.

These three reasons make me wonder: are LLMs useful at all in assisting us in building accessible front-end components? If there is a use, it's probably in helping developers discover resources that do contain nuance, not in code suggestions. Maybe there are also uses outside of component code, but that's for another post (see also Aaron Gustafson's Opportunities for AI in Accessibility).

The focus

Probably something for a post on its own, but I feel I should mention here: a focus on trying to find a “fix” or “solution” for accessibility constitutes a misunderstanding of what accessibility is about. When we make websites, the onus is on us to make them accessible. If we want to try and outsource that work to a tool (that we can't trust), we put the onus on disabled users (see also: disability dongles).

As Adrian Rosellli wrote in AI will not fix accessibility, accessibility is about outcomes, not outputs:

Accessibility is about people. (…) When we target output versus outcomes, we are failing our friends, our family, our community, and our future selves. We are excluding fellow humans while we try to absolve ourselves of responsibility. (…)

Eric Bailey posted:

Thinking AI will "solve" accessibility is a bad frame stemming from a technoableist mindset.

The industry seems to me hoping for a magic, binary solution (…) Personally, I'd look to the social model of disability for guidance here: what exactly are we looking to "fix" and why?

In summary

Is the nuance that accessibility usually needs generatable? I think not. Not reliably, anyway. If you take away one thing from this post, I hope it's this warning: LLM-based tools can't be the magic bullet for writing accessible component code that they promise to be. Because nuance, understanding and conveying intent are inherent to accessibility, LLMs cannot be of great help with the accessibility of component code. In addition, they hallucinate inevitably and tend to pose as authoritative while outputting (occassional, but real) falsehoods. The latter can be dangerous and is likely to come at the expense of users.

My suggestion to developers who want help building accessible components? Use a design system that's well tested with people, that is well documented, and that (at least) attempts to capture the nuance. Or get involved in building one. Not everyone wants to do this nuanced and precise work, not every organisation has the budget. That's fine, but let's not suggest it can be automated away, magically. Let's value the human effort that can make web products actually great.

---

Originally posted as “AI” and accessible front-end components: is the nuance generatable? on Hidde's blog.

Reply via email

The amount of content on the web is so large, that it's tricky to find the stuff worth reading. One of my strategies is to follow people I trust and read what they share. For anyone with interests similar to mine, I've opened a Links section on this website, too (with it's own RSS feed).

My plan is to publish no more than a couple of links per day (if any). They will mostly be related to technology and/or ethics. I have taken inspiration from many others, like Jeremy's Links section. Mu-An inspired me to use Shortcuts as a tool to create links and notes.

Why?

The reason I want to publish links on this site is mostly for selfish reasons. I've posted links on social media for a long time, but in the black box of algorithms, it's hard to recover them after time passes. I want to at least try and have some sort of system for organising and archive my interests (tags… I'm adding tags).

I also want to try and experiment with shorter, quicker posts.

How: low treshold publishing with a Shortcut

I read mostly on the go, when on public transport or waiting for an appointment. This means I usually am not logged into a CMS or near a computer where I can do version control. This site doesn't use a CMS, but I have (Markdown) files in version control that I populate a static site from. To appear on my site, links shared would ultimately need to exist as Markdown files in a specific folder.

This is what I wanted for my link sharing system:

• Very minimal effort
• Should work on all devices
• Should draft a note with both currently selected text and a link to the page, named after that page
• Should also include the current date in the draft and let me title the note
• Should place my draft somewhere that I can move to my site quickly

What I ended up with is an Apple Shortcut that takes the current text selection, page name and page URL on a given page in Safari and creates a blob of text with current date, selection and link prefilled. When I run it while in Safari, a popup opens with something like this prefilled:

--- 
tags: []
date: 2024-01-10
---

> // Selected text

(From: [Name of the page](link to the page))

I can then write some context around the link, optionally add a comma-separated list of tags and then save the file. The filename becomes YYYY-MM-DD-.md, where I can write a title for the post after the date. My site generator grabs that title from the file name.

At the time of writing, I haven't figured out how to then get this file in git, so I save it in a specific folder, requiring me to manually drag it into my site whenever I do reach a computer. That works fine for now, I don't write that many anyway.

Summing up

I'm looking forward to continue doing this for a while, and hope the low treshold publishing will make it so easy that I actually will. Check out /links to find out what I've posted so far.

---

Originally posted as Sharing links on Hidde's blog.

Reply via email

In the nineties, Clippy lived inside the Microsoft Office suite. It was an avatar of an actual paperclip that would interrupt your writing with tips. Did it provide lots of value? I can't remember. I recall there were good easter eggs. But I don't think it really fixed our productivity in some groundbreaking way. A cute relic of the past, in a time where a lot less people were using computers daily.

Fast forward to this week. I used the current version of Microsoft Word, on the web, to write a thing for work. I wrote a headline. Let's say it was “Marketing 2024”. You won't believe what happened next. A blue circle appeared next to the headline. When I hovered over it, it asked me if I wanted it to insert an image. “An image?”, I thought, somewhat surprised. Was this what the future looked like? Did Clippy return? It certainly got in the way of my writing. When I obliged, it put in a random image that felt very marketing-plan-y. It also showed me a gallery of other images for me to pick instead.

A blue dot appears. When clicked, it inserts an image into the writing flow

It left me with lots of feelings, the main one being how unnecessary all of this was. All that computing power!

The “AI” feature I described assists with images, but Microsoft are rolling out Copilot across many of their products. It can also write new text, or improve existing text in ways you specify, the kinds of features we've seen large language models provide. Copilot is a clever name, and a clever framing of what these chatbots can do: they're not the main driver, that's you as a user, but they can be there alongside you.

And yet, I'm not convinced these features are helpful. They interrupt a flow, the actual content production. And they're actively pushed onto users, from in-software notifications to promotional webinars. If that push is successful, everyone in the world will have to put up with the fruits of these features. It's to be seen what those fruits are: content that is better, content that is more superfluous or a bit of both?

User need?

Do these features empower users? They appear to, from the onset. Of course, a lot of the world's marketing plans look very similar, people probably already copy pasted them from one another or used boilerplates. This just makes recycling old ideas available through a different interface. It makes it easier to make something to that seems ok.

But ultimately, is it the right kind of support? Personally, I want software to push me not towards reusing what exists, but away from that (and that's harder). Whether I'm producing a plan or hefty biography, push me towards thinking critically about the work, rather than offering a quick way out.

To be fair, Word has some features today that help you improve content, but mostly to correct style, grammar and punctuation (nice!). Well, and, I'm not making this up, to help remove “sensitive geopolitical references”.

I did ok on formality and sensitive political references

Writing as trying to discover something new

When I write or (publicly) talk, I hope to write or say something that is relatively unique. Obviously, I don't have the illusion I actually do, unique things are rare, everyone's always iterating upon the work of others. But to at least try to find a new perspective is the point of most writing (if it's meant for reading or convincing). That's what we're all trying to do, right?

Whether it's marketing copy, a techy blog post, a stage play or a philosophical treaty, would anyone set out to produce something similar to what already exists? Would the stage play sell tickets if it sounded like the same old? Would your product sell if you use the same copy as your competitor? (Ok, that last one happens all the time)

Discovering new ways is central to creativity. When Miles Davis recorded his groundbreaking album Kind of blue, he gave musicians scales and melody lines, and asked them to improvise. They hardly practiced and did not know the music inside out before they started. Herbie Hancock explains this made the whole album a lot more spontaneous:

Miles' idea was that… he wanted to capture the spirit of discovery in the music. (…) He wants to capture discovering the music on the record.

(in: The making of Kind of Blue, 6:22)

I recommend that video, it's full of people who understand music and this music in particular. They try and explain some of the creativity of this particular piece.

Of course, most writing doesn't have to yield a creative masterpiece. But it's probably that sense of discovery where, like making music, writing gets most interesting. For writers and for readers. More than recycling old and making grammatical and stylistic improvements (quite useful), writing assistants could drive us in the direction of discovery more.

Unnecessary “AI”

It's not just for fun that I'm calling “unnecessary” on this feature (and who am I to judge, or, frankly, who do I think I am?). This is feature that people probably worked on hard and that some users may find very useful. But still, it's worth considering the need. Large language models, that most “AI” features are based on, cost a lot of computing power, both training them and running them. The training also involves a lot of (underpaid) labour of people, who classify content to make the magic work. And then there's all the knock-on effects on society of having harms perpetuated and being stuck with lots of content that nobody thought was worth writing in the first place.

We've got to think about whether or not the features we build with this are actually useful, actually unnecessary or somewhere in between. We don't have to add “AI” to every product, even in an industry where we see investors push for that, we can choose to do otherwise. (</man-yelling-at-cloud>). One company stood out thinking about this especially carefully in the last while: iA wrote Writing with AI.

---

Originally posted as Clippy returned (as an unnecessary “AI”) on Hidde's blog.

Reply via email

Wait, what, it's only a week until 2024? Time for what has become a yearly tradition… in this post, I'll review some of my 2023 in work, conferences, reading, writing, listening, music and learnings.

Note: like my posts about 2022, 2021, 2020, 2019, 2018 and 2017, this post is mostly ‘highlights’, lowlights left out intentionally.

Work

The work year was different than usual: my full time job (first in 10 years or so) ceased to exist and I switched back to freelancing.

Freelancing

In July I joined the NL Design System team (almost) full time. We support a community of web specialists from across the Dutch government to create open-source components and guidelines for all. I joined as an accessibility and developer relations specialist. Now I'm PR/communications lead, with still some time for accessibility (but none for devrel). The project is very much up my ally. I really like the colleagues. I also feel fortunate to bring my experience into this team. And, of course, just as fortunate to have made it once again into a team where I get to learn lots from the others (and all of the design system, government, accessibility and development experience between them).

Outside of NL Design System, I did some in-house workshops, talks about the web and web accessibility, and the occasional WCAG audit.

Volunteering

I continued in Open UI CG, mostly scribing, learning and then conveying some of the work to developer audiences in blog posts and talks. It's still one of my favourite meetings of the week.

Other volunteering didn't work so well. I joined the marketing team of my local solar energy community, made plans but found no time to properly work on them and left. I also joined the CSS WG as an Invited Expert, a long time dream come true. But so far, I only managed to join one meeting and only marginally kept up with emails and issues. Let's say between when I said yes and when it was official, a lot changed at work and I wanted to spend more time with family. I hope they don't revoke my status just yet and hope to start prioritising my WG involvement sometime in 2024.

Conferences

Last year, as the popover attribute started to make its way into browsers, I hoped I would do a talk about the nitty gritty of building popovers (repeating the phrasing I used last year). That happened. I wrote that talk and took it to a number of events. In addition, I made a new talk called “ARIA, The Good Parts” and did a 10 minute rant for the IAAP-EU called “Will tools save us?”.

At CSS Day, I met the hotel cat once again

All my talks in 2023:

• Shifting left: making accessibility easier, by doing it earlier at a11yTalks (online)
• “Dialog dilemmas and modal mischief: a deep dive into popovers and how to build them”, I like long names, what can I say? This popover talk happened at JSNation (Amsterdam) (7 minute edition, with transcript), CSS Day (Amsterdam), Front Conference (Zurich), HalfStack (Vienna), Covent of Wisdom (Eindhoven) and React Advanced (London)
• “ARIA: the good parts” at Paris Web (Paris, obvs), NDC (Porto) and WeAreDevelopers (online)
• “Will tools save us” - introduction to a panel for IAAP-EU's celebration of the 3rd anniversary of the Web Accessibility Directive (Brussels)

I also attended Beyond Tellerrand in Düsseldorf and State of the Browser in London.

Reading

I read a bit less than in the last year, most of them while on holiday or travel (see: full reading list if you like book covers). These are some books I can recommend:

• Just Human by Arielle Silverman (auto biography), in which she shares her own experience being blind within the context of society, from the moment she was born to the age of 36. Learned a few things about the workings of ableism.
• Make me one dimensional by Sang Young Park (novel). Murder mystery, friendships and growing up.
• Doe zelf normaal by Maxim Februari (non-fiction), on the role of “datafication” in society. Original, clear, funny at times, witty.
• Le Perfezioni (De perfecties) by Vincenzo Latronico (novel). On gentrification, Berlin, millenials.
• Tomorrow and tomorrow and tomorrow by Gabrielle Zevin (novel). The videogame industry, sudden success, weird relationships. Was told this is ‘a TikTok sensation’, but won't let that ruin my feelings about the book or my liking it. Consumed this as an audiobook on a few long drives.
• Binge by Douglas Coupland (short stories), 60 (!) stories, loved most of them, easy to read in between other tasks.

Music

Saw more live music than in the year before (yay, thanks to babysitters and being more ready for their help as a family). I got lucky that the artists I listened to also toured near me on the right dates, so I got to see:

• Little Simz, at North Sea Jazz. Was spectacular. Would have loved a full band, vocals etc, but even without that it was great. I was introduced to her No Thank You album, worked my way back through Sometimes I Might Be An Introvert and Grey Area, and now I'm all in Stillness in Wonderland, it was a fantastic musical rabbit hole.
• James Blake, at 070. I don't understand how he makes music (how do synthesisers work?), but the new album is great and sounded excellent live. Some fans knew it was birthday, so we all sang.
• Esperanza Spalding, also at North Sea Jazz. I'd not heard of the fairly sexist song ‘Girl Talk’, but the way she performed it with Fred Hersch was awesome (see Girl Talk on YouTube).

I also listened a lot to De Jeugd van Tegenwoordig, Sault, Sef, WIES, Massive Attack and Typhoon.

I made a live music log to track concerts publicly (inspired by Vasilis).

Writing

Including this one, I wrote 15 posts this year, some lengthier than others. Less than usual. There's a bunch of drafts that I'm looking forward to finish!
Of this year's posts, most shared were an FAQ on accessibility and one on ableism in the Vercel community.

Most sceptical were a few posts I wrote about ‘AI’, where I said that opt-out is rude, ‘AI’ content isn't user centered and LLMs are not artificial, nor intelligent.

Cities

I stayed in Taipei, Kaohsiung, Düsseldorf, Porto (2x), Berlin, Zurich, Vienna, Paris, Brussels, London (3x), Bristol and Porto.

I wanted and was able to do a lot of this year's conference travel by trains, twice by night train. The latter aren't cheap if you want some comfort (I went for beds), but very pleasant and lower in emissions than flights would have been.

Saw hot air balloons in Bristol

Learnings

Some random things I learned this year, in no particular order.

• Social media doesn't need algorithms for that real community feel. I still like it on Mastodon and still mostly left Twitter and Bluesky. I still use Instagram (to follow artists) and LinkedIn (to follow a random mix of people that never came to Mastodon).
• A lot about electric cars (and probably still too little). Quite the rabbit hole, but I bought one, so that I can stop buying petrol. I didn't want to buy one without researching some ins and outs.
• In git: to fix commits further back than the very last one, by doing fixup commits and autosquashing them. I hope I won't forget.
• Lots of words in Mandarin (~1000). I had some formal courses, but midway this year I started Duolingo (add me!). It uses simplified Chinese characters (not traditional) and Taiwanese friends and family scold me for that. But honestly, the upsides outweigh the downsides for me.
• Fresh YouTube accounts (I set one up for work) recommend even more extreme-right content than my many-years-old one does. I wish I could turn recommendations off altogether.
• Conference speakers are actually just humans.
• Lots of things about government departments and how they work together and the meanings of very specific acronyms.

Wrapping up

Thanks for reading, hope there's useful recommendations in this post. If you're still reading let me end with my best wishes to you for the new year, see you in the next!

---

Originally posted as 2023 in review on Hidde's blog.

Reply via email

Inspired by Chris Coyier' Answers to common (web) design questions, which was inspired by Dan Mall's earlier post, here's list of common accessibility questions.

Should I use links or buttons?

Yes.

Links if it takes the user somewhere, buttons if it performs an action. Also buttons if the action is submitting a form (even if the user is then taken somewhere). Trying to avoid nuance in this post, but here's some nuance around buttons and links.

Do we have users with disabilities?

Yes.

It's unlikely you know every single one of your users and exactly how they use the web. It's even more unlikely that the the group and the people within it stays exactly the same forever.

What's an accessibility conformance audit?

Someone will find out for every of the 55 Success Criteria in WCAG whether your site meets it or not (counting version 2.2, Level A + AA). Ideally, they also explain what the issues are and how to fix them (so that you can do it). This is also called a conformance evaluation.

Who should “do” accessibility in our team?

Everyone. Content folks, developers, designers and product managers all have accessibility tasks to do.

What are some quick tests I can do?

Use your UI with Tab / Shift Tab on a keyboard (check settings if on a Mac), can you reach everything without a mouse? Does the order make sense?

Click on labels for form fields, they should focus the field they are a label of.

Check if your videos and audio (podcasts?) have captions / transcripts.

Is accessibility ever done?

No. It's a continuous process, even if your audit says you meet all Success Criteria today, it's common to stop meeting it. Websites change. You'll want to continuously monitor accessibility, just like with security and privacy.

Do we have legal obligations to make our products accessible?

Very likely. Also if you're not government (for instance, see European Accessibility Act).

There are policies and laws all around the world.

Is it all my website's fault?

No, some problems can be solved by browsers, assistive technologies and/or authoring tools.

WCAG 3.0 will be released soon, right?

Not likely. The working group that made WCAG 2 (Accessibility Guidelines Working Group) is currently working on requirements and draft definitions for a next version that will likely be called WCAG 3.0. There is a Working Draft. It is very much in development in a very early phase, and likely to change considerably.

Will “AI” improve accessibility?

Machine learning can be a great tool for automating part of the captioning process in lots of languages, and various other things.

But it's unlikely LLMs, often called “AI”, will output accessible code. To train such an LLM, an enormous set of very accessible code would need to exist (it doesn't). Component-building and accessibility semantics also require intentionality, which these systems specifically aren't good at. See also my other post: Is the nuance generatable?, and Craig Abbott's post Deque Axe Assistant - First impressions.

Is the Axe / Page Insights score all that matters? Or the WCAG audit result?

No. Any system that scores your site and returns some number (including WCAG audits) does not fully describe your accessibility situation. Accessibility is, ultimately, about people and whether they can use your site. It's about recognising, then removing barriers. Metrics can help in various ways, but they are not the end goal. And the most easily measureable is not necessarily the most impactful.

More detailed accessibility posts can be found elsewhere on this blog.

---

Originally posted as Answers to common (web) accessibility questions on Hidde's blog.

Reply via email