p5*js

☼

FES

✎

Full Report

2022 p5js Friendly Errors System (FES) Survey

Before I talk about the FES survey, I want to share a story about my backyard. This year, my family got to rent a place with a backyard. It was our first time living in a home with a backyard, so we had no idea what to expect. There were many surprises. For example, I didn't know we would have so many visitors every day.
We discovered there were a bunch of neighborhood cats who came to our backyard almost every day. They'd appear from nowhere and quietly spend time in our backyard. They were always careful but looked comfortable exploring around. Soon they became an essential part of our backyard.
To further please our frequent visitors, we decided to provide a cardboard box for them. I mean, cats can't resist boxes, right? (at least that's what the Internet taught me).
Much to our surprise, our ambitious offering got absolutely no attention from the cats. We tried a few different arrangements to make it more appealing and accessible, but none worked.
After a while, we ended up removing the box. Life in the backyard continued without it.
Here is my confession: ever since the release of the friendly errors in 2017, I have been worried about it being an "overdesigned" feature, despite its good intentions. Something isolated from the experience of actual users, something similar to our backyard box.
My backyard amenity project was a bad design even though I followed general guidelines about my users. The specific cats who visited our backyard somehow didn't fit the widespread expectation about cats. Would the box project end up differently if I had spent more time communicating with my guests?
I couldn't do an open mic with my neighborhood cats, but I could hear directly from our community for Friendly Errors. So with generous support from the p5*js community, the 2021 FES survey happened. I wanted to see what kind of animal our friendly errors had become. And maybe I will collect some info that may help future contributors.
First, who responded to our survey? We gathered ~70 responses, mainly through our Instagram post and Discord channel. Most of our participants reported using p5*js for multiple purposes: 54% for learning, 47% for teaching, 57% for making, and 82% for experimenting.
Despite a wide range of technical backgrounds, about 91% of our respondents reported that they read friendly errors to some extent ("Frequently" 52% or "Sometimes" 39%).
Plus, about 60% of respondents said they learn new information often (17%) or occasionally (43%). Along with these positive results, we also received many encouraging messages. Thank you so much!
We also got a diverse array of feedback, reflecting on the limitations of the current design. We categorized some of the feedback into these: Confusing Messages, Unmatched Needs, Incomplete Info, False Positives ... and more. (you can read the full list of feedback at: https://observablehq.com/@almchung/2022-p5jsfes-survey#cell-1951)
Paired with the feedback, we also got a variety of specific suggestions: Center browser JS error messages, Good error writing, Support better workflow, Better integration with the web editor, Customization ... and more. (you can read the full list of suggestions at: https://observablehq.com/@almchung/2022-p5jsfes-survey#cell-2004)
It was also eye-opening to see how diverse our respondents' technical backgrounds were. The nuances of our users' needs will be ignored if we simplify the audience of friendly errors as "beginners in coding."
In the last section, we asked the participants to describe their ideal friendly errors. This was to probe the possibility of friendly errors and imagine the future of its design together.
We asked all participants to list three English adjectives to describe their ideal friendly error system. As expected, the collected answers wove a complex web! Some concurring. Some conflicting. You can read all responses in this section of the report: https://observablehq.com/@almchung/2022-p5jsfes-survey#cell-649
When these dreams are all combined together, what would they become? Is that even possible? We don't have to answer that right now. I'm grateful just to think this with you. Maybe I just wanted to create a nice excuse to hang out together in the backyard.
This survey is far from capturing the whole experience of friendly errors. But I hope it will provide a step before new attempts, new experiments. Thank you, and let's continue the talk.

 To read the full report ⇢ 2021-2022 FES Survey: Full Report

 To the p5*js project ⇢ p5*js

 To read about the beginning of the Friendly Errors project ⇢ Friendly Error System for p5.js