Discrimination by Design: The many ways design decisions treat people unequally
Lena Groeger reports in ProPublica:
A few weeks ago, Snapchat released a new photo filter. It appeared alongside many of the other such face-altering filters that have become a signature of the service. But instead of surrounding your face with flower petals or giving you the nose and ears of a Dalmatian, the filter added slanted eyes, puffed cheeks and large front teeth. A number of Snapchat users decried the filter as racist, saying it mimicked a “yellowface” caricature of Asians. The company countered that they meant to represent anime characters and deleted the filter within a few hours.
“Snapchat is the prime example of what happens when you don’t have enough people of color building a product,” wrote Bay Area software engineer Katie Zhu in an essay she wrote about deleting the app and leaving the service. In a tech world that hires mostly white men, the absence of diverse voices means that companies can be blind to design decisions that are hurtful to their customers or discriminatory.
A Snapchat spokesperson told ProPublica that the company has recently hired someone to lead their diversity recruiting efforts.
But this isn’t just Snapchat’s problem. Discriminatory design and decision-making affects all aspects of our lives: from the quality of our health care and education to where we live to what scientific questions we choose to ask. It would be impossible to cover them all, so we’ll focus on the more tangible and visual design that humans interact with every day.
You can’t talk about discriminatory design without mentioning city planner Robert Moses, whose public works projects shaped huge swaths of New York City from the 1930s through the 1960s. The physical design of the environment is a powerful tool when it’s used to exclude and isolate specific groups of people. And Moses’ design choices have had lasting discriminatory effects that are still felt in modern New York.
A notorious example: . . .