To doctors opening patients’ electronic records across the U.S., the alert would have looked innocuous enough.
A pop-up would appear, asking about a patient’s level of pain. Then, a drop-down menu would list treatments ranging from a referral to a pain specialist to a prescription for an opioid painkiller.
Click a button, and the program would create a treatment plan. From 2016 to spring 2019, the alert went off about 230 million times. (Source)
Even if code is modified with the aim of securing procedural fairness, however, we are left with the deeper philosophical and political issue of whether neutrality constitutes fairness in background conditions of pervasive inequality and structural injustice. Purportedly neutral solutions in the context of widespread injustice risk further entrenching existing injustices. As many critics have pointed out, even if algorithms themselves achieve some sort of neutrality in themselves, the data that these algorithms learn from is still riddled with prejudice. In short, the data we have—and thus the data that gets fed into the algorithm—is neither the data we need nor the data we deserve. Thus, the cure for algorithmic bias may not be more, or better, algorithms. There may be some machine learning systems that should not be deployed in the first place, no matter how much we can optimize them. (Source)
Developers cannot just ask, “What do I need to do to fix my algorithm?” They must rather ask: “How does my algorithm interact with society at large, and as it currently is, including its structural inequalities?”
Friere had warned early on that uncritical liberation can lead to the oppressed reproducing oppression. I see it all the time, I even probably do it all the time myself. It takes an incredible amount of reflectivity and soul searching to stop this cycle, and it needs a lot of support and care from those around us who can be our allies in this.
The ideal place for empathy games is within a structured environment, such as a game-of-the-month discussion club or a classroom. In order for empathy games to move past "pity tourism," they need to be explored more fully and fleshed out by a clearly defined and supported curriculum. (Source)
(Boy, is this true even in this relatively-ideal environment?)
Who has the power? Who’s vulnerable to harm? Where are the patterns? What’s the context?How does it make you feel? (Source)
In the calls for greater emotional intelligence as the answer to our economic situation, there is a core assumption that the harm to workers under capitalism comes not from a system which denies labor access to the value it creates, but fundamentally from the forms of affect that this system embodies. Change the affect, and not only will you resolve this harm, but — and perhaps more importantly — your institution will be more effective as a result. Workers might still be exploited, but they can feel better about it. (Source)
Games & sims reward interpreting what the system wants and feeding that back to them. Therefore, in capitalism, demonstrating measurable external signifiers of empathy is more important than developing empathy itself.