Ethics

Numerous executives in tech repeatedly talk about how they think “AI” is going to replace workers. By their own account, that seems to be the point of the technology.

So far, the impact seems limited to a few select fields. Copywriters have been hit hard. Translators and illustrators are losing gigs everywhere I look. Training, especially in software development, seems to have been hit hard. Voice actors are getting replaced with generated voices.

The goal, if we are to take tech executives at their word, is to make these trends the norm, not the exception.

That is a political project. Attacking labour, deskilling the work force, and driving down wages, is fundamentally a political project and an extremist one at that.

Centrally managing language, using pervasive chatbot adoption as a lever to change corporate writing at scale, is another explicit goal of these companies. When Musk and Altman argue about which of their respective chatbots is less “left-wing”, their intent is clearly that they want to make all writing done with their tools less left-wing.

Centralised ideological control over all corporate writing is, again, an extremist political project, historically associated with violent authoritarianism.

We could go through how these tools are being used to power a wholesale takeover of our education systems, create a tiered system of healthcare access, and automate decisions to ensure that nobody can be held accountable for atrocious decisions, but it all comes down to the same, repeated point:

The AI Bubble is a right-wing political project that goes hand-in-hand with the ongoing resurgence of fascism.

Tech is a pop culture. Very few of the decisions made in the industry are made rationally or empirically. Studies and tests are used to justify the emotional decisions of the executive or management class. Infrastructure and stack decisions are made hedonistically – “cool” tech that makes the engineers and devs feel good about themselves almost always gets a priority over “boring” tech that has no risks.

The industry, especially the software side of tech, is driven by emotion and a sense of what is fashionable. There is genuinely more grounded engineering – materials, machinery, process, supply chains, etc. – taking place in the fashion industry than there ever has been in the software industry.

That’s why it’s a pop culture, not a fashion culture.

I think of the engineers and designers who have spent decades honing their skills, deepening personal and public creative practices in service both to the users of the systems they built and to their own brilliant spirits, now being told to park themselves in front of a sycophantic oracle that can be appeased only through rote dictates, and which never tires of lying even as their own minds and muscles atrophy from disuse. What is being automated here: the work or the people?

The workers with knowledge of the project said it was initially intended to improve the chatbot’s voice capabilities, and the number of sexual or vulgar requests quickly turned it into an NSFW project.

“It was supposed to be a project geared toward teaching Grok how to carry on an adult conversation,” one of the workers said. “Those conversations can be sexual, but they’re not designed to be solely sexual.”

“I listened to some pretty disturbing things. It was basically audio porn. Some of the things people asked for were things I wouldn’t even feel comfortable putting in Google,” said a former employee who worked on Project Rabbit.

“It made me feel like I was eavesdropping,” they added, “like people clearly didn’t understand that there’s people on the other end listening to these things.”

Critics have already written thoroughly about the environmental harms, the reinforcement of bias and generation of racist output, the cognitive harms and AI supported suicides, the problems with consent and copyright, the way AI tech companies further the patterns of empire, how it’s a con that enables fraud and disinformation and harassment and surveillance, the exploitation of workers, as an excuse to fire workers and de-skill work, how they don’t actually reason and probability and association are inadequate to the goal of intelligence, how people think it makes them faster when it makes them slower, how it is inherently mediocre and fundamentally conservative, how it is at its core a fascist technology rooted in the ideology of supremacy, defined not by its technical features but by its political ones.

[…]

I am here to be rude, because this is a rude technology, and it deserves a rude response. Miyazaki said, “I strongly feel that this is an insult to life itself.” Scam Altman said we can surround the solar system with a Dyson Sphere to hold data centers. Miyazaki is right, and Altman is wrong. Miyazaki tells stories that blend the ordinary and the fantastic in ways people find deeply meaningful. Altman tells lies for money.

Angela Collier nailed exactly how the big tech hype and misleading claims would go when she posted this in 2023.

If you think [LLM] crawlers respect robots.txt then you are several assumptions of good faith removed from reality. These bots crawl everything they can find, robots.txt be damned, including expensive endpoints like git blame, every page of every git log, and every commit in every repo, and they do so using random User-Agents that overlap with end-users and come from tens of thousands of IP addresses – mostly residential, in unrelated subnets, each one making no more than one HTTP request over any time period we tried to measure – actively and maliciously adapting and blending in with end-user traffic and avoiding attempts to characterize their behavior or block their traffic.

We are experiencing dozens of brief outages per week, and I have to review our mitigations several times per day to keep that number from getting any higher. When I do have time to work on something else, often I have to drop it when all of our alarms go off because our current set of mitigations stopped working. Several high-priority tasks at SourceHut have been delayed weeks or even months because we keep being interrupted to deal with these bots, and many users have been negatively affected because our mitigations can’t always reliably distinguish users from bots.

All of my sysadmin friends are dealing with the same problems. I was asking one of them for feedback on a draft of this article and our discussion was interrupted to go deal with a new wave of LLM bots on their own server.

[…]

Please stop legitimizing LLMs or AI image generators or GitHub Copilot or any of this garbage. I am begging you to stop using them, stop talking about them, stop making new ones, just stop. If blasting CO₂ into the air and ruining all of our freshwater and traumatizing cheap laborers and making every sysadmin you know miserable and ripping off code and books and art at scale and ruining our fucking democracy isn’t enough for you to leave this shit alone, what is?

Training data are construction materials for a language models. A language model can never be inspired. It is itself a cultural artefact derived from other cultural artefacts.

The machine learning process is loosely based on decades-old grossly simplified models of how brains work.

[…]

It’s important to remember this so that we don’t fall for marketing claims that constantly imply that these tools are fully functioning assistants.

[…]

Who was the first man on the moon?

[…]

What you’re likely to get back from that prompt would be something like:

On July 20, 1969, Neil Armstrong became the first human to step on the moon.

This is NASA’s own phrasing. Most answers on the web are likely to be variations on this, so the answer from a language model is likely to be so too.

[…]

The prompt we provided is strongly associated in the training data set with other sentences that are all variations of NASA’s phrasing of the answer. The model won’t answer with just “Neil Armstrong” because it isn’t actually answering the question, it’s responding with the text that correlates with the question. It doesn’t “know” anything.

A core aspect of the theory-building model of software development is code that developers don’t understand is a liability. It means your mental model of the software is inaccurate which will lead you to create bugs as you modify it or add other components that interact with pieces you don’t understand.

Language model tools for software development are specifically designed to create large volumes of code that the programmer doesn’t understand. They are liability engines for all but the most experienced developer. You can’t solve this problem by having the “AI” understand the codebase and how its various components interact with each other because a language model isn’t a mind. It can’t have a mental model of anything. It only works through correlation.

For years, we studied teams rolling out new designs, to see if we could mitigate negative reaction to new releases and design changes. We studied hundreds of product and service rollouts. We watched and learned from the reactions of thousands of users.

When we dug into what those users’ reactions [were], patterns emerged. The users told us the changes inconvenienced them. They had no idea the change was coming and suddenly it was in their face. Users were upset because they were surprised.

They also told us the old version worked fine. Even when it took a while to get comfortable, they learned it. Many users mastered difficult-to-use designs.

Everything was different when the new version arrived. What they’d mastered before didn’t help them now. The company said it was an improved design, but they couldn’t see the improvements. Why should these users learn something new that doesn’t help them? Users were upset because they couldn’t see the value.

We also saw many instances where users didn’t react negatively to changes. Often, they didn’t react at all. We saw new designs that didn’t affect the users’ behaviors and they didn’t pay attention to it.

In these cases, the changes were often not noticeable. Sometimes the changes were small and isolated. Yet, we also saw users seemingly not notice several updates with extensive changes. (In more than one instance, an entire application’s infrastructure had been rewritten without a single user noticing.)

In cases when the design changes were noticeable, the designers gave the users control to switch when they wanted. The designers showed why the change was valuable to the users. And the designers made the transition easy by taking the knowledge and experience their users already had with the product into account.