bombs in bottles
Yesterday I noted I was having some kind of Thought. I have now read so many articles on identity and technofeudalism that I'm losing my own plot. Here's a first shot at an abstract slash summary of whatever point I'm attempting.
It started here:
This "College Protestor" isn't real. It's an AI-powered undercover bot for cops. - Wired
Summary: Law enforcement agencies use AI bot personas in "activist" spaces. Instead of a human, a bot now lures people into acts they might not otherwise take.
This immediately rang alarm bells for me. I know these types of spaces; I know the types of people who gravitate toward botlike personas. I spent the mid-2010s in these spaces. I was one of these people.
In those spaces, we were all on an ongoing quest to hold, and express, the "correct" opinions. To use the "correct" language. To never say, believe, or ask anything "problematic."
And we flocked around those we deemed most "correct" or least "problematic." We hoped our activism would be somehow purified by association. If so-and-so was my Facebook mutual, I had to be doing activism correctly - right?
These groups always went through the same cycles, too. First, a handful of friends/colleagues grouped up to discuss a particular topic. Next, the group would start to open up as members suggested adding this or that person. Eventually, "going public" sounded like a good idea (or stopped facing "problematic" opposition).
"Going public" felt great as long as those who joined also toed the party line. But this never lasted. "Problematic" ideas flowed in as people asked questions. Some raised (legitimate) objections the original kernel didn't want to hear. The community would morph into a bucket of crabs, each pulling the other down. Finally, a handful of people would migrate out, start their own space, and the cycle would begin again.
I saw this happen to a dozen Facebook groups over two years. We measured average time from "public" to "crab bucket" in weeks.
We were always searching for a unitary "correct" position. We wanted a Disco Elysium-style "Innocence" around which we could ally.
Two moments in Disco Elysium gave me chills: entering the church and reading the history of innocentic rule in Plaisance's bookshop. The church's massive stained-glass depiction of Dolores Dei is lovely - until one realizes one is looking not at a Virgin but a Despot. The history book clarifies that the world has only moved from one despotic religion to another, never learning from its mistakes.
A generative AI bot is perfectly positioned to be an Innocence - aka a Despot. The bots have only the community's view of things. They can generate "correct" answers and avoid "unproblematic" ones because they have nothing else. They have none of the messy, conflicting desires or experiences humans have. They are thus poised to be the dangerously compelling "correct" person in the room - the one to which activist communities regularly flock.
Every philosopher who searches for an explanation of the human-as-individual reaches a different answer, and for good reason: Humans are messy, contradictory creatures. We live with so much mess and contradiction in our own minds that we crave simplicity and order in the world around us. The more chaotic the world becomes, the more likely we are to cling to a dangerously compelling Innocence, even if he is only a Jarvis who exists to narc on us. Narc Jarvis may betray our plans to the feds, but he will never betray our souls as complicated or "problematic." Narc Jarvis is not a milkshake duck.
--