A few months ago, I read this post by Cory Doctorow about how shitty tech always tends to move upwards from the most marginalized groups to the more privileged and it’s a thought technology that has stuck with me. I’ve rolled it around like a smooth beach pebble in my hand for weeks and weeks because I keep seeing it in news cycles whenever a disastrous new tech policy is announced.
The gist of it is this: the worst ideas in technology often get deployed first in places like prisons, and then places like schools with young kids who can’t fight back. Then things move up the chain to say, people out on parole and older kids at school but often the people behind these decisions don’t meet much friction until they get to people like college students who can organize and strike against absurd tech, or middle-class people who don’t want to be subjected to constant surveillance and can do things like pay for the ability to opt-out.
I’ve seen it happen many times with GPS tracking technology and always-on cameras and microphones and pay-by-the-minute video conferencing and face and fingerprint and DNA-based recognition systems. You always subject people who don’t have the ability to complain about your new horrible ideas first.
This essentially is the playbook:
- devise a terrible new flavor of technology and/or policy
- deploy it to people with no rights and no ability to resist
- test and refine, and give it time to feel “normal” to those populations
- widen your deployment to larger populations (who have more autonomy) only after devising metrics that allow you to establish a track record of it “working” on the people you initially subjected it to
- repeat all the way up
Once you see the pattern, you can’t unsee it, and you start to see it everywhere.