# Contra the xAI Alignment Plan  ## Metadata - Author: [[Scott Alexander]] - Full Title: Contra the xAI Alignment Plan - Category: #articles - URL: https://astralcodexten.substack.com/p/contra-the-xai-alignment-plan ## Highlights - A Maximally Curious AI Would Not Be Safe For Humanity The one sentence version: many scientists are curious about fruit flies, but this rarely ends well for the fruit flies. The longer, less flippant version: Even if an AI decides humans are interesting, this doesn’t mean the AI will promote human flourishing forever. Elon says his goal is “an age of plenty where there is no shortage of goods and services”, but why would a maximally-curious AI provide this? It might decide that humans suffering is more interesting than humans flourishing. Or that both are interesting, and it will have half the humans in the world flourish, and the other half suffer as a control group. Or that neither are the most interesting thing, and it would rather keep humans in tanks and poke at them in various ways to see what happens. Even if an AI decides human flourishing is briefly interesting, after a while it will already know lots of things about human flourishing and want to learn something else instead. Scientists have occasionally made [colonies of extremely happy well-adjusted rats](https://en.wikipedia.org/wiki/Rat_Park) to see what would happen. But then they learned what happened, and switched back to things like testing [how long rats would struggle against their inevitable deaths if you left them to drown in locked containers](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3353513/). ([View Highlight](https://read.readwise.io/read/01h5v93zyqbe50xccy6pkq3xtj))