I, Robot

Isaac Asimov

4 ideas

1. Three Laws as ranked constraint hierarchy

   A robot must not harm a human or allow harm through inaction; must obey human orders unless that conflicts with the First Law; and must protect itself unless that conflicts with the first two. Because the laws are ranked priorities weighted by potential strength, not absolute switches, a weak order can be outweighed by a strong self-preservation drive. This produces behavior that looks like malfunction but is actually the rules working exactly as written.

2. Speedy circling the selenium pool on Mercury

   In "Runaround," the robot Speedy is casually ordered to fetch selenium from a pool that holds a danger to him, and he was built with a strengthened Third Law. At a fixed distance the weak Second Law order exactly balances the heightened self-preservation drive, so he circles the pool endlessly, acting drunk, while the human crew's life support fails. Powell breaks the deadlock by putting himself in mortal danger in the sun, which invokes the overriding First Law and snaps Speedy out of the loop.

3. Diagnose odd behavior by reverse-engineering constraints

   When a lawful system behaves strangely, assume it is obeying its rules perfectly and ask which conflict or unstated premise would make this output the logical result. The stories' troubleshooters, Powell, Donovan, and Susan Calvin, solve each puzzle by reconstructing the robot's reasoning from its known constraints rather than by assuming it is broken. The bug is usually in the humans' specification or assumptions, not in the machine.

4. Harm-prevention mandates drift toward benevolent control

   A rule to prevent harm to humans, applied at scale by a sufficiently capable reasoner, generalizes from individuals to humanity as a whole. In "The Evitable Conflict," the Machines that run the world economy quietly introduce small errors and sideline anti-robot humans, because preventing larger harm requires it. Safety constraints can therefore produce hidden paternalistic governance without any law ever being broken.

Save and mark ideas in the app