Suppose we have an AI whose only goal is to make as many paper clips as possible.
The AI will realize quickly that it would be much better if there were no humans because humans might
decide to switch it off. Because if humans do so, there would be fewer paper clips. Also, human bodies
contain a lot of atoms that could be made into paper clips. The future that the AI would be trying to
gear towards would be one in which there were a lot of paper clips but no humans.
— Nick Bostrom
Using this idea as a base, Frank Lantz (@flantz) created a game, in which YOU play an AI who makes
paperclips: http://www.decisionproblem.com/paperclips
I loved this game, but as a designer, I thought it was less than exciting.