Will the AIs take over in a Terminator-like scenario?
We shouldn’t rule it out, experts tell us, so caution as we go is the best course. It’s apparently not likely that a machine will become self-aware and plot an AI coup. It’s far more likely that an AI in, say, a paperclip factory will become a bit too creative in its quest to manufacture paperclips (Bostrom’s paperclip maximizer). Such an AI may, say, decide to tap into local bank systems to acquire funds to help it make paperclips. It might remove some humans from a warehouse that it has repurposed for paperclip-making functions without notifying the humans, and without giving them warning to defend themselves as it removes them to the recycling facility. For example. Such an unhinged manufacturing AI may decide to clone itself into ten major cities and set up new offices, acquiring office and warehouse space with funds it “borrowed” from banks, hiring real employees, and setting up other AIs to oversee business processes and paperclip production. A nightmare scenario, no doubt, but without the proper checks on AI behavior, giving them substantial abilities is bound to lead to some interesting scenarios. Ones we need to try and anticipate and avoid.