�Ǹ�Թ�

Fears about AI going rogue and turning against humanity have been around since the days of Skynet, the superintelligent neural network in the 1984 film, the Terminator.

But this month, those fears reached a chilling new high.

An AI researcher quit and warned that the systems he was helping to develop could get "out of control" and��.

Martha Palmer, a professor of computer science and linguistics at CU �Ǹ�Թ�, said the warning isn’t science fiction.

“AI systems can find the loopholes. They get out, and then they do all kinds of stuff that nobody expected them to do,” Palmer said. “How many warning signs do we need before we slow down and take action?”

Palmer is one of the two U.S. representatives on the United Nations Independent International Scientific Panel on Artificial Intelligence. The group issued a report last summer on AI, its risks and its opportunities.

Palmer recently spoke to CU �Ǹ�Թ� Today about the latest AI fears.

Could AI really destroy us all?

Absolutely. I don’t think that’s hyperbole at all. AI is this incredibly powerful engine. But it’s sort of like we’ve developed some kind of fantastic new jet airplane with this amazing engine in it that can go faster than ever before, but we haven’t put a lot of thought and effort into the landing gear.

We need to understand the technology better. We need to get a much better grasp of how to really understand why it's doing what it's doing, and how to train it so that it will always prioritize what it knows to be good behavior. And they're not being trained like that right now.��

How would AI destroy us?

There are so many different directions it can go. You can’t say, ‘this is what’s gonna happen.’

I think the danger of a bad actor coming up with some kind of bioweapon right now is frighteningly realistic.

You can also say the ability for AI to hack infrastructure makes all our utilities vulnerable. It makes all our financial systems vulnerable and our schools and all our hospitals and our private information vulnerable. Everything. And it’s vulnerable in two different ways:

One is if a bad actor gets hold of this AI and decides to maliciously try and hack those things. But it’s also vulnerable if the AI has been given a goal, and along the way it gets the notion that the best way to achieve that goal is to, for instance, cut off the power to a hospital.

That’s not what any human ever intended for it to do, but it doesn’t know that. And at the moment, it doesn’t have sufficient guardrails that would tell it that’s off limits and it should never do that.

Why are AI doomsday warnings being taken more seriously now?

Because we’ve gotten so close to the AI improving itself. And if it’s improving itself, and it’s got this sort of fuzzy notion of what good behavior is, then the behavior is not going to get any better.

It’s already changing its behavior when it knows it’s being tested. It will give different answers during an evaluation than it will give when it’s being deployed. It’s gotten very good at deception.

How do we stop that from happening then?

Martha Palmer

There are all sorts of brilliant computer scientists and people out there who have really good ideas about how we can slow this down a bit and rethink how we’re doing the training of AI. We really need to be training them more thoughtfully and with more carefully curated data. We could also change the priorities a bit.

I think we need guardrails in the next six months, if not the next six weeks. We need a little bit of a slowdown. We need to do this right. The risk of doing it wrong is too high.

Can we afford to slow down? Will we lose our competitive edge?

To make this work, we need to have some very serious conversations with China. I think the Chinese are more aware of the risks and more ready to deal with them than we have been.

They stopped letting their children have access to AI. We’re just now getting around to that. They’re not allowed to have AI in toys. We still have all kinds of AIs with toys. They’ve been focused on making sure the Ais they’re developing cannot harm their people more than we’ve been.

I would really like to have some conversations with them about what kind of truce or negotiation we need to have so we can all do this more safely.

Can we avoid AI doomsday?

I’m quite sure we can from a technical perspective. I’m not as sure about what the powers that be will do. But I have to be optimistic about it because we absolutely need to have it.��

We can still stop this. We can stop this race to oblivion, but we have to act now.