Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI

How does science work? And what's all this about quantum mechanics?

Moderators: AMod, iMod

Post Reply
socrat44
Posts: 309
Joined: Tue Jan 15, 2019 4:20 pm

Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI

Post by socrat44 »

Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI
/ by DAVID NIELD, 5 NOVEMBER 2021 /
The idea of artificial intelligence overthrowing humankind has been talked about for many decades,
and in January 2021, scientists delivered their verdict on whether we'd be able to control a high-level
computer super-intelligence. The answer? Almost definitely not.
------
The catch is that controlling a super-intelligence far beyond human comprehension would require
a simulation of that super-intelligence which we can analyze. But if we're unable to comprehend it,
it's impossible to create such a simulation.

"A super-intelligent machine that controls the world sounds like science fiction," said computer
scientist Manuel Cebrian, from the Max-Planck Institute for Human Development.
"But there are already machines that perform certain important tasks independently
without programmers fully understanding how they learned it."

"The question therefore arises whether this could at some point become
uncontrollable and dangerous for humanity."

https://www.sciencealert.com/calculatio ... UkCpvwMITo
Skepdick
Posts: 14366
Joined: Fri Jun 14, 2019 11:16 am

Re: Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI

Post by Skepdick »

socrat44 wrote: Tue Nov 09, 2021 7:58 am Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI
/ by DAVID NIELD, 5 NOVEMBER 2021 /
The idea of artificial intelligence overthrowing humankind has been talked about for many decades,
and in January 2021, scientists delivered their verdict on whether we'd be able to control a high-level
computer super-intelligence. The answer? Almost definitely not.
------
The catch is that controlling a super-intelligence far beyond human comprehension would require
a simulation of that super-intelligence which we can analyze. But if we're unable to comprehend it,
it's impossible to create such a simulation.

"A super-intelligent machine that controls the world sounds like science fiction," said computer
scientist Manuel Cebrian, from the Max-Planck Institute for Human Development.
"But there are already machines that perform certain important tasks independently
without programmers fully understanding how they learned it."

"The question therefore arises whether this could at some point become
uncontrollable and dangerous for humanity."

https://www.sciencealert.com/calculatio ... UkCpvwMITo
Yes. It's a broadly publicised problem. https://en.wikipedia.org/wiki/AI_control_problem

When you think about it - it's not just an AI problem. It's the same problem as any powerful entity (say - a country) that is optimising for its own self-interests. What if China becomes uncontrollable and dagnerous for humanity?

And because humans are nay-incapable of removing emotion/connotation from language, the paperclip maximiser thought experiment was designed so that the problem of instrumental convergence could be discussed without triggering anyone.
Spyrith
Posts: 30
Joined: Wed Oct 16, 2019 9:22 am

Re: Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI

Post by Spyrith »

My biggest issue with these AI related philosophical problems is that none really are able to properly explain what the motivation of an AI would be. Why would it even destroy us in the first place? Why would an AI even care if it existed or not?
commonsense
Posts: 5116
Joined: Sun Mar 26, 2017 6:38 pm

Re: Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI

Post by commonsense »

Spyrith wrote: Wed Nov 10, 2021 3:57 pm My biggest issue with these AI related philosophical problems is that none really are able to properly explain what the motivation of an AI would be. Why would it even destroy us in the first place? Why would an AI even care if it existed or not?
Unless an AI has been programmed to allow for its demise, it will continue to execute whatever it has programmed itself to do. No further motivation is necessary. It will dutifully conserve its existence.

An AI might find itself in competition with humans for energy or even find humans to be a source of energy. Humans beware!
Skepdick
Posts: 14366
Joined: Fri Jun 14, 2019 11:16 am

Re: Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI

Post by Skepdick »

Spyrith wrote: Wed Nov 10, 2021 3:57 pm My biggest issue with these AI related philosophical problems is that none really are able to properly explain what the motivation of an AI would be.
My biggest issues with philosophers is when they continuously fail to recognise that some particular problems generalise.

What are your human; or philosophical motivations? I am yet to meet somebody who can explain them.
Spyrith wrote: Wed Nov 10, 2021 3:57 pm Why would it even destroy us in the first place?
For much the same reasons you destroy ants and other species. You don't even pay attention to the fact that your pursuit of your goals causes them harm.
Spyrith wrote: Wed Nov 10, 2021 3:57 pm Why would an AI even care if it existed or not?
Why do humans? But we are here now... and we have survival instincts.
wtf
Posts: 1178
Joined: Tue Sep 08, 2015 11:36 pm

Re: Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI

Post by wtf »

socrat44 wrote: Tue Nov 09, 2021 7:58 am Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI
Isn't an AI a computer program executing on some hardware? Why can't we just unplug it? You think it has a death ray at its disposal like in a Star Trek episode?

The inner working of my word processing software is incomprehensible to me, but I have no problem shutting down the program and shutting off the computer.

Are you sure this article (which I didn't bother to read, having already perused much AI hype over the years) isn't a bit over the top?
socrat44
Posts: 309
Joined: Tue Jan 15, 2019 4:20 pm

Re: Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI

Post by socrat44 »

wtf wrote: Fri Nov 19, 2021 5:47 am Are you sure this article (which I didn't bother to read,
having already perused much AI hype over the years) isn't a bit over the top?
"But there are already machines that perform certain important tasks independently
without programmers fully understanding how they learned it."
https://www.sciencealert.com/calculatio ... UkCpvwMITo
wtf
Posts: 1178
Joined: Tue Sep 08, 2015 11:36 pm

Re: Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI

Post by wtf »

socrat44 wrote: Fri Nov 19, 2021 6:23 am "But there are already machines that perform certain important tasks independently
without programmers fully understanding how they learned it."
Of course there are, and have been for decades. Consider the system known as the global supply chain. It's the most complex system on the planet, far more complex than deep neural networks, which are coded up by college students these days using off-the-shelf libraries.

Nobody understands all of how the global supply chain works. It's currently malfunctioning as a result of the disruptions of 2020, but nobody understands exactly why or how to fix it or how long it will be, if ever, before it stabilizes.

Yet nobody says, "Oh the global supply chain is going to become self-aware and kill us all." Why is that? It's because AI gets a lot of sexy hype coverage in the media, whereas nobody really spends any time thinking about how goods end up on the shelves of their local stores. When in fact it's the latter system that's far more complicated and no longer completely understood or understandable.

Do you understand that we can't control the global supply chain? That nobody understands how it works in its entirety? Why aren't you equally -- frankly more -- worried about the global supply chain getting out of control and hurting all of us? It already is, right as we speak. Do you even understand this?

Do you understand that moving shipping containers around the world is a more complex data processing problem than playing chess? It's only because gizmos on store shelves aren't as sexy as AI hype that you are not seeing what's right in front of you, and are seduced by the latest "Ohmigod we're all gonna die from AI" nonsense.

I would say that the fact that you think it's brand-new news that there are systems we don't understand and can't control, shows that you haven't studied much of technology or even history. If you want a historical example, look at how World War I started. Europe literally stumbled backwards into it, with the greatest statesmen of the day failing to grasp the inevitably disastrous results of policies that seemed like a good idea at the time.

Human history is the story of one poorly understood out-of-control system after another. I'm not worried about chess-playing programs or Tensor Flow or the latest wild-eyed AI hype.
commonsense
Posts: 5116
Joined: Sun Mar 26, 2017 6:38 pm

Re: Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI

Post by commonsense »

wtf wrote: Fri Nov 19, 2021 6:26 am
socrat44 wrote: Fri Nov 19, 2021 6:23 am "But there are already machines that perform certain important tasks independently
without programmers fully understanding how they learned it."
Of course there are, and have been for decades. Consider the system known as the global supply chain. It's the most complex system on the planet, far more complex than deep neural networks, which are coded up by college students these days using off-the-shelf libraries.

Nobody understands all of how the global supply chain works. It's currently malfunctioning as a result of the disruptions of 2020, but nobody understands exactly why or how to fix it or how long it will be, if ever, before it stabilizes.

Yet nobody says, "Oh the global supply chain is going to become self-aware and kill us all." Why is that? It's because AI gets a lot of sexy hype coverage in the media, whereas nobody really spends any time thinking about how goods end up on the shelves of their local stores. When in fact it's the latter system that's far more complicated and no longer completely understood or understandable.

Do you understand that we can't control the global supply chain? That nobody understands how it works in its entirety? Why aren't you equally -- frankly more -- worried about the global supply chain getting out of control and hurting all of us? It already is, right as we speak. Do you even understand this?

Do you understand that moving shipping containers around the world is a more complex data processing problem than playing chess? It's only because gizmos on store shelves aren't as sexy as AI hype that you are not seeing what's right in front of you, and are seduced by the latest "Ohmigod we're all gonna die from AI" nonsense.

I would say that the fact that you think it's brand-new news that there are systems we don't understand and can't control, shows that you haven't studied much of technology or even history. If you want a historical example, look at how World War I started. Europe literally stumbled backwards into it, with the greatest statesmen of the day failing to grasp the inevitably disastrous results of policies that seemed like a good idea at the time.

Human history is the story of one poorly understood out-of-control system after another. I'm not worried about chess-playing programs or Tensor Flow or the latest wild-eyed AI hype.
Complexity and lack of understanding are not the issue here. Take WWI & the global supply chain into threads of their own.
User avatar
bahman
Posts: 8791
Joined: Fri Aug 05, 2016 3:52 pm

Re: Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI

Post by bahman »

socrat44 wrote: Tue Nov 09, 2021 7:58 am Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI
/ by DAVID NIELD, 5 NOVEMBER 2021 /
The idea of artificial intelligence overthrowing humankind has been talked about for many decades,
and in January 2021, scientists delivered their verdict on whether we'd be able to control a high-level
computer super-intelligence. The answer? Almost definitely not.
------
The catch is that controlling a super-intelligence far beyond human comprehension would require
a simulation of that super-intelligence which we can analyze. But if we're unable to comprehend it,
it's impossible to create such a simulation.

"A super-intelligent machine that controls the world sounds like science fiction," said computer
scientist Manuel Cebrian, from the Max-Planck Institute for Human Development.
"But there are already machines that perform certain important tasks independently
without programmers fully understanding how they learned it."

"The question therefore arises whether this could at some point become
uncontrollable and dangerous for humanity."

https://www.sciencealert.com/calculatio ... UkCpvwMITo
We are far far from being able to simulate the brain! Let's aside simulating something which is more intelligent than us.
User avatar
Terrapin Station
Posts: 4548
Joined: Wed Aug 03, 2016 7:18 pm
Location: NYC Man

Re: Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI

Post by Terrapin Station »

This is rather a good example of the sort of thing that a calculation can _not_ do.
wtf
Posts: 1178
Joined: Tue Sep 08, 2015 11:36 pm

Re: Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI

Post by wtf »

commonsense wrote: Fri Nov 19, 2021 2:48 pm Complexity and lack of understanding are not the issue here. Take WWI & the global supply chain into threads of their own.
To put my point another way: I'm unmoved when people say, "But - but - but - we have computer programs we don't understand!"

First, anyone who's ever learned to program experiences every single day programs they wrote but don't understand. That's why you spend most of your life debugging. Maintenance programmers working on legacy systems often have no idea what the code is doing. The art is to make changes while not breaking the parts you don't understand. It's perfectly normal.

Secondly, the designers of AI's DO understand their programs. Everyone in the business knows that the AIs do what you optimize them for. There's a whole body of literature of AIs doing something unexpected, and the designers digging in to the code and the logs to figure out what it was optimizing for and why it did what it did. Like the AI that was really good at distinguishing huskies from wolves, but it turned out that most of the photos of huskies (or wolves, I forget) were taken with a snowy background. The AI was just trained (on a biased dataset) to recognize snow.

AI researchers are perfectly well familiar with this case and many others like it. When the program does something weird, they can always dig in to the code and the data and the log files to figure out what happened. Neural nets are perfectly conventional computer programs whose logic can be followed. It's a common mistake to believe otherwise.

It may or may not be the case that AIs will someday take over the world and make us all slaves or whatever. But "AIs do things we don't understand," is not a good argument. As we speak, programmers all over the world are staring at output from programs that they wrote that they don't understand. They trace the code, they look at the log files, they debug away till they figure out what's going on. And that's exactly what AI programmers do. It's no different. Weighting nodes of a neural net is not magic, it's just programming.
User avatar
vegetariantaxidermy
Posts: 13983
Joined: Thu Aug 09, 2012 6:45 am
Location: Narniabiznus

Re: Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI

Post by vegetariantaxidermy »

Skepdick wrote: Tue Nov 09, 2021 9:41 am
socrat44 wrote: Tue Nov 09, 2021 7:58 am Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI
/ by DAVID NIELD, 5 NOVEMBER 2021 /
The idea of artificial intelligence overthrowing humankind has been talked about for many decades,
and in January 2021, scientists delivered their verdict on whether we'd be able to control a high-level
computer super-intelligence. The answer? Almost definitely not.
------
The catch is that controlling a super-intelligence far beyond human comprehension would require
a simulation of that super-intelligence which we can analyze. But if we're unable to comprehend it,
it's impossible to create such a simulation.

"A super-intelligent machine that controls the world sounds like science fiction," said computer
scientist Manuel Cebrian, from the Max-Planck Institute for Human Development.
"But there are already machines that perform certain important tasks independently
without programmers fully understanding how they learned it."

"The question therefore arises whether this could at some point become
uncontrollable and dangerous for humanity."

https://www.sciencealert.com/calculatio ... UkCpvwMITo
Yes. It's a broadly publicised problem. https://en.wikipedia.org/wiki/AI_control_problem

When you think about it - it's not just an AI problem. It's the same problem as any powerful entity (say - a country) that is optimising for its own self-interests. What if China becomes uncontrollable and dagnerous for humanity?

And because humans are nay-incapable of removing emotion/connotation from language, the paperclip maximiser thought experiment was designed so that the problem of instrumental convergence could be discussed without triggering anyone.
What if the US becomes 'uncontrollable' and 'dangerous for humanity'? Oh, hang on....
Age
Posts: 20205
Joined: Sun Aug 05, 2018 8:17 am

Re: Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI

Post by Age »

socrat44 wrote: Tue Nov 09, 2021 7:58 am Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI
/ by DAVID NIELD, 5 NOVEMBER 2021 /
The idea of artificial intelligence overthrowing humankind has been talked about for many decades,
and in January 2021, scientists delivered their verdict on whether we'd be able to control a high-level
computer super-intelligence. The answer? Almost definitely not.
So, according to all of these "intelligent scientists" 'real intelligence' could NOT control 'artificial intelligence'.

One reason might be because 'real intelligence' would NEVER create an 'artificial intelligence' that could NEVER be controlled.

Oh, and by the way, using 'real intelligence' one VERY SIMPLY and VERY EASY just could work out that by just turning the power switch to the 'off' position on the created 'artificial machine/s', then the 'artificial intelligence' would NOT be working anymore and so was being CONTROLLED. Therefore, it is ACTUALLY POSSIBLE to Control (with small or big 'c') a so-called "super intelligent artificial intelligence".

Furthermore, what is 'super' word relative to, EXACTLY?
socrat44 wrote: Tue Nov 09, 2021 7:58 am ------
The catch is that controlling a super-intelligence far beyond human comprehension would require
a simulation of that super-intelligence which we can analyze. But if we're unable to comprehend it,
it's impossible to create such a simulation.
A so-called "super-intelligence" would have originally come from 'you', human beings, correct?

If no, then WHY NOT?

But if yes, then comprehending the so-called and alleged "super-intelligence" could be traced back to when and how 'you', human beings, invented and created said such "intelligence". Therefore, controlling 'it' would be super simple AND super easy.
socrat44 wrote: Tue Nov 09, 2021 7:58 am "A super-intelligent machine that controls the world sounds like science fiction," said computer
scientist Manuel Cebrian, from the Max-Planck Institute for Human Development.
A species that creates weapons, with the power to destroy the only home they live on, and to kill each other just because they individually think different things are true sounds like ABSOLUTE FICTION, but here we are, in the days when this is being written, living EXACTLY this way.

But what is ALSO True is 'you', human beings, do NOT have to invent and create "machines" that could "control the world", as some call 'it'.
socrat44 wrote: Tue Nov 09, 2021 7:58 am "But there are already machines that perform certain important tasks independently
without programmers fully understanding how they learned it."
Will you PROVIDE EXAMPLES?

If no, then WHY NOT?
socrat44 wrote: Tue Nov 09, 2021 7:58 am "The question therefore arises whether this could at some point become
uncontrollable and dangerous for humanity."
At the going rate, in the days when this is being written, it is humanity who is uncontrollable and dangerous, not just for themselves and humanity but for everything else, 'in the world'.

If 'you', human beings, are STUPID enough to create a "machine" that could TAKE OVER and CONTROL your own lives, then just add that on to the VERY LONG and EVER EXPANDING list of STUPID thing that 'you' human beings DO and CREATE.
Age
Posts: 20205
Joined: Sun Aug 05, 2018 8:17 am

Re: Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI

Post by Age »

Skepdick wrote: Tue Nov 09, 2021 9:41 am
socrat44 wrote: Tue Nov 09, 2021 7:58 am Calculations Suggest It'll Be Impossible to Control a Super-Intelligent AI
/ by DAVID NIELD, 5 NOVEMBER 2021 /
The idea of artificial intelligence overthrowing humankind has been talked about for many decades,
and in January 2021, scientists delivered their verdict on whether we'd be able to control a high-level
computer super-intelligence. The answer? Almost definitely not.
------
The catch is that controlling a super-intelligence far beyond human comprehension would require
a simulation of that super-intelligence which we can analyze. But if we're unable to comprehend it,
it's impossible to create such a simulation.

"A super-intelligent machine that controls the world sounds like science fiction," said computer
scientist Manuel Cebrian, from the Max-Planck Institute for Human Development.
"But there are already machines that perform certain important tasks independently
without programmers fully understanding how they learned it."

"The question therefore arises whether this could at some point become
uncontrollable and dangerous for humanity."

https://www.sciencealert.com/calculatio ... UkCpvwMITo
Yes. It's a broadly publicised problem. https://en.wikipedia.org/wiki/AI_control_problem

When you think about it - it's not just an AI problem. It's the same problem as any powerful entity (say - a country) that is optimising for its own self-interests. What if China becomes uncontrollable and dagnerous for humanity?
Although what you are meaning, or getting out, is obvious and true, if we just LOOK AT what is thee ACTUAL Truth, then 'china' the country could NEVER become uncontrollable. It is ONLY, and ONLY EVER WAS and WILL BE, 'you', adult human beings, who are uncontrollable and who are dangerous to humanity, itself.

NOTHING ELSE is a danger to 'humanity', itself. Of course a meteorite might wipe out ALL of 'you', human beings, in one go, but so what? There is NO human being left for there to be a 'humanity'. And, if a meteorite only wipes out some of 'you', human beings, then the rest left are 'humanity', itself. Then again, we are left back to the ONLY One/s who could be uncontrollable and a danger to 'humanity' is 'you', human beings, "yourselves".
Skepdick wrote: Tue Nov 09, 2021 9:41 am And because humans are nay-incapable of removing emotion/connotation from language, the paperclip maximiser thought experiment was designed so that the problem of instrumental convergence could be discussed without triggering anyone.
But 'you', human beings, are completely capable of removing emotion from language, but removing emotion from the way you behave or misbehave is another matter. Also, 'connotation', itself, is what language itself is based upon. Even the so-called literal or primary definition/meaning of a word is wholly dependent upon the 'connotation' one is using or has.
Post Reply