Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[bruno putzeys]

That's comparing apples and oranges. Neurons are not fully programmable and they're mightily slow (in microprocessor terms). One fast microprocessor could probably simulate thousands of neurons in real time. The real problem is there's just a lot of them and they're hugely interconnected. For the equivalent of the human brain you'd still need a hundred million of such processors, all sharing the same memory.

People are just so fixated on neurons as the prerequisite for synthetic intelligence. It's not because evolution (or whatever the local equivalent in your epistemological neck of the woods) spawned intelligence by clumping together interconnected biological cells that this should be the most effective solution in the electronic domain. Biological cells simply happened to be a technology developed on an earlier occasion which availed themselves to the task. Our technology base is silicon and mathematical abstractions. The optimum way of getting intelligence using those is guaranteed to be something unlike the human brain, unless people remain so self-centered that they believe that the human brain must be the final answer.

Any desktop PC can do millions of floating point calculations pers second and store billions of results. So now we think the ultimate computer is a simulation of the human brain with a short-term memory of about 7 positions that will take up to a minute to multiply two 3-digit numbers together? I think not. If we want anything it's just to add those few specific functions human brains are good at. Putting together logical puzzles is barely one of them. You can't tell whether a solution to a problem is "creative" rather than the result of a blind extensive search, unless you were there to see it happen.

We fear "intelligent machines" because we expect that with intelligence must come forgetfulness, neuroses, delusions of grandeur, a desire to go forth and multiply or otherwise take over the world, the whole gamut of irrational emotional drives that shackle instead of constitute human intelligence.

[Tomas Danko]

bruno putzeys wrote on Thu, 06 January 2011 08:09

That's comparing apples and oranges. Neurons are not fully programmable and they're mightily slow (in microprocessor terms). One fast microprocessor could probably simulate thousands of neurons in real time. The real problem is there's just a lot of them and they're hugely interconnected. For the equivalent of the human brain you'd still need a hundred million of such processors, all sharing the same memory. People are just so fixated on neurons as the prerequisite for synthetic intelligence. It's not because evolution (or whatever the local equivalent in your epistemological neck of the woods) spawned intelligence by clumping together interconnected biological cells that this should be the most effective solution in the electronic domain. Biological cells simply happened to be a technology developed on an earlier occasion which availed themselves to the task. Our technology base is silicon and mathematical abstractions. The optimum way of getting intelligence using those is guaranteed to be something unlike the human brain, unless people remain so self-centered that they believe that the human brain must be the final answer. Any desktop PC can do millions of floating point calculations pers second and store billions of results. So now we think the ultimate computer is a simulation of the human brain with a short-term memory of about 7 positions that will take up to a minute to multiply two 3-digit numbers together? I think not. If we want anything it's just to add those few specific functions human brains are good at. Putting together logical puzzles is barely one of them. You can't tell whether a solution to a problem is "creative" rather than the result of a blind extensive search, unless you were there to see it happen. We fear "intelligent machines" because we expect that with intelligence must come forgetfulness, neuroses, delusions of grandeur, a desire to go forth and multiply or otherwise take over the world, the whole gamut of irrational emotional drives that shackle instead of constitute human intelligence.

I have no luck in finding the source right now, but IIRC scientists thought a neuron didn't carry that much information alone (30 registers or so?) and now it turns out the real answer is more around 1000 registers making it a lot more complex than they thought initially. (i.e. it can perform a lot of stuff without networking)

GHz is not everything, as you know if you remember the SGI RISC laden Onyx machines. Low clock speed, but very powerful indeed. As you point out, not all computational tasks benefit from this kind of a system.

Sometimes faster is, well, faster.

[Jay Kadis]

bruno putzeys wrote on Thu, 06 January 2011 00:09

We fear "intelligent machines" because we expect that with intelligence must come forgetfulness, neuroses, delusions of grandeur, a desire to go forth and multiply or otherwise take over the world, the whole gamut of irrational emotional drives that shackle instead of constitute human intelligence.

What I fear is a device that has self-awareness and will fight for its perceived life, however that might be created.  You can no longer turn it off.

[bruno putzeys]

A desire for self-preservation is by no means a logical consequence of intelligence or even of self-awareness. For self-preservation to be a necessary desire in intelligent beings, all intelligent individuals, bar none, should value their own lives infinitely. We know from practice that this is not the case. We see that self-preservation, like all mental aptitudes, varies across individuals.

So there's no reason for a designed computer program to have such an emotion unless it was deliberately programmed into it. In fact, for a computer program to exhibit *any* emotion it would have to contain the functional equivalent of a reward system. And it would then only exhibit those emotions that are implemented as feedback paths into that reward system. Humans that have one or more of those feedback paths damaged are by no means less intelligent, but they do miss certain emotions.

Only evolved systems naturally have the desire to survive because versions that don't have it are outevolved by those that do. And even then this desire is not absolute: most parents would choose to die if doing so would save the lives of their two children.

I wish more science fiction writers understood the above. We'd get much more interesting AI and robot stories, and rather fewer stereotypical robot-as-menace or robot-as-pathos stories. I'm quite sure it is because of those stories that you have the fear you describe.

[Jon Hodgson]

Jay Kadis wrote on Thu, 06 January 2011 15:17

bruno putzeys wrote on Thu, 06 January 2011 00:09 We fear "intelligent machines" because we expect that with intelligence must come forgetfulness, neuroses, delusions of grandeur, a desire to go forth and multiply or otherwise take over the world, the whole gamut of irrational emotional drives that shackle instead of constitute human intelligence. What I fear is a device that has self-awareness and will fight for its perceived life, however that might be created.  You can no longer turn it off.

Yes you can, that's what the power switch does.

[Jay Kadis]

bruno putzeys wrote on Thu, 06 January 2011 07:40

A desire for self-preservation is by no means a logical consequence of intelligence or even of self-awareness. For self-preservation to be a necessary desire in intelligent beings, all intelligent individuals, bar none, should value their own lives infinitely. We know from practice that this is not the case. We see that self-preservation, like all mental aptitudes, varies across individuals. So there's no reason for a designed computer program to have such an emotion unless it was deliberately programmed into it. In fact, for a computer program to exhibit *any* emotion it would have to contain the functional equivalent of a reward system. And it would then only exhibit those emotions that are implemented as feedback paths into that reward system. Only evolved systems naturally have the desire to survive because versions that don't have it are outevolved by those that do. And even then this desire is not absolute: most parents would choose to die if doing so would save the lives of their two children. I wish more science fiction writers understood the above. We'd get much more interesting AI and robot stories, and rather fewer stereotypical robot-as-menace or robot-as-pathos stories. I'm quite sure it is because of those stories that you have the fear you describe.

I hope you are right.  It is because I studied biology and neuroscience that I have those fears: we do not understand as much as we would like about the evolution of consciousness and what it entails.  We do not have experience with non-evolved systems on which to base our judgements and that is why I would be careful what we create.

[Jay Kadis]

Jon Hodgson wrote on Thu, 06 January 2011 07:54

Jay Kadis wrote on Thu, 06 January 2011 15:17 bruno putzeys wrote on Thu, 06 January 2011 00:09 We fear "intelligent machines" because we expect that with intelligence must come forgetfulness, neuroses, delusions of grandeur, a desire to go forth and multiply or otherwise take over the world, the whole gamut of irrational emotional drives that shackle instead of constitute human intelligence. What I fear is a device that has self-awareness and will fight for its perceived life, however that might be created.  You can no longer turn it off. Yes you can, that's what the power switch does.

It is quite possible for an intelligent device to learn to manipulate or alter its own power switch, isn't it?

[bruno putzeys]

Jay Kadis wrote on Thu, 06 January 2011 17:01

I hope you are right.  It is because I studied biology and neuroscience that I have those fears: we do not understand as much as we would like about the evolution of consciousness and what it entails.  We do not have experience with non-evolved systems on which to base our judgements and that is why I would be careful what we create.

If someone proposed to create intelligence by simply letting it evolve freely in cyberspace I would indeed hope they kept an eye on it. I don't think that it would be very useful because like anything that evolves naturally it'd only be spectacularly good at whatever it ends up doing. It would probably strongly believe that what it does was actually its purpose but I doubt we would agree...

[Jon Hodgson]

Jay Kadis wrote on Thu, 06 January 2011 16:03

Jon Hodgson wrote on Thu, 06 January 2011 07:54 Jay Kadis wrote on Thu, 06 January 2011 15:17 bruno putzeys wrote on Thu, 06 January 2011 00:09 We fear "intelligent machines" because we expect that with intelligence must come forgetfulness, neuroses, delusions of grandeur, a desire to go forth and multiply or otherwise take over the world, the whole gamut of irrational emotional drives that shackle instead of constitute human intelligence. What I fear is a device that has self-awareness and will fight for its perceived life, however that might be created.  You can no longer turn it off. Yes you can, that's what the power switch does. It is quite possible for an intelligent device to learn to manipulate or alter its own power switch, isn't it?

In the movies yes.

In real life, well once it's off it can't turn itself back on, so it would have to have a means of stopping people from turning it off, and an awareness of what the power switch did and its importance, and a self contained source of power (if it's running off the mains, you can always cut the power off further up out of its reach, if it's running on batteries, they go flat).

All this isn't even counting Bruno's point that being self aware doesn't mean it will have an overriding desire for self preservation.

[Jay Kadis]

Jon Hodgson wrote on Thu, 06 January 2011 08:12

All this isn't even counting Bruno's point that being self aware doesn't mean it will have an overriding desire for self preservation.

I suspect self-awareness and the desire for self-preservation go hand-in-hand, but I can only base that on my own experience of life.

[bruno putzeys]

Jay Kadis wrote on Thu, 06 January 2011 17:27

I suspect self-awareness and the desire for self-preservation go hand-in-hand, but I can only base that on my own experience of life.

It can't be too hard to find patients whose desire for self-preservation was knocked out after a stroke, but who are otherwise intelligent and fully self-aware. Such people exhibit extreme risk-taking behaviour not because they're seeking a thrill but because they get no emotional feedback (fear) from it whatsoever. They will profess to understand that they can get themselves killed but don't see what's so bad about that.

[Jay Kadis]

bruno putzeys wrote on Thu, 06 January 2011 08:34

Jay Kadis wrote on Thu, 06 January 2011 17:27 I suspect self-awareness and the desire for self-preservation go hand-in-hand, but I can only base that on my own experience of life. It can't be too hard to find patients whose desire for self-preservation was knocked out after a stroke, but who are otherwise intelligent and fully self-aware. Such people exhibit extreme risk-taking behaviour not because they're seeking a thrill but because they get no emotional feedback (fear) from it whatsoever. They will profess to understand that they can get themselves killed but don't see what's so bad about that.

Entirely possible, but that is a pathological case that doesn't really address the issue of whether the genesis of self-awareness also results in self-preservation since they DID have self-preservation when their brains functioned normally.  It is a question of co-evolution rather than one of separable functionality.

[bruno putzeys]

The pathological case shows that non-self-preservation *can* coexist physically with self-awareness, which is wholly sufficient to prove that self-awareness does not logically require self-preservation. Otherwise such cases could not exist. After all this was your question: whether it may be unavoidable or not.

Whether self-preservation is likely to evolve along with self-awareness is yet another matter. There too there doesn't seem to be a strong link, since self-preservation is the very first thing to evolve, to be followed by self-awareness only a few billion years later. From that it would be illogical to posit that self-preservation was caused by self-awareness. It would be like insisting that a fire was caused by someone throwing in a match while it was already roaring.

But again this is separate from your question of whether perhaps an intelligent computer must necessarily feel the urge to stay alive. "Must necessarily" really means that intelligence cannot ever occur without a desire for self-preservation. Pathological cases are very instructive in this regard because they help pry apart phenomena that usually coexist and therefore seem to be linked.

So long as we make sure that our intelligent computer is pathologically disinterested in self-preservation, there is no problem...

[Jon Hodgson]

bruno putzeys wrote on Thu, 06 January 2011 18:00

The pathological case shows that non-self-preservation *can* coexist physically with self-awareness, which is wholly sufficient to prove that self-awareness does not logically require self-preservation. Otherwise such cases could not exist. After all this was your question: whether it may be unavoidable or not. Whether self-preservation is likely to evolve along with self-awareness is yet another matter. There too there doesn't seem to be a strong link, since self-preservation is the very first thing to evolve, to be followed by self-awareness only a few billion years later. From that it would be illogical to posit that self-preservation was caused by self-awareness. It would be like insisting that a fire was caused by someone throwing in a match while it was already roaring. But again this is separate from your question of whether perhaps an intelligent computer must necessarily feel the urge to stay alive. "Must necessarily" really means that intelligence cannot ever occur without a desire for self-preservation. Pathological cases are very instructive in this regard because they help pry apart phenomena that usually coexist and therefore seem to be linked. So long as we make sure that our intelligent computer is pathologically disinterested in self-preservation, there is no problem...

Also we must add to this the fact that the "life" of an individual "organism" (be it an organic animal or some machine) is actually a pretty fragile thing. Species are tough, individuals are not, so the idea of a robot becoming self aware and taking over the world while humans look on helplessly is good for a bit of movie escapism, but not very realistic. If you  can't reach the power switch, hit it with a baseball bat, if that doesn't work, throw a couple of grenades in its general direction, etc.

[Jay Kadis]

bruno putzeys wrote on Thu, 06 January 2011 10:00

So long as we make sure that our intelligent computer is pathologically disinterested in self-preservation, there is no problem...

I guess after writing device drivers and other software I'm aware of unintended consequences of programing decisions that later show up and are then hard to eradicate.  We're messing with things we are not able to fully predict.  Then there's the problem of malicious intent.  I do agree that self-awareness and self-preservation are probably not inextricably linked since as you say self-preservation is present phylogenetically before self-awareness.