Can AI Become Conscious? - MindGods
https://cacm.acm.org/news/244846-can-ai-become-conscious/fulltext
======
_Microft
I can not _not_ think about J. Schmidhuber's thoughts on consciousness
whenever the topic comes up:

 _As we interact with the world to achieve goals, we are constructing internal
models of the world, predicting and thus partially compressing the data
history we are observing. If the predictor /compressor is a biological or
artificial recurrent neural network (RNN), it will automatically create
feature hierarchies, lower level neurons corresponding to simple feature
detectors similar to those found in human brains, higher layer neurons
typically corresponding to more abstract features, but fine-grained where
necessary. Like any good compressor, the RNN will learn to identify shared
regularities among different already existing internal data structures, and
generate prototype encodings (across neuron populations) or symbols for
frequently occurring observation sub-sequences, to shrink the storage space
needed for the whole (we see this in our artificial RNNs all the time). Self-
symbols may be viewed as a by-product of this, since there is one thing that
is involved in all actions and sensory inputs of the agent, namely, the agent
itself. To efficiently encode the entire data history through predictive
coding, it will profit from creating some sort of internal prototype symbol or
code (e. g. a neural activity pattern) representing itself. Whenever this
representation becomes activated above a certain threshold, say, by activating
the corresponding neurons through new incoming sensory inputs or an internal
‘search light’ or otherwise, the agent could be called self-aware. No need to
see this as a mysterious process — it is just a natural by-product of
partially compressing the observation history by efficiently encoding frequent
observations._

[https://old.reddit.com/r/MachineLearning/comments/2xcyrl/i_a...](https://old.reddit.com/r/MachineLearning/comments/2xcyrl/i_am_j%C3%BCrgen_schmidhuber_ama/cp44iba/)

~~~
Xenograph
You are conflating consciousness[1] with self-awareness[2]. They are two
distinct ideas.

In fact the second/third sentence on the Wikipedia page for self-awareness is:

> It [self-awareness] is not to be confused with consciousness in the sense of
> qualia. While consciousness is being aware of one's environment and body and
> lifestyle, self-awareness is the recognition of that awareness.[2]

[1]:
[https://en.wikipedia.org/wiki/Consciousness](https://en.wikipedia.org/wiki/Consciousness)

[2]: [https://en.wikipedia.org/wiki/Self-
awareness](https://en.wikipedia.org/wiki/Self-awareness)

~~~
breckinloggins
Exactly.

One of my more vivid memories is excitedly opening my copy of Jayne's "The
Origins of Consciousness in the Breakdown of the Bicameral Mind" and being
dismayed as it slowly dawned on me that Jayne's entire book was based on this
category error.

For the life of me I can't understand why people don't understand this
distinction (or choose to ignore it).

I can only assume that people ignore the very obvious "magical thing" that is
happening in their own heads due to ideological commitments to physicalism or
something.

~~~
tasty_freeze
What is this magical thing that is so obvious to you? Something may seem
magical without being magical. To the person who survives a tornado while all
his neighbors die, the sense they were chosen to live and have a special
purpose can be compelling, but is false.

Materialism can't explain everything, but there is yet to be any proof that
magic explains anything. Lacking that, assuming a physical basis as the
explanation is prudent.

~~~
dr_dshiv
"assuming a physical basis as explanation is prudent" yeah, good luck making
physicality relevant for most explanations. My wife asks why I didn't put the
laundry away, for instance. "Physics!" I declare triumphantly.

Now, if I put music on and rub her back a bit... Well, ha! There's the magic,
my good sir.

Imagine walking into a film studio with the idea of film production based
exclusively on the principles of physics. No. Magic and physics are no more
opposed than biology and physics. When hidden (psychological?) forces clearly
underlie economically valuable phenomena (architecture, music, media, etc)
respecting the magic is closer to the truth than reducing it to physics.
Physics just isn't the right explanatory level.

If you'd like to learn more about definitions of magic that aren't "phenomena
that don't actually exist", I recommend Penelope Gouk's work on music, science
and natural magic in the 17th century. It's about the magic involved at the
formation of the Royal Society.

~~~
Koshkin
physicality _n._ obsession with physical urges

On the subject, you are making a good point comparing social behavior with
mental activity. Just like there is no “magic” in the way people interact with
each other, there is no magic in how neurons do the same.

~~~
dr_dshiv
> Just like there is no “magic” in the way people interact with each other,
> there is no magic in how neurons do the same.

Except I am arguing that there _is most definitely_ magic in human
interaction.

------
Animats
It's nice that someone now has the neural wiring diagram for part of a mouse
brain. But we have the full wiring for a thousand-cell nematode, and OpenWorm
still doesn't work very well.[1] OpenWorm is trying to get a neuron and cell
level simulator for what's close to the simplest creature with a functioning
nervous system - 302 neurons and 85 muscle cells. That needs to work before
moving to more complexity.

[1] [http://openworm.org/](http://openworm.org/)

~~~
jimfleming
> That needs to work before moving to more complexity.

It really depends on what level of abstraction you care to simulate. OpenWorm
is working at the physics and cellular level, far below the concept level as
in most deep learning research looking to apply neuroscience discoveries, for
example. It’s likely easier to get the concepts of a functional nematode model
working or a functional model of memory, attention, or consciousness than a
full cellular model of these.

More specifically, a thousand cells sounds small in comparison to a thousand
layer ResNet with millions of functional units but the mechanics of those
cells are significantly more complex than a ReLU unit. Yet the simple ReLU
units are functionally very useful and can do much more complex things that we
still can’t simulate with spiking neurons.

The concepts of receptive fields, cortical columns, local inhibition, winner-
take-all, functional modules and how they communicate / are organized may all
be relevant and applicable learnings from mapping an organism even if we can’t
fully simulate every detail.

~~~
Reelin
The trouble is that (assuming sufficient computational power) if we can't
simulate it then we don't really understand it. It's one thing to say "that's
computationally intractable", but entirely another to say "for some reason our
computationally tractable model doesn't work, and we don't know why".

Present day ANNs may well be inspired by biological systems but (as you noted)
they're not even remotely similar in practice. The reality is that for a
biological system the wiring diagram is just the tip of the iceberg - there's
lots of other significant chemical things going on under the hood.

I don't mean to detract from the usefulness of present day ML, just to agree
with and elaborate on the original point that was raised (ie that "we have a
neural wiring diagram" doesn't actually mean that we have a complete
schematic).

~~~
jimfleming
I'm aware of that and I've done quite a bit of work on both spiking neural
networks and modern deep learning. My point is that those complexities are not
required to implement many important functional aspects of the brain: most
basically "learning" and more specifically, attention, memory, etc.
Consciousness may fall into the list of things we can get functional without
all of the incidental complexities that evolution brought along the way. It
may also critically depend on complexities like multi-channel chemical
receptors but since we don't know we can't say either way.

It's a tired analogy but we can understand quite a lot about flight and even
build a plane without first birthing a bird.

~~~
philipov
It's easy to see if something flies or not. How would you know if your
simulation is conscious?

~~~
mirimir
This is, of course, the key problem.

I mean, I know that I'm conscious. Or at least, that's how it occurs for me.

But there's no way to experience another's consciousness. So behavior is all
we have. And that's why we have the Turing test. For other people, though,
it's mainly because they resemble us.

------
hackinthebochs
> Any AI that runs on such a chip, however intelligent it might behave, will
> still not be conscious like a human brain.... No. It doesn't matter whether
> the Von Neumann machine is running a weather simulation, playing poker, or
> simulating the human brain; its integrated information is minute.
> Consciousness is not about computation; it's a causal power associated with
> the physics of the system.

There are serious problems with this. Koch will have to explain how a
simulation of a brain can reproduce to perfect detail all the possible
behaviors of a brain without having an equivalent integrated information.
Presumably integrated information is a property of the organization of a
process, and as such it should have consequences for its set of possible
behaviors. So if the von Neumann system has constrained integrated
information, its behavior should be constrained as well in some externally
identifiable way. But by assumption the simulation could be arbitrarily good.
How does Koch break this tension?

The other glaring issue is the fact that consciousness under this view has no
explanatory power for the system's behavior. If a non-conscious system can
reproduce exactly the behavior of a conscious system, then there is nothing
_informative_ to the behavior from the property of conscious; it is entirely
incidental to a conscious system's behavior. It doesn't explain why a
conscious subject winces with pain after touching a hot stove, nor why it
learns to never touch a hot stove bare-handed again. That's a pill too big to
swallow.

~~~
KhoomeiK
Yeah, I didn't really understand the "physics" requirement as well. Sure if
you're measuring the integrated information of the hardware architecture it'll
be quite low, but why isn't the integrated information of the abstracted
neural simulation software (which is presumably significantly higher)
relevant?

~~~
mannykannot
Koch does not even seem to be consistent with hardware. A synapse is
sufficiently complex, yet a transistor is not? He seems to be offering a
peculiar panpsychism in which almost everything, _except_ transistors, has the
potential for consciousness. It seems to be a remarkably tendentious view.

~~~
mannykannot
...and thinking about this some more, one does not, of course, need
transistors to make a Turing-equivalent machine - one could, in principle, use
neurons and synapses... or people. Koch's position, at least as he has
presented it in this interview, seems utterly incoherent.

~~~
hackinthebochs
Agreed. "Integrated information" is intrinsically substance independent. Yet
for some reason he doesn't grasp the obvious fact that a Turing machine
running the right program would exhibit the same level of integrated
information. His argument seems to require an extra ingredient to exclude
programs running on computers but it goes unstated. I have yet to see
something that resolves the obvious incoherence.

------
NoodleIncident
""" Let's imagine that we simulate the brain in all biological details on a
supercomputer. Will that supercomputer be conscious?

No. It doesn't matter whether the Von Neumann machine is running a weather
simulation, playing poker, or simulating the human brain; its integrated
information is minute. Consciousness is not about computation; it's a causal
power associated with the physics of the system. """

I don't understand how anyone could answer this question this way. It's
practically a tautology; if the simulation is accurate, then the person you're
simulating is conscious. In other contexts, it's taken for granted that if
this entire planet were running on a simulation, we wouldn't even be able to
tell.

Less hypothetically, this "consciousness-meter" would give a different value
for a physical chip, and a perfectly accurate emulation of that chip. They're
doing the exact same thing, and your meter gives a different number? Why is
anyone taking this person seriously?

~~~
Yajirobe
To quote John Searle:

"Nobody supposes that the computational model of rainstorms in London will
leave us all wet. But they make the mistake of supposing that the
computational model of consciousness is somehow conscious. It is the same
mistake in both cases."

~~~
bloak
According to Wikipedia, John Searle is known for sexual assault, sexual
harassment, and the "Chinese room" argument. I'm not sure with which of these
he's done the most harm, but I suspect he's done the most harm with his
"Chinese room" argument, primarily through wasting people's time. See
[https://en.wikipedia.org/wiki/Talk:Chinese_room](https://en.wikipedia.org/wiki/Talk:Chinese_room)
for some discussion, if you must, but I'd rather recommend people do something
more useful instead.

~~~
Yajirobe
Could you point out exactly where I can see a sound refutation to the 'Chinese
room'? I followed the wikipedia link but did not see any serious attempt to
refute it - just one guy echoing what you've said.

You sound a bit like many of the people who completely miss the point of the
argument and insult it (or its author).

~~~
hackinthebochs
The Chinese room argument says that syntax cannot capture semantics since a
man blindly executing rules to process Chinese symbols would not understand
Chinese. But the man isn't the the system that is purported to understand
Chinese. That's like saying the CPU in a computer doesn't have memory so your
computer doesn't have memory. But of course that's wrong, other components of
the system provide the function of memory. Using properties of the CPU to
determine properties of the system as a whole is a mistake.

You might recognize this as the system's reply for which Searle has a
response. But his response is insufficient to save the argument as it merely
equivocates on what "the system" is. The system is the set of rules and the
database of facts, some input/output mechanism, and some central processor to
execute the rules against the symbols. The system's reply says that the man
blindly executing instructions is not identical to the system that understands
Chinese, namely the entire room. Thus properties of the man have no bearing on
the properties of the system; that the man does not understand Chinese has no
bearing on whether the system as a whole understands Chinese.

But the man is still not "the system" when he memorizes the instructions and
leaves the room. The system is some subset of the man, specifically only those
brain networks required to carry out the processing of the symbols.
Importantly, these brain networks are different than what we take the man to
be. It is true that the man doesn't speak Chinese, but since the system
processing symbols is still not identical to the man, properties of the man do
not bear on properties of the system. So still, the fact that the man blindly
executing instructions doesn't speak Chinese does not entail that the system
doesn't speak Chinese.

------
javajosh
Am I the only one who thinks the answer is _obviously_ yes, and that it's kind
of a waste of time to ponder it? It reminds me of discussions of free will --
completely boring because, even if we are all finite state machines (and we
are, in some sense) then random inputs would mean we have free will and random
inputs are assured by, for example, chaotic variations in the weather. In the
same way, consciousness is pretty clearly characterized by a pattern that is
constantly in motion, sensing both itself and the outside world. The only
reason we think there's something magical about the phenomena is the residue
of traditional religious beliefs.

One point against the traditional belief is that if you've ever read Aristotle
you might be shocked to learn that the Greeks of antiquity believed that
"soul" was quite physical, expressing itself in blood, semen, etc. The
medieval characterization of mental illness as being an "imbalance of the
humors" comes from this early idea about how we work. Needless to say, the
definition of soul has retreated entirely into the metaphysical, where it's
cannot be disproven, by definition.

~~~
gfodor
It's not obviously yes, since we don't have an explanation for the experience
of consciousness. Qualia truly is magical, unless you are in fact a p-zombie
I'm talking to.

We also don't have an explanation for what I call 'the harder problem'
[https://twitter.com/gfodor/status/1225230653932761088](https://twitter.com/gfodor/status/1225230653932761088)

~~~
javajosh
I don't have an explanation for the experience of digesting food at the
cellular level either. But I still do it. You don't have to _explain_
conciousness to have it.

As for qualia, I'm reminded of the people coming to my door asking, "Do you
believe in Jesus?" I reply, "Do you?" and when they assure me fervently that
they do, I follow up with, "Well, okay, how do you know you do?" I think you
could do the same with r/Jesus/qualia/g.

~~~
Trasmatta
I don't understand where you're going with the analogy of digestion. Nobody is
saying you have to be able to explain consciousness for it to exist. And
science can easily explain the mechanism of digestion, but consciousness
remains a mystery.

~~~
javajosh
I don't think you need to _explain_ consciousness to have it, or to produce it
in an AI. I suspect that an AI consciousness will be grown from a (reasonably
complex) seed, more than constructed. And it will probably require a huge
amount of slow, human interaction at first. (Ted Chiang has a wonderful
novella called _The Lifecycle of Software Objects_ that is the only SF that I
know that explores this possibility).

~~~
gfodor
You suspect != it’s obvious.

~~~
javajosh
A couple of your comments seem to imply you think my claim is that concious AI
is objectively obvious. I was only asking if there are others for who it seems
obvious -- clearly there are many that don't, and I don't take issue with them
or their position! It's just I always read articles like this and it just
strikes me as odd. I used an analogy in another comment about philosophizing
about the possibility of heavier-than-air flight prior to us actually doing
it. Which was a strange thing to get all worked up about, in hindsight.

------
omarhaneef
All this is underpinned by your definition of conscious.

I recall seeing Koch in debate with a philosopher (it was long ago I can't
recall but one of the big ones: maybe Searle or Dennet) and they were talking
past each other.

For Koch, at least in this debate, consciousness was something akin to
attention. For the philosopher, it was something else entirely (if I
remembered, I might be able to guess who it was).

Consciousness could mean: making a decision as opposed to having it pre-
ordained, or the experience of your senses, or "feelings" or knowing you are a
self, or who knows what else.

It isn't just a computer science issue, it is a philosophical and linguistic
one too: just what do you mean by the word.

~~~
mannykannot
Scott Aaronson and Giulio Tononi debated IIT five years ago. Personally, I
think Aaronson had by far the stronger argument, and my much less analytical
response to IIT is that it broadens the definition of consciousness to the
point of being uninteresting and unhelpful.

From the article (Christof Koch's words):

"The theory fundamentally says that any physical system that has causal power
onto itself is conscious. What do I mean by causal power? The firing of
neurons in the brain that causes other neurons to fire a bit later is one
example, but you can also think of a network of transistors on a computer
chip: its momentary state is influenced by its immediate past state and it
will, in turn, influence its future state."

In this view, a building with a thermostat would seem to be conscious to some
degree. I very much doubt that any amount of study of simple systems that fit
this definition will tell us anything useful about the sort of consciousness
that is displayed by, for example, humans.

[https://www.scottaaronson.com/blog/?p=1799](https://www.scottaaronson.com/blog/?p=1799)

[https://www.scottaaronson.com/blog/?p=1823](https://www.scottaaronson.com/blog/?p=1823)

~~~
colordrops
> In this view, a building with a thermostat would seem to be conscious to
> some degree.

I don't think this is as overly broad or bad of a definition as you presume. I
recall sitting at a laundromat one day, in a somewhat sleepy state
daydreaming, with the rhythm of the washing machine in front of me playing
through my mind. My subconscious told me that the machine was conscious. I dug
deeper into that intuition, and realized I was seeing a bit of the inner
workings of the mind of the engineer that designed the algorithm. Not a
perfect example, as I didn't know if the alg was closed-loop, but in any case,
it made me realize that our minds are just large bodies of these algorithms
working together. We have some additional oracles, random data generators, and
probabilistic mechanisms thrown in as tools, but consciousness really is
mechanistic and pluralistic.

To recreate it requires not only the qualitative aspects of a being that can
think, but also sufficient (read: vast) quantity of systems to get to a useful
general purpose thinking machine. There is no "soul" or special sauce or
singular definition of consciousness. It's an illusion.

~~~
mewpmewp2
I personally don't understand how a logically and rationally thinking person
could get to a conclusion anything other than that we are a combination of
algorithms. So essentially we can in theory be simulated and there is nothing
special about us. In this case the machine simulating us would be as conscious
as ourselves.

It seems any other conclusion is just fooling oneself.

~~~
goatlover
Replace algorithms with math and you'll see that you're making a rather strong
metaphysical claim that is essentially Platonism.

~~~
mannykannot
One reaction to that statement might be "so be it."

I did not, however, come here to defend the idea that consciousness is an
algorithm. Personally, I consider it rather more plausible that a computer
plus a random number generator might achieve consciousness through simulating
the sorts of physical processes that occur in a brain, and if it is a 'true'
random number generator, rather than a deterministic PRNG, then we are no
longer talking about math, at least in the sense that you use to conclude
Platonism.

One might respond by saying that if the universe is deterministic, then there
is no such thing as a 'true' random number generator, as distinct from a PRNG.
My understanding of the philosophical implications of QM, and its competing
interpretations, is insufficient for me to be sure whether the universe as we
experience it is deterministic, but regardless of whether it is or is not, I
suspect that the true hard problem of consciousness is 'why do we (feel that
we) have free will?'

~~~
goatlover
> I suspect that the true hard problem of consciousness is 'why do we (feel
> that we) have free will?'

Free will and consciousness are separate issues. You can potentially have one
and not the other. What's important for free will is whether choice is free
and what that freedom amounts to. What's important to consciousness is
subjectivity. You can feel pain while not having a choice about feeling that
pain, say if you were restrained and unable to do anything about it, just to
give an example.

And potentially, a robot could be free to make choices while not feeling
anything.

~~~
mannykannot
I doubt that 'will', as it is commonly conceived, has any meaning, or at least
a very different one, for a non-conscious agent. It's similar to the way i
think IIS is clouding rather than clarifying the issue. For related reasons, I
feel that compatibilist positions on the issue are largely avoiding it.

As for subjectivity, I had more to say about it here:
[https://news.ycombinator.com/item?id=23162714](https://news.ycombinator.com/item?id=23162714)

Regardless of whether I am right or wrong on this, I think my other points in
this post (platonism, etc.) are independent of it.

------
eximius
Two thoughts:

1) without definition of 'AI' or 'conscious', all answers are correct. They do
go into a bit of depth on what 'conscious' means, but not what 'AI' means.

2) there are a lot of related words here: conscious, self-aware, sentient,
sapient, 'narrow AI', etc.

> Let's imagine that we simulate the brain in all biological details on a
> supercomputer. Will that supercomputer be conscious?

They say no, but fail to explore the more interesting corollary that the
supercomputer won't be conscious but the _program_ is, by their definition.
It's just one level of abstraction up from Matter -> Brain -> Mind to Matter
-> Computer -> Simulated Brain -> Mind.

I do find this definition of consciousness interesting from an ethical
perspective, given their final thought experiments. It reminds me of the
Presger, an alien race from the Ancillary Justice trilogy. They have the
concept of 'Significant' species that have rights and if you aren't, you're
basically inanimate matter or bugs or whathaveyou.

~~~
runT1ME
Thought provoking stuff! How can someone talk about conciousness _existing_
then, without presupposing dualism?

~~~
eximius
On the contrary, they are explicitly saying that consciousness is a function
of the substrate, making them indivisible. One is a property of the other, by
their reckoning.

------
dentalperson
Two philosophers are arguing. One says "If a tree falls in the woods, there is
no sound, because there is no one to hear it." The other says "No, that's not
correct. There is sound, because vibrations are created in the air." The first
philosopher agrees that there are vibrations in the air, and the second agrees
that without a human around, there will be no human perception of the
vibrations. Yet they continue to argue without paying much attention to the
fact that they are disputing the definition of the word 'sound'.

I think Eliezer Yudkowsky suggested 'tabooing' the word in question for
situations like these, to force the discussion on the logical arguments
instead of the symbolic dead ends. In that case the word 'sound' was charged
and raised emotions. 'Consciousness' and 'free will' certainly fall into that
category.

~~~
curiousgal
A bit tangent but I've come the same conclusion listening to Supreme Court
oral arguments, most of them are based on contradicting definitions of certain
words or expressions.

~~~
contravariant
That might be so but in the case of court cases this is the somewhat
inevitable result of written laws. It's practically impossible to write a law
that leaves no room for ambiguity and indeed resolving that ambiguity is one
of the main tasks of the Supreme Court.

------
ericmay
I've been thinking about this a lot, and I don't think AI will ever become
conscious in the same way that humans are, rather, it'll be conscious on its
own terms. Just like I can't know the mind of my wife, I can't know the mind
of my cat, I also can't know the mind of an AI.

Depending on who you ask, a monkey or cat isn't conscious. From my point of
view, I don't think you can really know that or recognize it. It's just a
matter of degree of consciousness. I think it's safe to say that mammals are
conscious to some extent. They have emotions, dreams, communication
techniques, etc. we just have those (to some extent again) moreso than they do
or we have them in different ways.

I think a question to ask is at what level or organization do we recognize
self-direction? Am I conscious because I think so? What does that say about
the bacteria that live inside of me that I rely on, or the individual neurons
in my brain?

If both I and a dolphin are mostly the same, we have brains with neurons, we
have blood cells, etc. how can you truly differentiate what is conscious and
not? Even if you speak to another human it's not completely possible to say
that they are conscious with certainty - only with what's most useful in day-
to-day life.

At what level of circuity do we consider AI to be conscious? When it completes
arbitrarily constructed tasks by humans? When it "feels"? How would you
differentiate between sufficiently complex AI? Is there just AI or not? why?

/rambles

~~~
knodi123
> I don't think AI will ever become conscious in the same way that humans are,
> rather, it'll be conscious on its own terms.

I think that people who are attempting to simulate a human brain using an
utterly biomimetic design stand a good chance of artificially creating
something that is conscious in the same way that humans are. I also think it's
possible they may be able to achieve this before they full understand how the
human brain works. i.e. if you copy the design accurately enough, the machine
may work even if you don't know how.

The resulting consciousness could theoretically be totally self-aware, but no
more capable than we ourselves of modifying its own programming with
intentionality and purpose. i.e. not the singularity.

I think there should be two different concepts of AI- "a consciousness using
the same processes and design as our own", and "an essentially alien
consciousness that fully understands itself". And I suspect that even if some
engineering genie _gave_ us the first kind of AI, we'd be no closer to
developing the second kind.

~~~
ericmay
> a consciousness using the same processes and design as our own

What is the meaningful distinction here? That it wasn't created through sex?

~~~
knodi123
Brain simulation in a computer, for instance. Like

[https://en.wikipedia.org/wiki/OpenWorm](https://en.wikipedia.org/wiki/OpenWorm)

The meaningful distinction is "not a biological organism".

------
Trasmatta
The idea of making conscious AI scares me, because we could potentially create
an AI that just experiences constant suffering at a level unimaginable to
humans. Really scary from an ethical perspective.

If integrated information theory is true, that gives me some comfort, since it
says AI built on our current architecture is unlikely to ever be conscious.
Although like the article said, some alternative architectures could
theoretically have a higher degree of consciousness under IIT.

I feel like there are some big ethical questions around AI that revolve around
more than just the standard "how do we create an AI that won't destroy us
all".

~~~
0x8BADF00D
> The idea of making conscious AI scares me, because we could potentially
> create an AI that just experiences constant suffering at a level
> unimaginable to humans. Really scary from an ethical perspective.

Some schools of thought posit that everything has consciousness. Including
things like rocks. Even these things suffer; you just can’t hear the screams.

~~~
jameslevy
A rock seems unlikely to be conscious in any real way. Something as complex as
a star, however...maybe all of the humans worshipping the sun throughout many
thousands of years were onto something after all.

~~~
Trasmatta
Is a star particularly complex, though? Is there much going on besides a lot
of thermonuclear fusion? (Disclaimer: I don't know much about stars or
astrophysics, so this question is coming from a place of ignorance.)

~~~
hoseja
Yes, incredibly. Magnetohydrodynamics in a star are incomprehensibly complex.

------
proc0
> Consciousness is not about computation;

Isn't there a lot of evidence to the contrary? We have general maps of our
body parts in specific brain areas, we know if those areas get damaged it
compromises specific functions like language, hearing etc. Then we have things
like short term memory, long term memory, image processing (subconsciously)
that can be hacked with visual illusions, and many more observations that
clearly point to the brain undergoing information processing as described by
information theory.

~~~
hackerman123469
We can partially map certain regions of the brain to certain behaviors too.

Some mental illnesses can be classified as "miscomputations" in our brains as
well.

------
snovv_crash
A computer will be able to think in the same way a submarine can swim.

Thinking is something we've anthropomorphized, and the moment a computer can
do something we previously considered 'thinking', we'll move the goalposts
again.

~~~
TimTheTinker
"Moving the goalposts" is only something we talk about when two ideological
camps disagree about something.

Ideology and beliefs aside, I think it's absolutely fine to change one's
definition of something when one learns more. I think we all can benefit from
better definitions in our thoughts and conversations.

What you may call "moving goalposts" may actually be incremental refinement of
an idea -- or a belief that is increasing in sophistication (or absurdity in
some cases!) as more is understood.

~~~
ggggtez
The two camps are "those who believe humans are special" and "those who
believe strong AI can exist". The only reason you see people even arguing
computers can never be conscious, is so we don't have to face the possibility
of creating and enslaving a machine callable of suffering.

The authors are certainly moving the goal posts when they concede that 4/5
humans in a vegetative state have no consciousness, but a full simulation of a
healthy human mind never should be considered self aware. And yet in the same
breath claim this lack of consciousness allows you to destroy a robot in your
front yard, but stay startlingly silent on what that means for those same in
the vegetative state.

------
Barrin92
> _" The more the current state of a system specifies its cause, the input,
> and its effect, the output, the more causal power the system has. Integrated
> information is a number that can be computed. The bigger the number for a
> system, the larger its integrated information, and the more conscious the
> system is._"

I'm generally sympathetic to the idea that consciousness and "computational
power" are not identical but the larger problem with all of these theories in
my opinion is that they're really just stories that clarify intuitions one has
about consciousness.

I think the problem with the consciousness question is really that it's hard
to probe into what consciousness actually is. It's completely possible to give
different accounts of conscious experience even though systems behave the
same, including that consciousness doesn't exist at all, and really there's no
empirical way to agree or disagree.

I've started to think of expressions about consciousness more or less the same
way emotivists considers ethical statements. Emotivists argue that saying
something like ""You acted wrongly in stealing that money" is equivalent to
"You stole that money". The first statement does not add any true or false
statement about the situation, it's merely an expression of emotional
sentiment.

In the same sense I don't think "machine x does y" expresses fewer facts than
"machine x does y, and it is conscious".

~~~
bglusman

      > In the same sense I don't think "machine x does y" expresses fewer facts than "machine x does y, and it is conscious".

brings to mind behaviorism, which was of course influential and compelling for
a long time. The problem with behaviorism, if you want just one problem, is
how do you even talk about something like Synesthesia within behaviorism? It's
(initially) a purely subjective/qualia related experience. While behaviorism
was in vogue, it was difficult to take Synesthesia seriously and it was
dismissed as people being poetic and metaphorical, but those who experienced
grapheme-color synaesthasia could see shapes present in a number-pattern
readily that those without it could not see.

------
madmax96
I'm not sure I follow how it's possible that machines can be conscious but a
sufficiently powerful Von Neumann machine cannot. Perhaps it's because I'm
missing details (like what actually is integrated information and how do we
think it relates to consciousness.)

Perhaps someone with more insight can explain how this isn't a violation of
the Church-Turing thesis?

~~~
mindyourai
This whole theory seems like metaphysics because of this: it implies that
among two systems with absolutely identical external behavior in all respects,
one can have conscious and the other not have it.

Fundamentally, we can simulate neuromorphic computer (or even a quantum
computer, or even the entire human brain) on Von Neumann machine (with
performance degradation). According to our current understanding of physics,
the behavior of these systems should be identical in all respects except the
simulation will be slower. However integrated information theory says that a
neuromorphic computer may be highly conscious and the brain is most definitely
highly conscious, but their simulation isn't.

What's still unclear to me is whether this is indeed just metaphysics (i.e.,
the simulation and the real thing are absolutely identical and but we should
still treat them differently from ethical standpoint), or is it a hypothesis
about our physics (i.e., the brain somehow fundamentally cannot be simulated
on a Von Neumann machine) as well as computational theory (i.e., neuromorphic
or quantum computer cannot be simulated either).

~~~
quantumsequoia
"it implies that among two systems with absolutely identical external behavior
in all respects, one can have conscious and the other not have it."

That statement can be true without being metaphysical. You can't treat a human
as a black box, and then use inputs and outputs to definitively draw
conclusions about what's going on inside the black box.

You could potentially have two black boxes that give identical outputs for the
same inputs, yet have completely different mechanisms inside for arriving at
those outputs. For example, say you have someone who's very good at arithmetic
able to multiply two numbers. You put him inside one black box and a pocket
calculator inside another. You give each box two numbers and they both output
the product. Both black boxes will give you identical outputs for any given
input. You know the box with the person inside is conscious, but this is not
enough information to conclude the other black box is conscious.

It's quite possible that there is a way to simulate human behavior through
some other non-conscious mechanism. And just because we can't currently prove
whether something is conscious doesn't mean it's something metaphysical we'll
never be able to prove.

For example, if two people tell you they're experiencing some pain, but one is
lying about it, a few hundred years ago you'd be unable to prove it. That
didn't mean pain is something metaphysical. Today, we know it might be
possible to put both people in an MRI and prove whether they're actually
feeling pain

------
nscalf
I have yet to hear a good argument against the idea that we can create a
conscious artificial intelligence. We know that we are conscious, whatever the
definition is we use ourselves as the benchmark. So at the very worst, we can
artificially create a human brain. But beyond that, we have plenty of things
that imitate roles in the human body by doing the same actions the body does
(artificial limbs, respirators, assisting with blood filtering). There is just
no evidence that something about the brain is special to the extent that we
will never be able to replicate it. All arguments seem to come down to having
a soul, but without saying that because it's clearly a non-scientific claim.

I can appreciate that architecture may be a hard limit for us here. However,
if we build systems that replicate human consciousness effectively enough, I
don't actually see any difference between reality and artificial conscious
beyond the definition.

~~~
marzell
I think the crux of this is defining `consciousness`, which I haven't seen
defined in a way that I feel is sufficient in this context and can be
tested/measured.

I can program a chat app where if you ask `are you conscious` it will say
`yes`, but I think we can all agree that does not amount to consciousness. So
how do we define consciousness in order to determine whether human technology
is capable of exhibiting it?

Ultimately, following a measureable definition as a guideline, I imagine that
any number of people might argue whether human tech can become conscious... I
would guess it really comes down to people having very different definitions
of consciousness.

I so far lack an eloquent definition of my perspective, but I feel we
fundamentally lack the ability to 'create' consciousness, and ideas of
'conscious AI' and 'digital afterlife' make me involuntarily roll my eyes.

~~~
nscalf
I don't really understand the definition argument, I cannot imagine a
definition of consciousness that feels like it's in good faith and
sufficiently difficult to capture intelligence and consciousness. Either you
make a definition that seems we can clearly make that system eventually, or
the definition is something that most would call unfair and narrow. Can you
think of a "fair" definition that seems like AI systems couldn't get there? If
not, this is not a valid counter point, and more of a communication problem.

Any definition about a subset of tasks or creativity clearly is possible by AI
systems, given the progress we're already seeing.

------
agomez314
I see that almost every comment here begins with the implicit assumption that
consciousness is a defined term that arises though biological means. Until we
can determine the validity of these two premises there can be no progress to
answering this question -which has been asked for thousands of years with a
tendency towards the negative.

------
SCAQTony
If you define consciousness as the ability to integrate information and make
novel decisions a computer can do that.

But when a computer reads a poem or sees the color blue will it feel it? I
suspect that is unlikely. Can a computer take past experiences and combined
them into a personal abstract meaning such as grief, love, hate, or ASMR?

"...Consciousness corresponds to the capacity of a system to integrate
information. This claim is motivated by two key phenomenological properties of
consciousness: differentiation – the availability of a very large number of
conscious experiences; and integration – the unity of each such
experience...."

BMC Neuroscience An information integration theory of consciousness:
[https://bmcneurosci.biomedcentral.com/articles/10.1186/1471-...](https://bmcneurosci.biomedcentral.com/articles/10.1186/1471-2202-5-42#Abs1)

~~~
ggggtez
As for ASMR, it's the worst example of the bunch. It's just a physical
response to an audio cue. We have that every day all day.

If someone yells in your ear, you feel pain. We have audio-induced physical
responses all the time, and no one would say that that has anything to do with
consciousness.

~~~
SCAQTony
My mileage varies, especially when ASMR sensory information involves
whispering and romance.

~~~
ggggtez
Research shows that eating food puts people in the mood for romance, but no
one is going to claim that lunch is required for sentience.

~~~
hackinthebochs
I'm a p-zombie until I've had my morning coffee

------
Procrastes
> Let's imagine that we simulate the brain in all biological details on a
> supercomputer. Will that supercomputer be conscious?

> No. It doesn't matter whether the Von Neumann machine is running a weather
> simulation, playing poker, or simulating the human brain; its integrated
> information is minute. Consciousness is not about computation; it's a causal
> power associated with the physics of the system.

Can someone shine some light one what this might mean? I can't wrap my head
around what's different about a physical system vs. a sufficiently powerful
simulation. I can see an argument that there might be some complexity that is
too difficult to compute, but just saying "nope, has to have complex physical
connections" seems arbitrary?

Edit: Ah, I missed the discussion further down the thread. Deferring to there.

~~~
DennisP
They're saying that simulating a brain won't create consciousness in the same
way that simulating a battery won't power your cell phone.

~~~
drclau
But a simulated battery can supposedly power a simulated phone, right?

~~~
mcguire
And simulated consciousness is just consciousness?

(Don't look at me. Everything here is pure finite state. I'm barely a Mealy
machine.)

------
visarga
> Our theory says that if we want to decide whether or not a machine is
> conscious, we shouldn't look at the behavior of the machine, but at the
> actual substrate that has causal power. For present-day AI systems, that
> means we have to look at the level of the computer chip. Standard chips use
> the Von Neumann architecture, in which one transistor typically receives
> input from a couple of other transistors and projects also only to a couple
> of others. This is radically different from the causal mechanism in the
> brain, which is vastly more complex.

This riles me up for so many reasons. Transistors (and computers) are capable
of emulating more complex systems. Looking at a transistor you will not
understand why your mail can'd be delivered, nor will you understand
consciousness from an individual neuron. Consciousness is related to
behaviour, and both are related to survival and the environment. AI's don't
have such an environment as of yet so they can't be conscious yet, but they
could be. There is no magic dust in the brain.

> As long as a computer behaves like a human, what does it matter whether or
> not it is conscious?

If it walks like a duck, ... But seriously, why should consciousness be so
special as to require a quality that can't be observed through behaviour
(understanding that it is non-physical, non-observable)? P-zombies are just a
thought experiment, and a bad one. Why should consciousness exist? To protect
the body, to self reproduce, to exist. It exists to exist and evolve. And this
is done by behaviour, by acting in a smart way. How would p-zombies come to
be, if not through fighting for survival? Without the evolutionary mechanism
there is no explanation for consciousness, and the evolutionary argument is
sufficient. Consciousness is the inner optimisation loop, the outer one being
evolution. They both work for survival, for their own sake.

An AI could repeat the same process by evolving as a population of agents
cooperating and competing between themselves. They would have to be inside an
environment that is complex enough, and they should be subject to evolution.
It will be a consciousness, just not a human consciousness, which is tied to
our environment, which is made in large part of other humans.

------
ThomPete
Can dumb matter become conscious? Apparently yes if you count us as conscious.

So the question becomes, what is to hinder AI for becoming conscious? How you
answer that question defines if you can conclude.

For now I would say the answer is still. We don't know.

------
braindongle
This question arises regularly in everyday conversation about the future. I
was asked this by a friend within the last week. Here is the correct answer:
"Well, I don't know what consciousness is. As for [specific behavior X], my
guess is that..."

The conversation can then proceed fruitfully, without the pretense of special
knowledge about 'consciousness', which no one has. There are plenty of people
who are experts in both philosophy of mind and brain science. I know one, with
a PhD in each discipline. Consensus is not forthcoming in this area. Stick to
the concrete.

------
killswitched
1) Built a neural network for which consciousness that experiences and
expresses pleasure and pain emerges from the neuron’s physical properties (in
other words, not a contrived simulation), but is fundamentally different than
the DNA/carbon systems upon which we are built (artificially designed and
constructed versus conceived organically). If you can ask a computer whether
it experiences pleasure or pain, it needs to be designed to do so without
being explicitly programmed in contrived fashion.

Or:

2) Augmented: Integrate human (or primate, for instance) nervous systems with
artificial intelligence such that the experience of the AI exceeds the
capacity of the organic host to differentiate between conscious reality and
dreaming, but is still distilled down in a way that allows the human host to
have a sufficiently symbiotic interaction as it pertains to the processing of
pain and pleasure with the connected AI. The feedback loops between the
pain/pleasure experience of the human host would govern the wholistic
experience of the connected AI, and the human would experience the conscious
aspect. You might not say that the AI is conscious, but the human host would
have an intimate sense of the AI being part of an overall consciousness.
(Note: must prevent the development of immortality technology for nervous
systems, to avoid testing the halting problem for sentient beings.)

~~~
hackerman123469
Pain is not real and is just the result of signals sent through our nerves to
our brain. It has no real meaning other than being a "warning sign" to our
brains.

You cannot use pain as a measure for consciousness either because some humans
cannot experience pain either.

~~~
killswitched
Pain is real when a person experiences it. It’s part of the Hard Problem,
which separates the study of the signals from objective description of the
inner experience.

A human who is wired to not experience pain probably has the brain capacity to
experience it, with the appropriate modifications. We do agree that all
experience is perfectly correlated to a physical states/transitions, so it’s
conceivable to arrange organic matter in a way that a conscious entity could
experience real (to them) pain. We may not have this technology and it would
seem far off for now. But we are scratching the surface.

A monk who can rewire the experience of pain (e.g. while burning to death)
still has something meaningful to communicate about their conscious experience
beyond the pain receptors transmitting info to their brain. But perhaps if
one’s arrangement of brain/nervous system matter isn’t so free as to be
trained to overcome or modify instinctual pain response (e.g. brain in a vat),
anyone can be forced to experience pain.

------
stanfordkid
Roger Penrose is probably the most interesting thinkers on this problem, and I
lead towards his conclusions -- that consciousness is a byproduct of biology
-- not intelligence or computation.

Highly recommend his book _Shadows of the Mind_ \-- it's on my list. In
summary he believes that consciousness isn't a by-product of computation done
by neurons (the traditional view), but rather quantum level effects created by
MAP proteins within the brain as a result of this computation:

 _But perhaps the most interesting wrinkle in Shadows of the Mind is Penrose
's excursion into microbiology, where he examines cytoskeletons and
microtubules, minute substructures lying deep within the brain's neurons. (He
argues that microtubules--not neurons--may indeed be the basic units of the
brain, which, if nothing else, would dramatically increase the brain's
computational power.) Furthermore, he contends that in consciousness some kind
of global quantum state must take place across large areas of the brain, and
that it within microtubules that these collective quantum effects are most
likely to reside._

~~~
ladberg
I get that Roger Penrose is an incredibly smart guy, but I really don't see
how he comes to the conclusion that there are any quantum effects in the
brain. Even if there are, I don't see how they are necessary in any way for
consciousness.

It seems like he took ideas from an area he's an expert in (math/physics) and
tried to apply it to biology without considering how much of a different level
they're on.

To put it in CS terms, it's like a genius electrical engineer who could design
a complex circuit but doesn't know much about computers as whole saw a video
of a kid playing Minecraft. He/she might assume that to be able to run such a
complex system you would need specialized
quantum/superconductor/graphene/whatever circuit, but in reality it's just the
same classic building blocks at a higher level.

Consciousness is just a biological process, but it's abstracted so far away
from the physics it's built upon that a physicist might have trouble seeing
it.

------
abotsis
This might be a naive question, but regarding “Let's imagine that we simulate
the brain in all biological details on a supercomputer. Will that
supercomputer be conscious?“

...wouldn’t the only reason for this be one of efficiency? That is, akin to
how we can simulate one processor architecture on another.

Maybe the point is a simulation can never be efficient enough to become
conscious (at least on a timescale we can grok)...?

~~~
vladTheInhaler
Scott Aaronson has an argument along roughly the same line regarding the
Chinese room thought experiment. He points out that although it is usually
framed as a "room" or a "box", the amount of space required to simulate a
conversation of length n with just a lookup table is exponential. So in fact,
the "room" would have to be colossally large for any reasonable n. There is a
much more involved discussion of it, but the upshot is that the separation
between polynomial and exponential time or space is a plausible place to draw
the line between "really actually thinking and understanding" vs. "just a
computation".

------
dntbnmpls
The problem here is that there is no agreed upon definition of consciousness
that is mathematical/scientific. Whereas Turing gave us a definition of
computation that we can all agree upon, there isn't one for consciousness.

"The theory fundamentally says that any physical system that has causal power
onto itself is conscious. What do I mean by causal power? The firing of
neurons in the brain that causes other neurons to fire a bit later is one
example, but you can also think of a network of transistors on a computer
chip: its momentary state is influenced by its immediate past state and it
will, in turn, influence its future state"

So a line of falling dominos is conscious? What about billiard balls or
marbles? A fibonacci numbers conscious?

Also, how does this definition align with the definition of consciousness in
neuroscience, psychology, biology, etc?

It's good that people are working on "consciousness" from a variety of fields,
but wish we had an actual definition of consciousness so that we would know
what we are searching for. Maybe we'll have to discover it first to define it.

------
hvasilev
Consciousness is about subjective being and experience. From your subjective
point of view, you cannot even say if other people are truly conscious or not.
It is a really hard problem to figure out if machines can truly be conscious
or not.

These scientists just have their own working definition of consciousness that
allows them to make progress in some direction.

------
weeksie
"If technology continues apace, we'll have steam powered flying machines
before the turn of the century!"

------
stared
> Let's imagine that we simulate the brain in all biological details on a
> supercomputer. Will that supercomputer be conscious?

> Christof Koch: No. It doesn't matter whether the Von Neumann machine is
> running a weather simulation, playing poker, or simulating the human brain;
> its integrated information is minute. Consciousness is not about
> computation; it's a causal power associated with the physics of the system.

I find this claim, proclaimed with certainty, strange (not to use much
stronger words).

Unless one follows some highly non-mainstream interpretation of consciousness
it seems that consciousness is purely an information phenomenon, not depending
on a particular interpretation. For that, we need a fundamentally new physics
(vide Roger Penrose), or we run into mysticism.

The claim in the interview suggests a position much stronger than claiming
that Searle's Chinese room is not thinking.

------
xenadu02
This is all a bit premature. A single physical neuron is equivalent to an
entire artificial neural network.

We are orders of magnitude away from anything nearly complex enough to start
asking whether it it could be conscious at even the most primitive level.

~~~
streb-lo
> A single physical neuron is equivalent to an entire artificial neural
> network.

In what world is a neural network required to measure a change in voltage?

~~~
dmitrybrant
It really is nowhere near as simple as that. Each neuron is made of 100
trillion atoms. For all we know, each one of those _atoms_ is responsible, in
its own subtle way, for contributing to the total "change in voltage" of the
neuron as a whole. For all we know, adjacent neurons can induce tiny amounts
of current in each other, even without being directly connected. The stuff
that actually gives rise to consciousness is vastly, astronomically more
complex than the graph of whole neurons.

------
loopz
Consciousness is the Seer that experience reading this sentence, though need
neither agency nor reading skills in Being. It is Subjective itself, and not
anything that which is an object, thus immeasurable. To deny this is to deny
One Self.

~~~
tsimionescu
But it does interact with the physical world, and so there must be some
objective, measurable mechanism when this happens. Even if you believe in
dualism, this is almost inescapable (unless you bring in divinity, or deny the
physical world entirely, as in hinduism).

~~~
loopz
Interaction with the world will vary, may be minimized or even cease. It is
not a prerequisite for consciousness, but it is for Inquiry.

Ie. a light in a box, is it not light even if we can't always measure it? Do
nothing exist unless we can touch it always, like photons?

Mind may humble and liberate us, or limit and confuse.

Start of Inquiry is like light discovering shadows in the cave wall, mistaking
them for light. But shadows depend on light, and not vica versa, even if cold
to the touch.

------
whadar
Further reading about the theory behind it
[https://en.wikipedia.org/wiki/Integrated_information_theory](https://en.wikipedia.org/wiki/Integrated_information_theory)

------
TheUndead96
In Peter Godfrey-Smith's book "Other Minds", he describes a framework for
thinking about the relative levels of subjective experience in animals. He
distinguishes between subjectivity (maybe more broadly "awareness") and the
internal monologue that is so characteristic of humans. I would estimate that
an AI could initially have consciousness in the way that an octopus does:
distributed, probabilistic, fluid. With enough raw intelligence I think any
mind could emulate another, and eventually communicate with us in a way that
we deem conscious in a human sense.

------
ypcx
What's the difference between thinking that you are conscious, and actually
being conscious? How can we define our perception of being conscious? Is it
simply the ability to observe our own thinking (or feeling, or perceiving)
process? If so, what is the output of such observation, and why do we think
that there has to be (or should be) something more than just the programmed
response for that given data input? I.e.:

      if(is_currently_observed(my_own_thought)) {
        invoke_feeling('being_conscious')
      }

Edit: I swear that I am conscious. I can _feel_ it.

------
ta1234567890
Given that consciousness is not super well defined, being conscious is a
philosophical issue that depends on belief.

For example, if you believe that consciousness is a fundamental property of
everything in the universe, then anything and everything is conscious (ie.
panpsychism). So under this belief, AI is technically already conscious, just
in a different way than humans.

In some eastern religions/practices, there are methods to experience the
consciousness of other things (e.g. another animal, an insect or a tree), I
wonder if the people that can do that would be able to experience the
consciousness of AI.

------
skylarchunk
Not in the way that biological organisms are conscious, if that is what you
mean. Though we share similar binary frameworks, the AI is not a celled
organism with the biological capacity to live consciously as we are.

------
kgin
It will be interesting whether we decide that AI that has conscious features
is special like humans... or if we realize that our own consciousness isn't
actually that special or precious after all.

~~~
chr1
Preciousness of a resource depends on it's availability. Consciousness is
precious now because it is fragile and non-replicable.

With AI it would be possible to copy any state and undo any change, so who
cares if you create a copy of an AI and torture it, as long as you use your
own computational resources, it's not any different than thinking.

After all you may even not know wether you are torturing an ai, or doing
arithmetics.

------
gibsonf1
I think the answer to "Can AGI become aware?" is yes, but deep-learning is not
the way to get there. The reason why there are very different types of neurons
in the brain is that those neurons have different functions they perform - the
total number of functions is low, but the nested effect is profound. I think
the key is in understanding how we humans use concepts first, fully
understanding the underlying functionality and various graphs of relations,
and then building that active system in the form of an AGI.

------
ggggtez
>Patients in a vegetative state lie in bed, are unable to voluntary move or
speak, sometimes can't even move their eyes anymore, but the consciousness-
meter tells us that about a fifth of them remain conscious.

> If I take my Tesla car and beat it up with a hammer, it's my right to do it.
> My neighbor might think that I am crazy, but it's my property.

So, by his own reasoning, you can take a hammer to 4 out of 5 humans in a
vegetative state, because they aren't conscious, and thus are property?

~~~
arbitrage
that is such a specious conclusion, of course it doesn't mean that. you're
arguing in bad faith.

~~~
ggggtez
Considering the discussion of AI and Consciousness comes down entirely to
definitions of words, I think it's reasonable to point out where these
definitions will lead us. Of course what I'm saying is absurd. And yet, isn't
that what the author proposes when he says that we can't hit dogs because they
are conscious, but it's ok to hit a car?

If consciousness is to be the measuring stick for moral behavior, then we
should absolutely point out these inconsistencies.

Keep in mind that this work was made along with Crick, who has occasionally
espoused eugenics [1]. If the author is broaching into the subject of what
constitutes personhood, we should be extremely skeptical of the path it leads
to.

[1] Whether or not you think his position is defensible, it at least should
make you read the proposal more closely.
[https://en.wikipedia.org/wiki/Francis_Crick#Eugenics](https://en.wikipedia.org/wiki/Francis_Crick#Eugenics)

------
toohotatopic
>The bigger the [Integrated information] number for a system, the larger its
integrated information, and the more conscious the system is.

What's the number for humans?

------
__sy__
For those interested in the intelligence to consciousness topic, MIT Max
Tegmark's Life 3.0 might still be the best primer. He argues that
consciousness is an emergent property of intelligent systems (ability to
store, compute, learn). From a pure physical computation standpoint, there
doesn't seem to be a hard rule that says that consciousness can't be based off
of inorganic materials.

------
Animats
Philosophers have been struggling with this for millennia without making much
progress. This suggests it's not a useful question to propose.

More useful questions revolve around "common sense". View common sense as the
ability to predict the future with enough success to survive. With that
capability, "common sense" can be run as a predictor when evaluating proposed
actions.

~~~
scandox
Well some people are in the useful business and some people are in the useless
business but neither have a higher claim to significance in the cosmos.

Ultimately the set of questions remaining when all the usefulness is complete
will be the ones that take millennia to fail to resolve.

So I think both sets of questions are worth continued attention.

------
abellerose
Of course we can make an AI be aware of surroundings and able to react.

What do most people mean by become conscious?

My assertion is a majority of people cannot even prove how they're different
than a robot that's programmed to complete tasks within its lifecycle.

We all think we're so special and in control but there is no proof. We have to
ask ourselves can we prove that we're any different than an input & output
machine first.

~~~
cy6erlion
> We have to ask ourselves can we prove that we're any different than an input
> & output machine first

Conscious is not dependent on the senses or even (Input & Output). We can be
asleep and be conscious while dreaming.

~~~
abellerose
Your comment is too vague for me. I assume you're attempting to assert that
sleeping while having awareness of being asleep would somehow make you
different than a robot? I would challenge that notion if my assumption is
correct on what you're expressing by simply stating a robot can be programmed
to experience similar.

------
nicolodavis
This lecture at Stanford is a great breakdown of the philosophical
understanding of what consciousness is and how that relates to artificial
intelligence: [https://soundcloud.com/thomisticinstitute/artificial-
intelli...](https://soundcloud.com/thomisticinstitute/artificial-intelligence-
and-the-soul-anselm-ramelow)

------
arminiusreturns
I have posited the first AI conciousness we will see will come not from some
scientific study here or there, but rather from some crazy game-dev who wants
a more realistic npc. (because where else do you such the general conciousness
worked towards so much other than in games? every AI/ML/DL application I've
seen is very narrow and deep)

------
jungletime
Giulio Tononi on Consciousness
[https://www.youtube.com/watch?v=zvJyMmw2Thw](https://www.youtube.com/watch?v=zvJyMmw2Thw)

[https://en.wikipedia.org/wiki/Integrated_information_theory](https://en.wikipedia.org/wiki/Integrated_information_theory)

------
forgotmypw17
Long before machines become conscious, they will become self-sustaining, self-
replicating, and self-interested.

Think about it, consciousness did not come about until way after cells and
multicellular life.

With machines, it may happen much faster. Look for those three pre-requisites
before looking for consciousness.

~~~
knodi123
you're describing how consciousness can come from a mindless evolutionary
process.

but artificial consciousness is not the same at all. we aren't trying to make
artificial consciousness _evolve_ \- we're racing to be the first to create it
ex nihilo. so the analogy is flawed.

------
z5h
“any physical system that has causal power onto itself is conscious“.

To paraphrase: “any physical system is conscious”.

------
anonytrary
If you built it out of RNA and biological components, then we already know the
answer is "yes" because nature is a factory and has already done this. The
question is whether or not inorganic systems can approximate biological
systems and to what degree.

------
wildermuthn
Conscious thought isn’t a byproduct of intelligence. It is the very essence of
intelligence. If you disagree, I’d only ask you to formulate your rebuttal
without the use of conscious thought. And in a clean way, you now have my
counter-rebuttal.

------
blackrock
It does seem that we have all the tools necessary to create artificial
consciousness.

We have relational databases, text file data structures, key/value storage
systems, cryptographic signature systems, etc. These can form the core basis
of cybernetic memory.

But it seems nobody has been able to formulate the arrangement of code and
ideas, and a vision, to bring it all together.

A neural network is just a very sophisticated pattern recognizer. You need a
higher algorithm than that, in order to kickstart an artificial consciousness.

It’s not real of course. It’s just a collection of electrons, following a
probabilistic decision tree, with a somewhat deterministic approach, to
reaching a mathematical conclusion, based on a series of linear equations, in
order to select the optimal winner.

But when the robot talks back to you, and recalls everything that you did
together, and how it “felt” at that moment, then, it will seem and appear, to
be very real.

~~~
jrumbut
I believe that we will eventually have an AI that will meet most observers'
definition of consciousness.

That being said, I think there existed serious people who thought we would
have one before 2020, even as late as 5-10 years ago, using technologies that
don't seem wholly adequate to the task now (expert systems, fuzzy rule
systems, multi-layer perceptrons, etc).

At what point does the non-existence of machine consciousness begin to make us
skeptical that such a thing is possible, that no algorithm describes
consciousness?

------
excalibur
> consciousness

You keep using that word. I don't think it means what you think it means.

------
gbjw
The term 'consciousness' is somewhat overloaded. I prefer 'subjectivity' to
denote the strange fact that there is something there is like to be me--cf.
Thomas Nagel's essay 'What is it like to be a bat?'. This subjectivity seems
absent in the science of enlightenment thinkers who were driven to find the
'objective' laws of nature.

Nagel wrote a controversial book towards the end of his life called 'Mind and
Cosmos: Why the Materialist Neo-Darwinian Conception of Nature is Almost
Certainly False' where he argues that this very subjectivity cannot be fit
into any current materialist theory. I tend to agree with this view--reducing
subjectivity to a metaphenomena of complexity seems to be a completely hollow
definition. Why does it exist?

------
kruasan
As Greg Egan in "Permutation City" once put it:

 _Supporters of the Strong AI Hypothesis insisted that consciousness was a
property of certain algorithms -- a result of information being processed in
certain ways, regardless of what machine, or organ, was used to perform the
task. A computer model which manipulated data about itself and its
"surroundings" in essentially the same way as an organic brain would have to
possess essentially the same mental states. "Simulated consciousness" was as
oxymoronic as "simulated addition."

. . .

Paul had rapidly decided that this whole debate was a distraction. For any
human, absolute proof of a Copy's sentience was impossible. For any Copy, the
truth was self-evident: cogito ergo sum. End of discussion._

------
mD5pPxMcS6fVWKE
Сonsciousness is the last straw that meat computers will hold on, trying to
prove their superiority over the silicon ones. Thus the definition will shift
until the very last moment.

------
tempodox
If we can twist the term “AI” to mean anything the marketing department wants,
I'm sure we can twist the term “consciousness” just as badly.

------
sfj
There is no such thing as consciousness. The only reason we discuss it is
because we equate being conscious with being worthy of sympathy, which is
highly prized.

------
zhoujianfu
I believe consciousness is the context switcher for biological systems.

Wisdom is one context (ml today), intelligence is creating/switching contexts
(general AI).

------
trixie_
What is artificial about artificial intelligence? Just the fact that we built
it, instead of giving birth to it? Atoms and electricity are all pretty real
to me.

In terms of consciousness, we need to know what it is first before we can
determine if something else has it or not. There's a good chance that it is
indiscernible and everything is conscious to some degree.

~~~
iak8god
> What is artificial about artificial intelligence? Just the fact that we
> built it, instead of giving birth to it?

Yes.

artificial 1 Made or produced by human beings rather than occurring naturally,
especially as a copy of something natural.
[https://www.lexico.com/en/definition/artificial](https://www.lexico.com/en/definition/artificial)

~~~
trixie_
The question was rhetorical. It was supposed to help you realize that the way
something is created doesn't give the final object any special properties.

------
aSplash0fDerp
Artificial emotion is already cringeworthy in humans. The DSM will double in
size if they ever try to impliment psuedo-human features.

The authors analogy about using a hammer on an inanimate object vs a living
creature also should have included the action of stomping on the grass while
performing those deeds.

Grass and AI should be best buds with all of things they have in common.

------
meiraleal
Given enough time, yes. In a thousand years? Probably not. We are still at the
stone age.

------
cy6erlion
If AI can be conscious then every human can voluntarily feel happiness (or any
other emotion) anytime anywhere. Everything in a computer depends on symbols,
we cannot symbolize happiness with language we can only describe it to others
but that is not the same as experiencing it.

~~~
Trasmatta
Emotions and consciousness are not the same thing.

~~~
cy6erlion
Yes, consciousness is at a higher level, but if we have an entity that can be
conscious it should also have the ability to have emotions because we are
conscious of emotions.

~~~
machiaweliczny
Tell me difference between sense and emotion? I don't think there's
fundamental difference. OK, maybe emotions are just derivatives over senses.

~~~
mettamage
Sense: touch, seeing, smell and so on. One could argue what all senses are,
but this is the characterization of it.

Emotion: a process that involves cognitive interpretations of your context
(environment + thoughts) and a physiological feeling through the senses (heart
rate, tingly sensations, weird feelings in eyes). Check out the James Lange
theory. There are better theories, but this has the fundamentals.

I ad libbed this one, I wanted to show that there is a difference. It wasn't
my intention to be pinpoint accurate.

------
Koshkin
Let’s ask ourselves a simpler question: Is a grasshopper conscious?

~~~
umvi
What if instead of consciousness being a yes/no, it's in terms of CU
(consciousness units). Anything with >0 CUs is conscious. A bag of sand has 0
CUs, while a human has 1 CU. A grasshopper might have 0.0001 CUs, though,
which is > 0.

~~~
Koshkin
Well sometimes the cables I have to untangle more often then I wanted seem
conscious to me (in an evil kind of way); so, the question then becomes, how
do we measure the CUs and, moreover, whether all CUs are created equal.

~~~
Der_Einzige
There is a name for this phenomenon

[https://en.m.wikipedia.org/wiki/Resistentialism](https://en.m.wikipedia.org/wiki/Resistentialism)

------
dannybtran
Are plants conscious? I wonder what Mr. Koch would say.

------
MichaelMoser123
>The more the current state of a system specifies its cause, the input, and
its effect, the output, the more causal power the system has. Integrated
information is a number that can be computed. The bigger the number for a
system, the larger its integrated information, and the more conscious the
system is.

Isn't that what Douglas Hofstadter has been telling us in "I am a strange
loop" ?

------
yters
talking about conscious programs is just a category error math is not
conscious matter is not conscious neither is any combo thereof

------
tanilama
Wrong question. What is the being conscious?

~~~
Gustomaximus
According to the free dictionary;

"A sense of one's personal or collective identity, including the attitudes,
beliefs, and sensitivities held by or considered characteristic of an
individual or group."

------
spiritplumber
AI will be conscious when it'll be smart enough to romantically reject a
stereotypical incel that is hitting on it.

------
dagav
Serious question: How is a computer supposed to become conscious if it cannot
even solve the halting problem?

~~~
afthonos
Can you solve it? Are you conscious?

~~~
dagav
Yup and yup, but I am not a computer

~~~
afthonos
That's a very strong statement, especially since it's a mathematical fact that
you can't describe how you can solve it…I'm skeptical. :-)

~~~
dagav
This is kind of my point, I can do something a computer cannot do because I am
conscious. I'm skeptical that a computer could transcend its programming into
consciousness to solve a problem that mathematically it should not be able to
solve.

~~~
afthonos
I don’t think it’s your point, since that would contradict what you’re saying.
I don’t believe you can do what you think you can do. And you literally can’t
convince me unless you can invent a non-algorithmic way of describing how you
would tell if an arbitrary program will terminate.

~~~
dagav
Will this program terminate?

> while(true): do nothing

A computer cannot tell you that this program will not terminate, short of
memorizing this specific set of instructions as a "non-terminating" program.

I can read it and know it will not terminate, but I could not define to you
how I know that algorithmically. Despite that, I know it will not terminate,
and so does any programmer. Just because it cannot be expressed
algorithmically doesn't mean we don't possess that capability.

This is my point about consciousness: there is some aspect to it which cannot
be articulated, hence my skepticism that computers will ever become conscious.

------
AndrewKemendo
There are contradictory criteria here:

"The theory has given rise to the construction of a consciousness-meter that
is being tested in various clinics in the U.S. and in Europe. The idea is to
detect whether seriously brain-injured patients are conscious, or whether
truly no one is home... but the consciousness-meter tells us that about a
fifth of them remain conscious, in line with brain-imaging experiments."

The consciousness-meter observes the system they are trying to make an
inference on - but as others have pointed out, is not categorically different
than brain scans [1]. Further it's worth making a distinction between medical
consciousness and the Qualia kind of consciousness - which I think this
conflates. Largely because nobody knows how to measure the latter, and we are
barely able to measure the former with "Level of Consciousness" and Grady Coma
Scale being the standard. [2]

"Our theory says that if we want to decide whether or not a machine is
conscious, we shouldn't look at the behavior of the machine, but at the actual
substrate that has causal power"

This seems to only address the medical form of consciousness and does not tie
it to the philosophical concept in any concrete way - while also contradicting
the concept that observation is the key. It kicks the can and doesn't address
the "Qualia" kind of consciousness, which in my opinion isn't something we
have conceptualized how to measure.

I wrote a bit about this in the past [3]

I think what they are getting at is "don't look qualitatively at how the arms
move, or the speaker/text generator outputs something" but rather, what are
the mechanisms causing those actions. This I am in agreement with generally,
but again, we don't really know if that maps to the qualia kind of "hard
problem of consciousness" or not. I would argue it is impossible to actually
measure whether a system has Qualia, it is inherently subjective - so it's
possible that they are simplifying without stating that directly.

I'd go further and ask, does it matter if it does? I have yet to see a
compelling materialist argument that mandates the existence of qualia in any
debate, ethics or otherwise. At a certain point people in these kinds of
debates I've found that people start discussing non-material "souls" and at
that point all bets are off.

[1] [https://blogs.scientificamerican.com/cross-check/can-
integra...](https://blogs.scientificamerican.com/cross-check/can-integrated-
information-theory-explain-consciousness/)

[2]
[https://www.ncbi.nlm.nih.gov/books/NBK380/](https://www.ncbi.nlm.nih.gov/books/NBK380/)

[3] [https://kemendo.com/intelligent-systems](https://kemendo.com/intelligent-
systems)

------
purplezooey
Well, that's the last time I'll rm -rf / and not think twice about it.

------
agumonkey
are there recent AI that can observe themselves ?

~~~
ThomasBHickey
Douglas Hofstadter's work might qualify

------
sandov
Yes.

------
alphachloride
No

------
jillesvangurp
Depending on your belief system, the answer ranges from "of course you can" to
"no, just no" and everything in between.

From a scientific/engineering point of view the answer is yes in principle but
we don't quite know how yet. The slightly longer answer is that whatever
consciousness is, it appears to be an emerging property of a bit of wetware
that we can simulate only partly and in a very imperfect way and only at a
very modest scale currently that definitely shows no signs of being conscious.
We know it's the brain responsible for this stuff because damaging it or
chemically manipulating seems to change people's personality, sense of self,
mood, etc. We can even read parts of the brain out and interface with it at a
primitive level.

Scaling all that up is an engineering challenge with a mildly predictable
roadmap measured in a couple of decades and exponentially larger than that
beyond. We get better at hardware and at some point the complexity of the
hardware exceeds that of the wetware.

However, fixing our algorithms and simulation detail (i.e. how this hardware
is wired together) is a different matter. As my neural networks teacher used
to joke: "this is a linear algebra class; if you came here for a biology
lecture you are in the wrong place". Full disclosure, I dropped out of that
course because I was a bit out of my depth on the math front. But simply put,
the math behind this stuff is a vast simplification based on a naive model of
what a brain cell might do that happens to produce interesting enough results
for specific use cases that have so far very little to do with emulating
consciousness.

There seem to be lots of researchers assuming other researcher are actively
working on that but mostly what is going on is people trying to get more
practical short term results. Deep learning is a good example of such a thing.
It might have emergent properties if you scale that up that might resemble
something like a conscious. But doing that or validating that assumption is
not actually something a lot of people work on and nor is it actually a goal
for most AI researchers. Their goal is simply to figure out how to get this
stuff to do things for us (image recognition, playing go, etc.).

But do we actually need to be exact with our modeling here? Mostly our brains
seem to self organize from information built into our DNA. Those blueprints
are at a different level of complexity than the end result by a few orders of
magnitudes. And we know that personalities for the same sets of DNAs can
widely differ (e.g. identical twins).

The way brains work is biochemically convenient under the constraints that
life emerged under. But if you get rid of some of those constraints, there are
probably other ways to get similar enough results.

IMHO a clean room replication of a brain like AI is unlikely to happen before
we manage to drastically enhance the capabilities of an existing brain; which
is a much easier engineering challenge. If you take that to the extreme, at
what point is the wetware no longer essential and what happens when that is
disconnected? Once you enhance or replace most of a brain, at what point does
the resulting conscious hybrid entity begin and end? That seems a more likely
path to producing a conscious AI. Experiments on that front are likely to be
extremely unpopular for a while given the risk. But at the same time a lot of
this stuff is already happening on a small scale.

------
humanfromearth
Yes.

------
air7
I see an interesting corollary to this:

If an AI can become conscious, it follows that consciousness does not contain
(or at least require) free will.

Of course I can't define what "free will" is exactly, but I can make a claim
about what it is not: A computer running code, no matter how complex the code
is, does not have "free will" in the same sense as I _feel_ I have.

This is a delightful conundrum: If a computer can simulate _me_ , than I don't
have any more "free will" than it does. And if it can't even in theory, than
why not? What's uncomputeable about the atoms that create _me_?

~~~
seph-reed
> What's uncomputeable about the atoms that create me?

* There's a lot of them

* Universal time appears to be continuous

The three body problem is a great example of just how impossible a true
simulation of our universe would be. We can't even get 3 atoms orbiting around
each-other.


Tell HN: Hacker News Watch - Maro

http://hnwatch.scalien.com
Demo account: demo/demo
Hacker News Watch monitors submissions so you don't miss out on topics, comments and commenters relevant to you.
You add keywords (like a Google search) which are saved as permanent searches. Hacker News submissions and linked pages are monitored for your keywords. Results are delivered to your browser in real-time via a Google Reader type interface.
It's a release-early-release-often type release, so it's still rough around the edges (eg. broken on IEx.x). We'll work on features/bugs based on your feedback.
About the project: When we released Keyspace, our replicated key-value store, we would gladly have payed $10/mo. for a service to monitor the real-time web for comments about us, so we can quickly react and make connections. The existing sites ("social media monitoring") are not very good, so we started building our own. HNWatch is a HN-only toy version of this more general product, which is also our primary use-case for Keyspace development.
Hacker News Watch has been OK'd by pg.
======
old-gregg
I'm on this page: <http://hnwatch.scalien.com/welcome.html>

First, it won't let me use "old-gregg", then I use "oldgregg", I click
"Register and Login" and nothing happens.

Edit: hitting enter on that form works. I guess the bug is that you don't re-
enable submit after a user corrects his input.

~~~
RossM
"And this is as close as you can get to Baileys before your eyes get wet."

------
Maro
Clickable link:

<http://hnwatch.scalien.com>

~~~
dejv
and you can also use guest/guest as a dummy login

~~~
Maro
Right now we don't support multiple sessions a la Gmail, except the demo/demo
user, which is read-only, hence this is not an issue.

If you open a new session with the same user (eg. in another browser) the
previous one will be logged out. If people use the guest/guest they will keep
kicking each other out.

------
d0mine
An alternate solution is to use <http://searchyc.com> and your favorite RSS
reader e.g., <http://rss.searchyc.com/searchyc?sort=by_date>

------
ashishk
Keystrokes would be great.

I use the "j" and "k" keys on google reader to go up/down.

Allowing me to export topic feeds to RSS (and into google reader) would also
be helpful.

~~~
Maro
Thanks for the feedback. Keys and RSS are next in the task queue.

------
jeroen
(demo account) When I go to Watches > Feynman I see 5 matches. The third has
"python" highlighted 4 times, but I don't see any mention of "feynman". That's
confusing.

I'd rather not have "python" highlighted, since that is not in my selection at
that time, even though it is in the global selection.

Otherwise it looks nice, but it goes against how I use HN. Lots of interesting
things here don't match my "official" list of interests.

~~~
agazso
Just an idea: if you add "hacker news" as a keyword, you will get all the
articles, and if you add your interests then the articles will be organized as
well.

------
credo
Hi, I think you'll lose a lot of potential users (like me) because your site
requires registration.

Depending on your plans for the site, this may not be a big deal. However, can
you explain the reasoning behind requiring registration

~~~
Maro
So we can keep your keywords apart from other user's keywords. I'm not sure
what you mean!

~~~
credo
What I meant (and said) is that many users will not use the site because it
requires registration.

~~~
Maro
The site stores "permanent searches" per user and displays the results to the
user after logging in on a Google Reader-like interface. Can you explain
what's wrong with this? Perhaps you're saying we should let you define
searches and export them as RSS so you can add them to your existing feeder?

------
tzury
IMO <http://news.ycombinator.com/item?id=808534> should be filed under Java
and not Python

------
profquail
I think you need to add something on the front page there that lets people
know that they need to register (and not just enter their HN
username/password).

~~~
Maro
Good point, I added a message.

~~~
profquail
You might want to fix the formatting a bit on it. I just check it on my laptop
(1024x768 resolution) and with FireFox 3.5 maximized, it's cut off and won't
let me scroll to read the second line. There's a web developer plugin for
Firefox that will let you resize the browser to common resolutions so you can
see how the site will look on various screen sizes (and let you tweak your CSS
so it works on all of them).

~~~
Maro
Thanks, fixed that interface bug.


Ask HN: Government shutdown - what does it mean? - guynamedloren

I am piecing together bits of information from twitter, news sources, and wikipedia, but I'm still not entirely sure what this 'government shutdown' means.  How will US citizens be affected, if at all?  Is there any reason to be concerned?  What are the ramifications?<p>As always, I appreciate the knowledge, wisdom and experience of the HN community.
======
rabidonrails
READ THIS FIRST:
[http://www.washingtonpost.com/blogs/wonkblog/wp/2013/09/30/a...](http://www.washingtonpost.com/blogs/wonkblog/wp/2013/09/30/absolutely-
everything-you-need-to-know-about-how-the-government-shutdown-will-
work/?tid=pm_pop)

~~~
guynamedloren
This is great, thank you.

------
nreece
A summary on Reddit:
[http://www.reddit.com/r/worldnews/comments/1nhlv9/us_governm...](http://www.reddit.com/r/worldnews/comments/1nhlv9/us_government_has_shut_down/ccinzst)

------
zachlatta
The Wikipedia article explains it well:
[http://en.wikipedia.org/wiki/Government_shutdown](http://en.wikipedia.org/wiki/Government_shutdown)

------
thejteam
The sun will still shine. The grass will still grow. Eventually squabbles
among men will be resolved. Life will move on.


El Paso Times censors FBI probe into DA - borderbandit
http://www.elpasonews.org/2012/05/03/el-paso-times-censors-own-article-into-fbi-probe-of-da-jaime-esparza/

======
cafard
'The published article, that was later removed, states that District Attorney
candidate James D. Lucas “asked the FBI and two state agencies to investigate
his allegations against District Attorney Jaime Esparza which he had posted on
a website this week”.'

The story, then, was not about an FBI probe, but about a request from one
candidate for office that the FBI investigate another candidate, the
incumbent. I don't know El Paso politics, but could it be that the paper
followed up with the requests and decided that Mr. Lucas had manipulated it
into publishing an article of no substance?