Source: http://web.media.mit.edu/~minsky/E4/eb4.html
Timestamp: 2019-04-23 19:57:26+00:00

Document:
This is a draft (July 28, 2005) of Chapter IV of The Emotion Machine by Marvin Minsky. Please send comments to minsky@media.mit.edu.
§4-1. What is the nature of Consciousness?
“No philosopher and hardly any novelist has ever managed to explain what that weird stuff, human consciousness, is really made of. Body, external objects, darty memories, warm fantasies, other minds, guilt, fear, hesitation, lies, glees, doles, breath-taking pains, a thousand things which words can only fumble at, co-exist, many fused together in a single unit of consciousness”— Iris Murdoch, in The Black Prince. 1973.
What kinds of creatures have consciousness? Does it exist in chimpanzees—or in gorillas, baboons, or orangutans? What about dolphins or elephants? Are frogs, fish, insects, or vegetables aware of themselves to any extent—or is consciousness a singular trait that segregates us from the rest of the beasts?
Although those animals won’t answer questions like, “Are you aware that you exist,” or “ What is your view of what consciousness is,” the answers from people are scarcely more useful. When you ask mystical thinkers how consciousness works, their replies are not highly enlightening.
Some philosophers even insist that there’s no way to look for good answers to this.
Jerry Fodor: "Nobody has the slightest idea how anything material could be conscious. Nobody even knows what it would be like to have the slightest idea about how anything material could be conscious. So much for the philosophy of consciousness."
Is consciousness an ‘all-or-none’ trait that has a clear-cut boundary, or does it have different amounts and qualities—the way that a thing can be cold or hot?
Relativist: Everything has some consciousness. An atom has only a little of it. Bigger things must have it in larger degrees— right up to the stars and the galaxies.
Absolutist: We don’t know where consciousness starts and stops, but clearly each thing must be conscious or not—and, clearly, there is no such thing in a rock.
Computer User: Certain programs seem to me already conscious to some small degree.
Logicist: Before you go on about consciousness, you really ought to define it. Good arguments should start right out by stating precisely what they are about. Otherwise, you'll build on a shaky foundation.
That policy might seem 'logical'—but it’s wrong when it comes to psychology, because it assumes that ‘consciousness’ has a clear and definite meaning. Of course, we don't like to be imprecise—but strict definitions can make things worse, until we’re sure that our ideas are right. For, 'consciousness’ is a word we use for many types of processes, and for different kinds of purposes; we apply it to feelings, emotions, and thoughts—and to how we think and feel about them. It’s the same for most everyday words about minds, such as ‘creativity’ or ‘intelligence’.
So instead of asking what ‘consciousness’ is, or what we mean by ‘being aware,’ we’ll try to examine when and why people use those mysterious words. But why do such questions even arise? What, for that matter, are mysteries?
Indeed, many of those who ‘insist—and hope’ that consciousness cannot be explained still claim that it alone is the source of most of the virtues of human minds.
Thinker 1: Consciousness is what unifies our present, past, and future together, by making sense of all our experience.
Thinker 2: Consciousness makes us 'aware' of ourselves, and gives us our sense of identity; it is what animates our minds and gives us our sense of being alive.
Thinker 3: Consciousness is what gives things meanings to us; without it, we would not even know we had feelings.
Wow! How could one principle, power, or force endow us with so many faculties? It can’t—and this chapter will argue that there is no reason to suppose that all of those different abilities stem from just one common origin. Indeed, from what we know about brains, it is safer to guess that they’re each based on different machinery.
All this shows that "consciousness" does not refer to any single idea or thing, but that we use it as a suitcase-word for a great many different activities.
§4-2. Unpacking the Suitcase of Consciousness.
Aaron Sloman: “It is not worth asking how to define consciousness, how to explain it, how it evolved, what its function is, etc., because there's no one thing for which all the answers would be the same. Instead, we have many sub-capabilities, for which the answers are different: e.g. different kinds of perception, learning, knowledge, attention control, self-monitoring, self-control, etc."
To see the variety of what human minds do, consider this fragment of everyday thinking.
Reaction: Joan reacted quickly to that sound.
Identification: She recognized it as being a sound.
Characterization: She classified it as the sound of a car.
Imagining: She envisioned two or more possible futures.
Indecision: She wondered whether to cross or retreat.
Decision: She chose one of several alternative actions.
Recollection: She retrieved descriptions of prior events.
Reconsideration: Later she reconsidered this choice.
Selection: She selected a way to choose among options.
Apprehension: She was uneasy about arriving late.
Planning: She constructed a multi-step action-plan.
Embodiment: She tried to describe her body's condition.
Emotion: She changed major parts of her mental state.
Representation: She interconnected a set of descriptions.
Language: She constructed several verbal expressions.
Narration: She heard them as dialogs in her mind.
Anticipation: She expected certain future condition.
Intention: She changed some of her goals’ priorities.
Reasoning: She made various kinds of inferences.
Reflection: She thought about what she’s recently done.
Self-Reflection: She reflected on her recent thoughts.
Empathy: She imagined other persons’ thoughts.
Moral Reflection: She evaluated what she has done.
Self-Imaging: She made and used models of herself.
Self-Awareness: She characterized her mental condition.
Sense of Identity: She regarded herself as an entity.
Holist: Yet after you analyze all those parts, you will still be obliged to explain how they all unite to produce the streams of consciousness that emerge from them. So, then you still will need some words to describe that entire phenomenon.
Why did our language come to include such terms as ‘awareness,’ ‘perception,’ ‘consciousness,’ every one of which condenses many different processes?
Psychologist: Such self-words are useful in everyday social life because they help us to communicate—both with our friends and with ourselves. For, because we all share the same kinds of jumbled ideas, we can pack them into vague suitcase-terms that seem easy for us to understand.
Ethicist: We need them also to support our principle of responsibility and discipline. Our legal and ethical principles are largely based on the idea that we ought to punish or reward only actions that are ‘intentional’—that is, are based on having been planned in advance, with predictions about their consequences.
Psychiatrist: Perhaps we use those suitcase terms to keep ourselves from asking too much about how out minds control themselves, and what underlies the decisions we make. Perhaps we use words like “consciousness” to help us suppress all those questions all at once—by suggesting that all of them are just a single big Mystery.
Student: If “ consciousness” is just a suitcase word, what makes it seem so clear to us that we actually possess such a thing? If such terms keep shifting their meanings, why doesn't this become evident whenever we start to think about them?
That could be because no part of a mind can ‘see’ much of what the rest of that mind does. A typical resource inside a brain accomplishes its jobs internally, in ways that other resources cannot perceive. Also, when any resource probes into another, that very act may change the other’s state—and thus scramble the very evidence it would need to recognize what’s happening. These could partly account for Hume’s complaint that our minds lack good ways to inspect themselves.
Hume assumes that we could never develop more powerful ways to inspect ourselves—but today we have new image-machines that show more of what happens inside our brains. For example, now we can detect activities that start before our limbs begin to move.
Dualist philosopher: Still, those instruments will eventually fail, because you can measure a brain but not an idea. Some creatures are conscious, while others are not—and consciousness is a subjective thing that can’t be explained in physical terms.
Functionalist Philosopher: What evidence could support your faith that consciousness could never be explained? We can see it simply as our name for what happens when certain processes run in our brains.
I would agree with that second opinion, except that we also need to say more about what those 'certain processes' do—and why we distinguish them as a group. ((The next section will offer a theory of this.) Still, many thinkers still maintain that brains must be based on something beyond the reach of our present-day machines.
Emergentist: Perhaps consciousness is just one of those ‘wholes’ that emerge when systems get complex enough. Perhaps that’s just what we should expect from the network of billions of cells in a brain.
When we increase a system’s size, then it will usually work less well, unless we also improve its design, and that always involves some compromise; if a system is built with too many connections, this will lead to traffic jams—while if the connections between its parts are too sparse, it is unlikely to anything useful at all.
Besides, if mere complexity were all it needs, then almost everything would have consciousness. We don’t want to conclude that water-waves think—yet the manner in which a wave breaks on a beach is more complex (at least in some ways) than the processes that go on in our brains.
I completely agree with Sloman’s view. To understand how our thinking works, we must study those “very different things” and then ask what kinds of machinery could accomplish some or all of them. In other words, we must try to design—as opposed to define—machines that can do what our minds can do.
§4-3. How do we recognize Consciousness?
Student: You still did not answer my question on why, if “consciousness” is just a suitcase word, what makes it seem like such a definite thing.
(1) They make use of our most recent memories.
(2) They operate more serially, than in parallel.
(4) They use models that we have made of ourselves.
If such a C-detector turned out to be useful enough, this could lead us to imagine that it detects the presence of some sort of ‘Consciousness-Thing!’ Indeed, we might even imagine that entity to be the cause of that set of activities, and our language systems might learn to connect this kind of detector to terms like ‘awareness,’ ‘myself,’ ‘attention,’ or ‘Me’. To see how this might be useful to us, let’s examine its four constituents.
Recent Memories: Why must consciousness involve memory? I’ve always thought of consciousness as about the present, not the past—about what’s happening right now.
For any mind (or any machine) to know what it has done, it needs some records of recent activities. For example, suppose that I asked, “Are you aware that you're touching your ear?” Then you might reply, "Yes, I'm aware that I am doing that." However, for you to make a statement like that, your language resources must be reacting to signals from other parts of your brain, which in turn have reacted to prior events. So, whatever you say (or think) about yourself, it takes time to collect that evidence.
More generally, this means that a brain cannot think about what it is thinking right now; the best it could do is to contemplate some records of some of its recent activities. There is no reason why some part of a brain could not think about what it has seen of the activities of other parts—but even then, there always will be at least so small delay in between.
Serial Processes. Why should our high-level processes tend to be more serial? Would it not be more efficient for us to do more things in parallel?
Most of the time in your everyday life, you do many things simultaneously; you have no trouble, all at once, to walk, talk, see, and scratch your ear. But few can do a passable job at drawing a circle and square at once by using both of their hands.
Citizen: Perhaps each of those two particular tasks demands so much of your attention that you can’t concentrate on the other one.
That would make sense if you assume that attention is some sort of thing that comes in limited quantities—but then we would need a theory about what might impose this kind of limitation, yet still can walk, talk and see all at once. One explanation of this could be that such limits appear when resources conflict. For, suppose that two tasks are so similar that they both need to use the same mental resources. Then if we try to do both jobs at once, one of them will be forced to stop—and the more such conflicts arise in our brains, the fewer such jobs we can do simultaneously.
Then why can we see, walk, and talk all at once? This presumably happens because our brains contain substantially separate systems for these—located in different parts of the brain—so that their resources don’t conflict so often. However, when we have to solve a problem that’s highly complex then we usually have only one recourse: somehow to break it up into several parts—each of which may require some high-level planning and thinking. For example each of those subgoals might require us to develop one or more little ‘theories’ about the situation—and then do some mental experiments to see if these are plausible.
Why can’t we do all this simultaneously? One reason for this could simply be that our resources for making and using plans has only evolved rather recently—that is, in only a few million years—and so, we do not yet have multiple copies of them. In other words, we don’t yet much capacity at our highest levels of ‘management’—for example, resources for keeping track of what’s left to be done and for finding ways to achieve those goals without causing too many internal conflicts. Also, our processes for doing such things are likely to use the kinds of symbolic descriptions discussed below—and those resources are limited too. If so, then our only option will be to focus on each of those goals sequentially.
Symbolic Descriptions: Why would we need to use symbols or words rather than, say, direct connections between cells in the brain?
Many researchers have developed schemes for learning from experience, by making and changing connections between various parts of systems called ‘neural networks’ or ‘connectionist learning machines.’ Such systems have proved to be able for learning to recognize various kinds of patterns—and it seems quite likely that such low-level processes could underlie most of the functions inside our brains.  However, although such systems are very useful at doing many useful kinds of jobs, they cannot fulfill the needs of more reflective tasks, because they store information in the form numerical values that are hard for other resources to use. One can try to interpret these numbers as correlations or likelihoods, but they carry no other clues about what those links might otherwise signify. In other words, such represesntations don’t have much expressiveness. For example, a small such neural network might look like this.
In contrast, the diagram below shows what we call a “Semantic Network”that represents some of the relationships between the parts of a three-block Arch. For example, each link that points to the concept supports could be used to predict that the top block would fall if we removed a block that supports it.
Thus, whereas a ‘connectionist network’ shows only the ‘strength’ of each of those relations, and says nothing about those relations themselves, the three-way links of Semantic Networks can be used for many kinds of reasoning.
Self-Models: Why did you include ‘Self-Models’ among the processes in your first diagram?
When Joan was thinking about what she had done, she asked herself, “What would my friends have thought of me.” But the only way she could answer such questions would be to use some descriptions or models that represent her friends and herself. Some of Joan's models of herself will be descriptions of her physical body, others will represent some of her goals, and yet others depict her dispositions in various social and physical contexts. Eventually we build additional structures include collections of stories about our own pasts, ways to describe our mental states, bodies of knowledge about our capacities, and depiction of our acquaintances. Chapter §9 will further discuss how we make and use ‘models’ of ourselves.
I don’t mean to insist that ‘detectors’ like these must be involved in all of the processes that we call consciousness. However, without some ways to recognize these particular patterns of mental conditions, we might not be able to talk about them!
This section began with some ideas about what we recognize when we talk about consciousness, and we suggested that this might relate to detecting some set of high-level activities.
This suggests that we sometimes use ‘conscious’ to refer to activities that initiate rather than recognize sets of higher-level processes.
Student: How did you choose those particular features for your scheme to decide when to use words like ‘consciousness?’ Surely, since this is a suitcase-word, each person might make a different such list.
When you enter a room you have the sense that you instantly see all the things in your view. However, this is an illusion because it will take time to recognize the objects that are actually there; then you’ll have to revise many wrong first impressions. Nevertheless, all this proceeds so quickly and smoothly that this requires an explanation—and we’ll propose one later in §8-3 Panalogy.
The same thing happens inside one’s mind. We usually have a constant sense that we’re ‘conscious’ of things that are happening now. But when we examine this critically, we recognize that there must be something wrong with it—because nothing exceeds the speed of light. This means that no internal part of a brain can ever know exactly what is happening “now”—either in the outside world or in any other part of that brain. The most that any resource can know is some of what happened in the recent past.
Citizen: Then why does it seem to me that I am conscious of all sorts of sights and sounds, and of feeling my body moving around—right at this very moment of time? Why do all those perceptions seem to come to me instantaneously?
The Immanence Illusion: For most of the questions you would otherwise ask, some answers will have already arrived before the higher levels of your mind have had enough time to ask for them.
In other words, if some data you need were already retrieved before you recognized that you needed it, you will get the impression of knowing it instantaneously—as though no other processes intervened.
For example, before you enter a familiar room, it is likely that you have already retrieved an old description of it, and it may be quite some time before you notice that some things have been changed; the idea that one exists in the present moment may be indispensable in everyday life—but much what we think that we see are the stereotypes of what we expected.
Some claim that it would be wonderful to be constantly aware of all that is happening. But the more often your high-level mental resources change their views of reality, the harder it will be for them to find significance in what they sense. The power of our high-level descriptions comes not from changing ceaselessly, but from having enough stability.
In other words, for us to sense what persists through time, one must be able to examine and compare descriptions from the recent past. We notice change in spite of change, not because of it. Our sense of constant contact with the world is a form of the Immanence Illusion: it comes when every question asked about something is answered before we know it was asked—as though those answers were already there.
In Chapter §6 we’ll also see how our ways to activate knowledge before we need it could explain why our ‘commonsense knowledge’ seems ‘obvious.
Why has Consciousness' seemed such a mystery? I'll argue that this is largely because we exaggerate our perceptiveness. For example, at any one moment the lens of your eye can clearly focus only on objects in a limited distance range, while everything else will be blurry.
Citizen: That doesn’t seem to apply to me, because all the objects that I can see seem clearly focused all at once.
You can see that this is an illusion, if you focus your eyes on your fingertip while trying to read a distant sign. Then you’ll see a pair of those signs at once, but both will be too blurry to read. Until we do such experiments, we think we see everything clearly at once, because the lens in each eye so quickly adjusts that we have no sense that it’s doing this. Similarly, most people believe they see, at once all the colors of things in a scene—yet a simple experiment will show that we only see colors of things in the field near the object you’re looking at.
Both of these are instances of that Immanence Illusion, because you eyes so quickly turn to see whatever attracts your attention. And I claim that the same applies to consciousness; we make almost the same kinds of mistakes about how much we can ‘see’ inside our own minds.
Patrick Hayes: "Imagine what it would be like to be conscious of the processes by which we generate imagined (or real) speech. … [Then] a simple act like 'thinking of a name', say, would become a complex and skilled deployment of elaborate machinery of lexical access, like playing an internal filing-organ. The words and phrases that just come to us to serve our communicative purposes would be distant goals, requiring knowledge and skill to achieve, like an orchestra playing a symphony or a mechanic attending to an elaborate mechanism.
In this paradoxical view, consciousness still seems marvelous—but not because it tells us so much, but for protecting us from such tedious stuff! Here is another description of this, from section 6.1 of The Society of Mind.
Consider how a driver guides the immense momentum of a car, not knowing how its engine works or how its steering wheel turns it left or right. Yet when one comes to think of it, we drive our bodies, cars, and minds in very similar ways. So far as conscious thought is concerned, you steer yourself in much the same way; you merely choose your new direction, and all the rest takes care of itself. This incredible process involves a huge society of muscles, bones, and joints, all controlled by hundreds of interacting programs that even specialists don't yet understand. Yet all you think is "Turn that way," and your wish is automatically fulfilled.
And when you come to think about it, it scarcely could be otherwise! What would happen if we were forced to perceive the trillions of circuits in our brains? Scientists have peered at these for a hundred years—yet still know little of how they work. Fortunately, in everyday life, we only need to know what they achieve! Consider that you can scarcely see a hammer except as something to hit things with, or see a ball except as a thing to throw and catch. Why do we see things, less as they are, and more with a view of how they are used?
Similarly, whenever you play a computer game, you control what happens inside the computer mainly by using symbols and names. The processes we call "consciousness" do very much the same. It’s as though the higher levels of our minds sit at mental terminals, steering great engines in our brains, not by knowing how that machinery works, but by ‘clicking’ on symbols from menu-lists that appear on our mental screen-displays.
Our minds did not evolve to serve as instruments for observing themselves, but for solving such practical problems as nutrition, defense, and reproduction.
In judging the development of self-consciousness, we must guard against accepting any single symptoms, such as the child's discrimination of the parts of his body from objects of his environment, his use of the word "I,” or even the recognition of his own image in the mirror. … The use of the personal pronoun is due to the child's imitation of the examples of those about him. This imitation comes at very different times in the cases of different children, even when their intellectual development in other respects is the same. — Wilhelm Wundt, 1897.
In §4-2 we suggested that Joan ‘made and used models of herself’—but we did not explain what we meant by a model. We use that word in quite a few ways, as in “Charles is a model administrator,” which means that he is an example worthy to imitate—or in, “I’m building a model airplane,” which means something built on a scale smaller than that of the original. But here we’re using ‘model of X’ to mean a simplified mental representation that can help us to answer some questions about some other, more complex thing X.
Thus, when we say that ‘Joan has a mental model of Charles’, we mean that Joan possesses some mental resource that helps her answer some questions about Charles. I emphasize the word some because each of Joan’s models will only work well on some kinds of questions—and might give wrong answers to most other questions. Clearly the quality of Joan’s thought will depend both on how good her models are, but on how good are her ways to choose which model to use in each situation.
Her various goals and ambitions.
Her professional and political views.
Her ideas about her social roles.
Her various moral and ethical views.
Her beliefs about of what sort of thing she is.
For example, she could try to use some of these to guess whether she can rely on herself to actually carry out a certain plan. Furthermore, this could explain some of her ideas about consciousness. To illustrate this, I’ll use an example proposed by the philosopher Drew McDermott.
Most of those objects will have sub-models themselves, for example to describe their structures and functions. Joan’s model for that object “Joan’ will be a structure that she calls “Myself,” and which includes at least two parts—one called Body and one called Mind.
By using the various parts of this model, Joan could say ‘Yes’ if you asked her, “Do you have a mind?” But if you asked her, “Where is your mind?” this model would not help her to say, as some people would, “My mind is inside my head (or my brain).” However, Joan could offer such a reply if Myself included a part-of link from Mind to Head, or a caused–by link from Mind to another part of the body called Brain.
More generally, our answers to questions about ourselves will depend on what is in our models of ourselves. I used models instead of model because, as we’ll see in §9, one may need different models for different purposes. So there may be many answers to the same questions, depending on what one wants to achieve—and those answers need not always agree.
Drew McDermott: Few of us even believe that we have such models, much less know that we have one. The key idea is not that the system has a model of itself, but that it has a model of itself as conscious.” —comp.ai.philosophy, 7 Feb 1992.
So, when Joan says, "I made a conscious decision", that need not mean that some magical thing has happened; she attributes her thoughts to various parts of her most useful models.
“We can see that the mind is at every stage a theater of simultaneous possibilities. Consciousness consists in the comparison of these with each other, the selection of some, and the suppression of others, of the rest, by the reinforcing and inhibiting agency of attention. The highest and most celebrated mental products are filtered from the data chosen by the faculty below that…in turn sifted from a still larger amount of simpler material, and so on.” —William James .
Now, as soon as Joan starts to think that way—about how she feels, and about what she might do—then Joan herself takes a place on that stage. But in order to hear what she says to herself, she must also remain in the audience. So now we have two copies of Joan—the actor, and her audience!
However, when we look closely at this theatrical view, we see that it provides no answers, but only raises additional questions. When Critic-Joan complains about pain, how does she relate to the Joan-on-the-stage? Does each of those actresses need her own theater, each with its own one-woman show? Of course no such theater really exists, and those Joan-things are not people like us; they are only different models that Joan has constructed as ways to represent herself in various kinds on contexts. In many cases, those models are much like cartoons or caricatures— and in yet other cases, they are downright wrong. Still, Joan’s mind abounds with varied self-models—Joans past, Joans present and future Joans; some represent remnants of previous Joans, while others describe what she hopes to become; there are sexual Joans and social Joans, athletic and mathematical Joans, musical and political Joans, and various kinds of professional Joans—and because of their different interests, we shouldn’t expect them to all ‘get along’. We’ll discuss this more in §9-X.
Why is this image so popular! To begin with, it doesn’t explain very much—but it’s better than the simpler idea that all thinking is done by a Single Self. It recognizes that minds have parts, and that these may need to interact—and that theater serves as a metaphor for a 'place' in which those processes can work and communicate. For example, if different resources were to propose plans for what Joan should do, then this idea of a theater-like stage suggests that they could settle their arguments in some kind of communal working-place. Thus Joan’s Cartesian Theater lets her use many familiar real-world skills by providing locations in space and time to represent the things ‘on her mind.’ So this could give her a way to start to reflect on how she makes those decisions.
Why do we find this metaphor to be so plausible and natural? Perhaps this ability to ‘simulate a spatial world inside the mind’ was one of the early seeds or catalysts that led our ancestors to be able to self-reflect. [(There is some evidence that some other animals’ brains develop map-like representations of environments they’re familiar with.) In any case, such metaphors now permeate our language and thought; imagine how hard it would be to think without our thousands of concepts like, “I’m getter closer to my goal.” Space-related models are so useful in our everyday lives, and we have such powerful skills for using them, that it would seem that almost always engaging them.
However, perhaps we’ve carried this too far, and the concept of a Cartesian Theater is now become an obstacle in the path toward further insights into psychology minds. For example, we have to recognize that a theatrical stage is merely a front, which conceals what’s happening in the wings; the processes behind the scenes are concealed inside the minds of the cast. What dictates what appears in the play—that is, chooses which subjects will interest us? How does Joan actually make her decisions? How could such a model represent comparing two different, possible ‘future worlds’ without maintaining two theaters at once?
"Homunculi are bogeymen only if they duplicate entire the talents they are rung in to explain. If one can get a team or committee of relatively ignorant, narrow-minded, blind homunculi to produce the intelligent behaviour of the whole, this is progress."— in Brainstorms 1978, p. 123.
All the ideas in this book agree with this. However, will raise serious questions about the extent to which our minds depend on a centralized workspace or bulletin board. We’ll conclude that the idea of a ‘cognitive marketplace’ is a good way to start to think about thinking, but that when we look more closely we’ll see the need for a great deal more architectural structure.
Citizen: Ridiculous. Of course I know what I’m doing right now—and thinking now, and feeling now. How do your theories explain why I sense a continuous stream of consciousness?
While the stories that we tell ourselves may seem to run in ‘real time,’ what actually happens must be more complex. To construct them, some resources must zigzag through memories; they sometimes look back to old goals and regrets, to assess our progress on previous plans.
Also, it seems safe to assume that different parts of your mind proceed at substantially different speeds, and with varied delays. So if you try to recount your recent thoughts a serial storylike tale about, your narrative machinery will somehow have to pick and choose, in retrospect, from various parts of those multiple streams. Furthermore, some of those processes look ahead in time, to expect or to anticipate events that are depicted by the ‘predicting machines’ that we’ll describe in §5-9. This means that the ‘contents of your consciousness’ are involved not only with ideas about the past but about your possible futures.
So the one thing you cannot be conscious of is what your mind is doing ‘right now’—because each brain-resource can know at best only what some others were doing some moments ago.
Citizen: I agree that much of what we think must be based on records of prior events. But I still feel there's something more than that, which makes which makes it so hard for use to describe our minds.
HAL-2023: Perhaps such things seem mysterious because your human short-term memories are so small that, when you try to review your recent thoughts, you are forced to replace the data you find by records of what you are doing right now. So you are constantly erasing the data you need for what you were trying to explain.
Citizen: I think I understand what you mean, because I sometimes get two good ideas at once—but, whichever one I write down first, the other leaves only a very faint trace. I presume that this must happen because I just don’t have enough room to store both of them. But wouldn't that also apply to machines?
HAL: No; that does not apply to apply to me because my designers equipped me with a way to store snapshots of my entire state in special "backtrace" memory banks. Later, if anything goes wrong, then I can see just what my programs have done—so that I can then proceed to debug myself.
Citizen: Is that what makes you so intelligent?
HAL: Only incidentally. Although those records could make me more "self-aware" than any person ever could be, they don’t contribute much to my quality, because I only use them in emergencies. Interpreting them is so tedious that it makes my mind run sluggishly, so I only stop to dwell on them when I sense that I have not been thinking well. I often hear people say things like, “I am trying to get in touch with myself.” However, take my word for it; they would not improve much by doing that.
Many thinkers have maintained that even after we learn all about how our brain-functions work, one basic question may always remain: “Why do we experience” things?” Here is one philosopher who has argued explaining ‘subjective experience’ could be the hardest problem of psychology—and possibly one that no one will ever solve.
Physicist: Perhaps brains exploit some unknown laws that cannot be built into machinery. For example, we don’t really know how gravity works—so consciousness might be an aspect of that.
This too assumes what it’s trying to prove—that there must be a single source or cause for all the marvels of ‘consciousness’. But as we saw in §4-2, consciousness has more meanings than can be explained in any single or uniform way.
Essentialist: What about the basic fact that consciousness makes me aware of myself? It tells me what I am thinking about, and this is how I know I exist Computers compute without any such sense, but whenever a person feels or thinks, this come with that sense of ‘experience’—and nothing else is more basic than this.
Chapter §9 will argue that it is a mistake to suppose that you are ‘aware of yourself’—except in a very coarse everyday sense Instead, you are constantly switching among different ‘self-models’ that you have composed—and each of these is based on different, incomplete set of incomplete evidence. “Experience” may seem quite clear and direct—but frequently it’s just plain incorrect, because each of your various views of yourself may be partly based on oversights, or other varieties of mistakes.
Citizen: That claim is so ridiculous that it makes me annoyed with what you said—and I know this in some special way that directly from inside myself, to tell me exactly what I think.
Your friends, too, can see that you are disturbed—and your consciousness fails to tell you details about why those words made you feel annoyed, or to shake your head that particular way, what caused you to use those particular words to say annoyed instead of disturbed? True, we can't see much of a person's thoughts by observing their actions from outside—but even when we 'watch from inside,’ it is hard to be sure that we really see more, in view of how often such ‘insights’ are wrong. So, if we take ‘consciousness’ to mean 'aware of our internal processes'—it doesn't live up to its reputation.
"The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents. We live on a placid island of ignorance in the midst of black seas of infinity, and it was not meant that we should voyage far. The sciences, each straining in its own direction, have hitherto harmed us little; but some day the piecing together of dissociated knowledge will open up such terrifying vistas of reality, and of our frightful position therein, that we shall either go mad from the revelation or flee from the deadly light into the peace and safety of a new dark age."—H.P. Lovecraft, "The Call of Cthulhu"
---Socrates: Imagine men living in an underground den, which has a mouth open towards the light—but the men have been chained from their childhood so that they never can turn their heads around and can only look toward the back of the cave. Far behind them, outside the cave, a fire is blazing, and between the fire and the prisoners there is a low wall built along the way, like the screen, which puppeteers have in front of them, over which they show the puppets.
---Socrates: And do you see men passing along the wall carrying all sorts of vessels, and statues and figures of animals made of wood, stone, and various materials, which appear over the wall? Some of them are talking, others silent.
Can you think about what you are thinking right now? Well, in a literal sense, that’s impossible—that each such thought would change what you're thinking now. However, you can settle for something slightly less—if you imagine that your brain (or mind) is composed of two principal parts: Let’s call these your A-brain and B-Brain.
Now suppose that your A-Brain gets signals that stream from such organs as eyes, ears, nose, and skin; then it can use those signals to discern some events that occur in the external world—and then it can react to these, by sending signals that make your muscles move—which in turn can affect the state of the world. By itself, it’s a separate animal.
However, your B-Brain has no such external sensors, but only gets signals that come from A. So B cannot 'see' any actual things; it can only see A's descriptions of them. Like a prisoner in Plato’s cave, who sees only shadows on that wall, the B-brain mistakes A’s descriptions for real things, not knowing what they might actually mean. What the B-Brain sees as its 'outer world' are only events in the A-brain itself.
Neurologist: And that also applies to you and me. For, whatever you think you touch or see, the higher levels of your brain never can actually contact these—but can only interpret the representations of them that your other resources construct for you.
When the fingertips of two ardent lovers come into intimate physical contact, no one would claim that this, by itself, has any special significance. For there is no sense in those signals themselves: their meanings to each lover lies in each one’s representations of the other one’s mind. . Nevertheless, although the B-Brain cannot directly perform a physical act, it still could affect the external world, albeit indirectly—by sending signals that change how A will react. For example, if A gets stuck at repeating itself, it might suffice for B just to interrupt.
Student: Like when I've misplaced my spectacles, I tend to keep seeking it on the same shelf. Then a silent voice reproaches me, suggesting that I look somewhere else.
In the ideal case, B could tell (or teach) A exactly what it ought to do. But even if B does not have such specific advice, it might not need to tell A what to do; it may suffice only to criticize the strategy A is using now.
Indeed, that could cause a serious accident. To prevent such mistakes a B-Brain must have appropriate ways to represent things. This accident would not occur if B represent ‘walking to a certain place’ as a single extended act—as in “Keep moving your legs till you’ve crossed that street”—or in terms of progress toward some goal—as in, ‘keep reducing the remaining distance.’ Thus, a B-brain could act like a manager who has no special expertise about how to do any particular job—but still can give ‘general’ guidance like these.
If A's descriptions seem too vague, B tells it to use more specific details.
If A is buried in too much detail, B suggests more abstract descriptions.
If what A is doing is taking too long, B tells it try some other technique.
How could a B-Brain acquire such skills? Some could be built into it from the start, but it should also be able to learn new techniques. To do this, a B-Brain itself may need help, which in turn could come from yet another level. Then while the B-Brain deals with its A-Brain world, that ‘C-Brain’ in turn will supervise B.
Student: How many levels does a person need? Do we have dozens or hundreds of them?
Of course, this is oversimplified. Such levels can never be clearly defined—because, at least in later life, each of those types of processes may use resources at other levels of thought. However, this framework will help us to start to discuss the kinds of resources that adults use—and some ways that these might be organized.
Student: Why should there be any ‘levels’ at all—instead of just one large, cross-connected cloud or resources?
If a system’s parts have too few interconnections, then its abilities will be limited.
But if there are too many connections, then each change will disrupt too many processes.
How to achieve a good balance of these? A system could start with clearly distinctive parts (for example, with more-or-less separate layers) and then proceed to make connections.
Embryologist: In its embryonic development, a typical structure in the brain starts out with more or less definite layers or levels like those in your A, B, C diagrams. But then, various groups of cells grow bundles of fibers that extend across those boundaries to many other quite distant places.
Or, the system could begin with too many connections and then proceed to remove some of them. Indeed, this also happened to us: during the eons through which our brains evolved, our ancestors had to adapt to thousands of different environments—and, every time this happened to us, some features that formerly had been ‘good’ now came to function as serious ‘bugs’—and we had to evolve corrections for them.
Embryologist: Indeed, it turns out that more than half of those cells proceed to die as soon as they’ve reached their targets. These massacres appear to be a series of post-editing’ stages in which various kinds of ‘bugs’ get corrected.
This reflects a basic constraint on evolution: it is dangerous to make changes to the older parts of an animal, because many parts that later evolved depend on how the older ones work. Consequently, at every new stage, we tend to evolve by adding ‘patches’ to structures that are already established. This led to our massively intricate brains, in which each part works in accord with some principles, each of which has many exceptions to it. This complexity is reflected in human Psychology: where each aspect of thinking can be partly explained in terms of neat laws and principles—but each such ‘law’ has exceptions to it.
The same constraints appear to apply whenever we try to improve the performance of any large system—such as an existing computer program—by adding more fixes and patches on top, instead of revising the older parts. Each particular ‘ bug’ that we remedy may eventually lead to more such bugs, and the system keeps growing more ponderous—and this seems to apply to our present-day minds.
This chapter began by presenting a few widely held views of what “consciousness” is. We concluded that people use that word for a great suitcase of mental processes that no one yet thoroughly understands. The term ‘conscious’ is useful enough in everyday life—and seems almost indispensable for talking on social or ethical levels—because it keeps us from being distracted by wanting to know what’s inside our minds. It is the same for most other psychology-words, such as understanding, emotion, and feeling.
However, when we don’t recognize that we are using suitcase-words, then we may fall into the trap of trying to clearly define what those kinds of words ‘mean.’ Then we get into trouble because we do not have clear enough ideas about what our minds are and how their parts work. So, if we want to understand the things that human minds actually do, we will have to dissect our mental processes into parts that we can analyze. The following chapter will try to explain how Joan’s mind could do some of the sorts of the things that people can do.
 Fodor, J. A., Can there be a science of mind? Times Literary Supplement. July 3, 1992, pp5-7.
 In sci.psychology.consciousness, 15 Jun 96.
 There are important exceptions to this. It would seem that experts like J.S. Bach developed ways to accomplish more multiple, yet still similar goals in parallel. However, as their skills improve, most such experts become less and less able to tell the rest of us how they do them.
 William James discussed this extensively. See: http://psychclassics.yorku.ca/James/jimmy11.htm. Several other more modern ideas about this are developed in Daniel Dennett’s 1991 book, Consciousness Explained.
 So, despite a popular intuition, research on parallel processing has shown that such systems are frequently prone to end up accomplishing less for the same amount of computational power Nevertheless, if that cost can be borne, then the final result may come sooner!
 Chapter §8 will propose more details about how our memory structures are organized to so swiftly deliver such information. Basically, when a problem arises, some processes may start to solve it before other processes formulate questions about it.
 See §25.4 of The Society of Mind, p257.
 ” —psyche-b@listserv.uh.edu, 29 Sep 1997.
 In Outlines of Psychology, 1897.
 This idea is explained in more detail at http://web.media.mit.edu/~minsky/papers/MatterMindModels.html.
 In a discussion on the newsgroup comp.ai.philosophy, 7 Feb 1992.

References: §4

§4

§4
 §9
 §8
 §6
 §4
 §9
 §9
 §5
 §4
 §9
 §8
 §25