PROCEEDINGS of the
23rd International Congress on Acoustics
9 to 13 September 2019 in Aachen, Germany

How does what we hear sound? The qualia problem in acoustics
Michael HAVERKAMPCologne, Germany

ABSTRACTThe development of quantitative methods for description of auditory perception has extended the possibilityof detailed analysis and optimization of sound properties. Psychoacoustic quantities help to assess audible
features like loudness, roughness, sharpness, tonality and pitch. But why is it still not possible to precisely
describe how an auditory phenomenon really manifests itself within the subjective perception of anindividual? This would be essential in terms of defining targets for sound design, the legal determination ofbranding sounds and comprehension of the annoyance of auditory incidents.
In order to approach this question, the so-called qualia problem shall be outlined in view of auditory
perception. It is evident that development of effective metrics for sound evaluation is hindered by this matter.
Even under the hypothetical presumption that a complete set of relevant parameters could be established,trying to find a definite description would fail due to the impossibility of including the perceived subjective
content. Moreover, the holism of perception impedes achieving this goal.
Various approaches will be discussed briefly:
1.
2.
3.

Verbalization of sensations and onomatopoeia
Inclusion of further modalities as tertium comparationis
Parameters related to the human body, like movement

Keywords: Philosophy, Psychoacoustics, Sound Perception, Qualia, Phenomenology

1. INTRODUCTION
Since mid-20th century, numerous quantitative methods for the description of auditory perception
have been developed by means of psychoacoustic research. This has provided a variety of possibilities
for the evaluation and optimization of sound features. Psychoacoustic quantities are indispensable
measures for the assessment of audible sound properties like loudness, roughness, sharpness, tonality,
pitch, and many others. But it is evident, however, that it is not yet possible to achieve a precise andcomplete description of how sounds manifest themselves within an individual’s consciousness.
The definition of sound properties via psychoacoustic quantities as required by sound design andnoise control faces two major problems. First, usually various sounds occur with the same set ofquantitative descriptions, but sounding rather different. Secondly, a set of psychoacoustic parameters
includes neither representational aspects of function and meaning nor iconic (associative) attributions
to sound sources and complex environments. This problem similarly exists for the design of objects
with respect to other modalities. In this context, the word modalities is used as synonym of senses. As
an example, a variety of parameters can be found which relates to the visual and haptic appearance ofa tea cup, like color, saturation, grain depth, surface roughness, weight, grip and many others. Such setof parameters, however sized, will not entirely tell us what the cup looks like, or how it feels if wereach for it. In fact, quantitative perceptual parameters are indispensable for product optimization andmanufacturing control. The sets of parameters applied are often effectual with respect to the given task.
In order to provide a total description, which includes all aspects of auditory sensation, however, it is
not sufficient to characterize single perceptual aspects by means of single physical measures, even if1418

those values have been carefully measured and verified for relevance by means of perception
experiments.
The challenge is to establish ways to appropriately communicate what it is like to see, feel, taste orhear something within our individual consciousness. First, the principal problem shall be described. Is
it possible to sidestep the self-contained, subjective character of perceptual content?

2. CAN WE COMMUNICATE HOW IT SOUNDS?
The phenomena which constitute the content of subjective experience of a mental state shape theentity of perceptual qualities, the qualia (Latin qualis: “what is it like”). These qualia form thephenomenal consciousness of every individual, i.e. what we experience as world, as reality during
daily life. Usually the projection processes of our perceptual system affect us to mistake thisimpression of reality with physical facts. In our awareness, the physical world is merely accessible via
the filtering and model-generating processes of perception. Thus, qualia exclusively exist within thelimitations of individual consciousness. Communication about the content of qualia needs a reference
to objects or animals outside individual consciousness which are perceived – and can be referred to –by the participants of the communication process. When participants perceive the tone of an oboe, they
are thus able to agree on a common description that ideally appears plausible for everyone. If thishappens, we feel that the problem of communicating subjective content may be solved for situations ofdaily life. Contrary to those feelings, however, the qualia problem persists. The subjective certainty of
having solved this problem is illusive, but it enables us to shape our life in a practical manner. We
simply presume a priori that the perception of other human beings is similar to ours.
In order to approach this complex of problems, it is essentially required to verbally separat e thefield of subjective phenomena from the field of physical facts which stimulate our senses. Therefore,
as an example, we need to distinguish auditory and visual phenomena from the acoustic and optic
instances. This is not self-evident in the case of common speech. As an example, we need to avoid theterm optic illusion in favor of visual illusion, because the illusion does not exist physically. While thequalia problem persists for all senses, examples applicable to various modalities will be discussed
below, with a focus on visual sensation. All assumptions, however, can easily be transferred to theauditory field and the other senses. With regard to the qualia problem, the term color thus can besubstituted by timbre, as a synonym of tone color.
Above all, the fundamental boundedness of language accounts for the problem. Descriptions ofperceptual content are essentially vague. In many cases, we cannot find any description which seems
to be appropriate. Various descriptions can only be found if based on others which are already known.
For example, communication of odor is usually based on iconic features: it smells like citrus, apple,
coal fire, licorice or fuel. Communication regarding odor, however, fails if we do not find any
associative term that fits. Then, few basic aspects of smell like sharp or musty are left. These words
only convey a rudimentary impression of the respective sensation. This also applies to verbalizationsof color sensations. It can be assumed that the subjective perceptual qualities which are expressed withsuch words are also not constant between individuals, but feature are typically blur. Borders between
color categories shift individually. They also can deviate depending upon ethnic group or culturalcircle (1).
On the occasion of the description of auditory perception, technical functions which are actually or
seemingly reflected by the thus generated sounds, play a role. It may sound like a defective wheel
bearing or it may suggest that coffee beans are missing in the coffee grinder. Communication is thus
enabled with respect to the mapped function.
The qualia problem includes much more than simply the lack of accuracy of language. We need toderive our vocabulary by means of references towards objects which persist outside of ourconsciousness. This is due to the fact that we are unable to create the required expressions from thecontent of consciousness itself. The transformation of all the features of external objects into theindividual consciousness cannot be understood from outside. The conclusive presumption is that thosetransformations may be quite different for every subject. We simply know that we have agreed oncommon descriptions.

1419

Nonetheless, what could be the result of a direct comparison of the content s of consciousness?
Some thought experiments can illustrate the challenge.
Do we really know whether a timbre which is described by various listeners in a similar way, e.g. as
ringing of a bell, is represented by the same sensation within a single individual? This problem is
equivalent to the uncertainty over a similar perception of color. As an example, we have learned toname light of the wavelength of 460nm as “blue.” In this case, however, it is absolutely undetermined
what it is like to see this color in a person’s consciousness. It is not possible to prove the similarity of
individual sensations by means of verbalization. It has merely been agreed on to use consistentwording, i.e. a linguistic convention for the individually perceived hue that is stimulated by a specificwavelength of light. This agreement functions independently of the fact which color actually appearson the subjective “inner screen.” The verbal assignment is definite even in the case the blue of one
person is the red of another subject. Thus, variations of the subjective content of perception cannot be
verified on the basis of linguistic conventions. This is due to the fact that it persists in the nature ofconventions to eliminate differences. Communication becomes thus very clear, but with the drawbackthat subjective differences are eliminated from the communication process, as if they actually do notexist at all. Although these conventions normalize the usage of language, during daily life a conceptual
blur between physical (here: optic) and mental field (here: visual perception) impedes the discussion :1.

“The sensation blue” does not mean a subjective impression similar to all individuals, but it
simply is the linguistic convention for the hue that is visualized when seeing a light of awavelength of 460nm.

2.

With the term “blue light,” subjective perceptual content is transformed and thus projected into
the physical world. The physical phenomenon, however, exhibits a spectrum that ischaracterized by means of the intensity of its spectral components. In contrast to its perception,
the physical phenomenon itself does not include any color.

3.

It can be verified by means of a physiological or behavioristic experiment (i.e. by observing
behavior), if an animal is able to perceive “blue light.” From these results, however, it cannot be
concluded that a particular kind of color sensation may already exist within the animal’s
consciousness.

As mentioned before, it is not possible to prove the identity of a subjective color sensation of different
individuals. Moreover, it is impracticable to verify whether an individual perceives colors which arecompletely unknown to someone else. It is well known that various animals are able to perceive lightof different wavelengths than humans. The question is whether light stimulus of those wavelengths
will cause sensations unknown to us, or will the human palette of color sensations be adapted to thedifferent wavelength range of the stimulus. Does a fixed set of color sensations exist (i.e. the various
colors perceived) which is available for all animals and human beings independently of the range of
wavelengths/frequencies? Alternatively, amongst the numerous possibilities, it could be the fact thatthe perceived hues feature a fixed relation to the stimulating wavelength. In the case of an animal
which is able to see ultraviolet light, this would certainly cause color sensations which are unknown
and even unimaginable to us.
The question is similar for auditory perception: how can we imagine the subjective sensation ofanimals which are able to hear ultrasonic sounds? Does this cause pitch and timbre perceptions wealready know from our own experiences? In this context, bats are of special interest. Some bat species
can hear sounds up to a frequency of 120kHz to orientate themselves and to locate insects. We are notable to imagine the subjective world of a bat, i.e. to imagine how it is like to be a bat. Nagel referred tothis fact in order to demonstrate the implications of the qualia problem for the relation between theobjective, physical world on one hand and the subjective world of any individual on the other (2).

3. SELF-AWARENESS AND THE OTHER MINDS PROBLEM
Nagel’s point is neither to understand the perceptual ability of a bat in detail nor to imagine with ahuman’s consciousness how it would be like to feature the physical skills or behavior of these animals.

1420

“In so far as I can imagine this (which is not very far), it tells me only what it would be like for me tobehave as a bat behaves. But that is not the question. I want to know what it is like for a bat to be a bat(2, p.439).”
The only experience which an individual gains with the manifestations of consciousness relates to
one’s own self-awareness. This also applies between very similar animals. The content of other minds
is always inaccessible. “Reflection on what it is like to be a bat seems to lead us, therefore, to theconclusion that there are facts that do not consist in the truth of propo sitions expressible in a humanlanguage (2, p.441).” Thus, due to their subjective nature, the individual representations ofconsciousness cannot be conveyed by language. At least, the possibilities of verbal communication arevery limited.
An extreme conclusion from this fact would be to suppose that for each subject it is certain that only
one consciousness already exists. Each subject is exclusively aware of their own consciousness (3).
The probability that one’s own awareness plays a special role amongst all human beings, however, issmall. Therefore, such solipsism is not a promising approach. At least, the impossibility of real precise
communication of the subjective content of consciousness indicates that the relation of our perception
to the external world shows fundamental uncertainties. A radical consequence has been stated by
Wittgenstein within the preface of his Tractatus Logico-Philosophicus: “What can be said at all can besaid clearly; and whereof one cannot speak thereof one must be silent. ” / “Was sich überhaupt sagen
lässt, lässt sich klar sagen; und wovon man nicht reden kann, darüber muss man schweigen ” (4,preface, English translation by Ogden/Ramsey).
Even if it is rather unlikely that the conscious content of similar animals or of human beings shows
extreme deviation among various individuals, it is evident that a specific point of view is applied
towards everything that can be sensed. “Whatever may be the status of facts about what it is like to bea human being, or a bat, or a Martian, these appear to be facts that embody a particular point of view ”(2, p.441). This is particularly essential regarding the inner view of conscious beings. With respect tothe external world, however, Nagel admits that it already is possible to find objective descriptions. In
fact, the external world is also observed from individual points of view. But contrary to the subjective
content of consciousness, it is independent of any individual prospect (2, p.443).
As a core challenge of philosophy, the qualia problem touches questions on the level of connectionof consciousness to the body, especially to its electro-chemical brain activity, and to what extent thepsychic phenomena can be derived from (i.e. be reduced to) the physical processes. In ancient andmedieval times, the mind–body problem has usually been explained as a result of dualism between thefelt body and a pure spiritual entity named the soul. During the modern era concepts gained importancewhich tended to trace all activities of the mind back to physical processes of the body. In the context ofapproaches towards physicalism, one tries to completely reduce psychic incidents like perception andrepresentations like qualia back to physical processes (5). That is the approach favored by actual brain
research. Although sophisticated, up-to-date measurement techniques have been applied, it is still notpossible to verify the correlation of content of consciousness with physical brain activity in theminutest detail, as needed.
Referring to the cognition science of the 1980s and 90s, Varela distinguishes among four modern
schools of thought around the relation of brain and consciousness (5, p.339f). This list excludes boththe classical dualistic approach and attempts at solutions focused on quantum mechanics:-

Neuro-Reductionism, which eliminates “the pole of experience” and the psyche itself bydescribing brain functions as manifestation of neuronal activity.

-

Functionalism, which tends to “replace the link between cognition and consciousness (…) by
the link between cognition and its corresponding functional or intentional states” (5, p.340).

-

Mysterianism, which states that in general the connection of consciousness to body functions
and physics cannot be clarified. “The hard problem is unsolvable” (5, p.340). In Varela’s view,
Nagel is a representative of this approach.

1421

-

Phenomenology, an approach which is “grounded on a peculiar move to explore experience” (5,p.341). The school of phenomenology accepts that subjective experience cannot be completely
reduced to neuronal processes. The individual experience is the foundation for description of
consciousness.

The radical reduction of the base question as intended by neuro-reductionist thinking does not help
with our question, because it does not enable a conclusive approach which is appropriate for thecomplexity of all aspects of brain activity. It eliminates the most challenging aspects from the question.
On the contrary, phenomenology is a promising approach which does not put the subjective aspects ofperception and awareness into question. Thus, phenomena of individual consciousness gain a crucialrole in the description and explanation of mental processes, and even for the external world. During the20th century, this school of modern philosophy had significant advocates, from Husserl and Heidegger
to Varela and Schmitz (5, 6, 14-16).
In the context of the challenges described above, and with a view to the considerable progressalready made, a psychoacoustic description of sounds is still fragmentary. A principle solution for thequalia problem would be essential. For example, this includes the definition and communication ofdevelopment targets for sound design, the legislative fixing of “sound marks,” and the understandingof annoyance caused by auditory events. As mentioned above, conventions of communication can only
be developed in the case that two individuals perceive an object simultaneously that is located outsideof their own subjective world. This object then serves as tertium comparationis, an object one refers to,because it is accessible to both subjects and not influenced by the observer’s subjective points of view.
Are there any ways to circumvent the qualia problem?

4. APPROACHES
Three basic approaches shall briefly be discussed below. The related topics are just slightly touched
upon. Those approaches look promising with a view to the enhancement of possibilities to communicate
the subjective content of perception. It is intended to roughly outline potential fields of activity in which
in-depth work could be beneficial.
4.1 Onomatopoeia
A classical method for the communication of auditory qualities is the vocal imitation of sounds byonomatopoeia (echoism). Such reference sounds add sound qualities to the communication whichcannot be expressed by the semantic content of language. It is also common practice to add plausibility
to words which describe sounds with additional nonverbal content by means of onomatopoeia. As
examples, sound can show auditory qualities like rattling or squealing. Those onomatopoetic aspects
are reproduced when reading a text. Those expressions are thus very popular for comics, like crank,
honk or ding, in order to indicate specific vehicle sounds, amongst many others (7).
In the field of sound design and acoustics engineering, the computer-based generation of reference
sounds is a contemporary method to demonstrate sounds as a basis for communication. Even then, thesubjective manifestation of the reference sound within each individual is still unknown. Thus, bymeans of both onomatopoeia and artificial reference sounds, it is not possible to avoid the uncertainty
around subjective qualia. An additional objective representation is simply added that serves as tertiumcomparationis, but it likewise persists outside the individual perceptual world. This representation issuitable for comparisons of different sounds, but does not clarify what it is like to hear a sound within
another mind.
4.2 Cross-modal comparisons
In various cases subjective experience extends the sensory modes which are concurrently
stimulated. Will these cross-modal interactions serve as tertium comparationis in order to specify the
subjective perceptual content more precisely? Notably audition often provokes additional visualsensations, which may be enabled by iconic (i.e. associative) or synesthetic connections within theperceptual system (8, for differentiation of various ways of cross-sensory connection see 9). Specific,
genuine types of synesthesia occur quite rarely. The subjective perceptual world of every human,1422

however, is shaped by multisensory processes. These processes form a subjective environment, which
Emrich names a “private perceptual world” (“private Wahrnehmungswelt”). Furthermore, he statesthat it can be attributed to be “transmodal” (10, track 4). In his view, synesthetic connections can“deepen the experience of reality” (“Wirklichkeitsvertiefung”; 10, track 12). Thus, the impression
could arise that corresponding visual sensations are indeed feasible as tertium comparationis to more
precisely specify the communication of perceived sound features.
An interesting question is the ability of perfect pitch perception: which qualia provide theinformation about the pitch presented? How is this information coded within the perceived world? Is
the denomination of the perceived musical note (e.g. “Gis”) represented in the visual system? Or is thenote visualized as seen in the score? Or is the specific key of the piano visualized, as it is located on the
keyboard? In fact, some subjects with perfect pitch perception use synesthetic color sensations for theidentification of pitch. During an investigation of 69 individuals with perfect pitch, Wellek identified
30% (21 test subjects) with synesthetic perception, who unintentionally connect tone pitch with visualcolor (11, p.296). As described by this early study, the visual color indeed serves as tertiumcomparationis for recognition of tone pitch.
Beside specific synesthetic phenomena, all humans intuitively make use of visual analogies(correspondences) for description of auditory features (12). Typically, such analogies are applied to
the notation of music for musical scores as well as for the documentation of acoustic measurements. In
this case, the fundamental cross-modal relation between height of visual objects and tone pitch isapplied. Such correspondences are spontaneously configured by the perceptual system. Therefore, thecorrelation of high tone symbols and high pitch within a musical score appears to be plausible anduseful for most people. In principle, there are some indications that common cross-modal correlations
may help to paraphrase qualia.
The description of visual phenomena which are connected to auditory incidences, however, does
not generally avoid the qualia problem: Auditory events, which cannot be described exactly, aresupplemented by visual phenomena, which are also not precisely describable.
4.3 Body-related quantities and movement
An additional method for the communication of perceived sound features can include taking bodystates and body movement into consideration. Conditions of tension, which are well known frombodily perception, can thus be found as attributes of sound. Initially, movement is an activity that canbe provoked by music – dance is one example. Moreover, movement is sensed as an essential attributeof music and sound. It is closely connected to the temporal variation of perceived auditory occurrences(13). In principle, sensation of movement is a multi-sensory incident. Schmitz outlines the importanceof the relation between the stimuli emerging from exterior and the body of the perceiving subject (14,15). Perception is by no means a pure neuronal process which is executed in complete isolation fromany bodily activity outside the central nervous system. Schmitz outlines that movement thus ranks
amongst a set of “bridge qualia.” These qualia are perceived at both one’s own body and at
encountering entities – “Brückenqualitäten, die sowohl am eigenen Leib gespürt als auch an
begegnenden Gestalten (…) wahrgenommen werden können (16, p.40).”
Frequently, stimulation of sense organs causes bodily reactions which can be measured objectively.
In the past, in view of behavioristic concepts, measurements have been judged to be the only reliable
method for evaluating perceptual phenomena. This excludes vague descriptions about subjective
perceptual worlds. Such an approach, however, systematically ignores the fundamental peculiarity ofthe perceptual content as a major part of individual consciousness. Damasio concludes frominvestigations of brain activity that even emotions are not always connected to measurable bodyreactions. In many cases it is just a simulation of these reactions by the brain which leads to feelings
rather similar to real body states (17). Nevertheless, this simulation is only possible if the body and its
reactions have been involved before.
However, even if interactions of the perceptual system with body activities are considered, thesubjective world is entirely unapproachable from other people’s minds. Again, the qualia problem
cannot easily be circumvented.

1423

5. CONCLUSIONS
It is evident that a principle discrepancy exists between the stimuli of the physical world and thesubjective representations which appear within individual consciousness. Furthermore , it results fromthe above-mentioned considerations that this gap cannot be bridged logically by means of today’s
knowledge. Measurable quantities of physical stimuli are thus not sufficient to completely describe theindividual perceptual world which they evoke. Finding a solution for this challenge is not simply aquestion of the appropriate effort required by subjective studies or a question of the time spent withpsychophysical tests needed to achieve a complete reduction of perceptual content down to thefeatures of the acoustic stimuli. The effort spent over past centuries has already approached a borderwhich in principle hinders a satisfying solution. It is difficult to precisely define, however, what it isthat prevents the reduction. Fundamental understanding of this difficulty would be a major step of aconstructive approach. Nevertheless, it is evident that this would require a solution for the classicalmind–body problem (“Leib-Seele-Problem”).
Shall we thus dismiss any attempts to use verbal descriptions and cross-modal comparisons toparaphrase subjective qualia? Wittgenstein’s famous sentence (which he revised later) rejects any
possibility of approximating the content of perception by circumscriptions instead of fumbling for
precise verbalizations. For this purpose, it will be necessary to apply all feasible ways of description inparallel, including verbal and non-verbal approaches. This tends to advance the inner world of otherminds by orbiting the subjective core. By the means mentioned before it will presumably not be
possible to achieve a precise transformation of subjective content into language. Nevertheless, theresult might come close to appropriate communication. This already happens via specific modes of
expression in the arts, design and music, which help to access the unutterable. Literature aspires
towards a similar objective, especially lyric poetry.
With respect to the future of the field of psychoacoustics, we cannot count on the assumption that acomplete description of phenomena of auditory perception by means of sound measurements will soon
be feasible. It is thus essential to always understand both the physical and the perceptual side.
Perceived qualia must be approached by means of sophisticated experiments based on extensive
groups of test individuals. Experts working with auditory perception need to be proficient in bothfields. In the future it will be essential that research and educational activities are more strongly
focused on the indispensable broadening of the scientific horizon towards the perceptual side.

ACKNOWLEDGEMENTS
I would like to express my gratitude to Dr. Monika Gatt for initiating the session on philosophy andacoustics and for her excellent support on this topic.

REFERENCES
1. Davidoff J, Davies, IRL, Roberson D. Colour categories in a stone-age tribe. Nature 1999; 398(6724):
203-4, DOI: 10.1038/18335
2. Nagel T. What Is It Like to Be a Bat? The Philosophical Review 1974, Vol.83, No.4:435-50,
http://www.jstor.org/stable/2183914 (14.1.2019)
3. Nagel T. What Does It All mean? Oxford: University Press; 1987.
4. Wittgenstein L. Tractatus Logico-Philosophicus. Wien: 1918. Side-by-side edition, v0.53, Feb. 5 2018.
http://people.umass.edu/klement/tlp/ (10.5.2019)
5. Varela FJ. Neurophenomenology. A Methodological Remedy of the Hard Problem. In: Shear J, editor.
Explaining Consciousness – The 'Hard Problem'. Cambridge, MA: MIT Press; 1997. p.337-57.
6. Merleau-Ponty M. Phänomenologie der Wahrnehmung. Berlin: de Gruyter: 1966. Orig.:
Phénoménologie de la Perception. Paris: Gallimard; 1945.
7. Havlik EJ. Lexikon der Onomatopöien. Die lautimitierenden Wörter im Comic. Frankfurt: Dieter
Fricke; 1981.
8. Simner J, Hubbard EM, editors. The Oxford Handbook of Synesthesia. Oxford: Univ. Press; 2013.
9. Haverkamp M. Synesthetic Design. Handbook for a multi-sensory approach. Basel: Birkhäuser; 2013.
10. Emrich HM, Lube M, Waldeck M. Ist mein Blau dein Blau? Neurophilosophische Überlegungen zur
Synästhesie. Audio-CD. Berlin: c+p supposé; 2008.
11. Wellek A. Das absolute Gehör und seine Typen. Leipzig: Johann Ambrosius Barth; 1938.

1424

12.Spence C. Crossmodal correspondences: A tutorial review. Atten Percept Psychophys (2011) 73:971–
995. DOI 10.3758/s13414-010-0073-7
13.Truslit A. Gestaltung und Bewegung in der Musik. Berlin: Chr. Friedrich Vieweg; 1938.
14.Schmitz H. Die Wahrnehmung. System der Philosophie, Bd. 3 Der Raum, Teil 5. Bonn: Bouvier; 1978.
15. Schmitz H, Müllan RO, Slaby, J. Emotions outside the box—the new phenomenology of feeling andcorporeality. Phenom Cogn Sci (2011) 10:241–259, DOI 10.1007/s11097-011-9195-1
16. Schmitz H. Kurze Einführung in die Neue Phänomenologie. Freiburg: Karl Alber; 2010².
17. Damasio AR Descartes’ Error. Emotion, Reason and the Human Brain. New York: G. P. Putnam’s Son;
1994.

1425