# Explaining Color Evolution, Color Blindness, and Color Recognition by the Decoding Model of Color Vision

Chenguang Lu

► To cite this version:

Chenguang Lu. Explaining Color Evolution, Color Blindness, and Color Recognition by the Decoding Model of Color Vision. 11th International Conference on Intelligent Information Processing (IIP), Jul 2020, Hangzhou, China. pp.287-298, 10.1007/978-3-030-46931-3_27. hal-03456975

HAL Id: hal-03456975

https://inria.hal.science/hal-03456975

Submitted on 30 Nov 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Explaining Color Evolution, Color Blindness, and Color Recognition by the Decoding Model of Color Vision

Chenguang Lu

College of Intelligence Engineering and Mathematics,
Liaoning Technical University, Fuxin, Liaoning, 123000, China
lcguang@foxmail.com

**Abstract.** The author proposed the decoding model of color vision in 1987. In- ternational Commission on Illumination (CIE) recommended almost the same symmetric color model for color transform in 2006. For readers to understand the decoding model better, this paper first introduces the decoding model, then uses this model to explain the opponent-process, color evolution, and color blindness pictorially. Recent references on the decoding and reconstruction of colors in the visual cortex induce a new question: what is the decoding algorithm from gan- glion cells to the visual cortex? This paper also explains the decoding algorithm. The decoding model is explained as a fuzzy 3-8 decoder. The fuzzy logic used is compatible with Boolean Algebra. The model first obtains the median *M* of three cones' outputs *B*, *G*, and *R*, and then obtain three opponent signals by *B*, *G*, and *R* minus *M* respectively. This model can unify Young and Helmholtz's tri-pig- ment theory and Hering's opponent theory more naturally than the popular zone models. It is symmetrical and compatible with the popular color transform method for computer graphics. The transform from the RGB system to HSV sys- tem according to the decoding model is introduced. Several fuzzy 3-8 or 4-16 ... fuzzy decoders can be used to construct a Decoding Neural Network (DNN). The decoding algorithm from ganglion cells to the visual cortex can be explained with a two-layer DNN. The reasonability of the decoding model and the potential ap- plications of the DNN are discussed.

**Keywords:** Neural Computing, Color Blindness, Color Evolution, Color De- coding, Color Recognition, Neural Network, Computer Vision.

## 1 Introduction

Young and Helmholtz's tri-pigment theory [1] and Hering's opponent theory [2] on color vision have been competing for a long time. A compromising viewpoint accepted widely is that color signals exist in tri-pigments at the zone of visual cones and in op- ponent signals at the zone of visual nerves [3]. The mathematical model with this view- point is called the zone model [4]. There are many improved versions of the zone model [5-8]. However, why are color signals processed in this way? and how has color vision been evolving? The answers are still unclear. To answer these questions, the author of this paper built a model of color vision named the decoding model [9], which is new
version of the zone model, and has been verified by predicting color appearance [10]. After the author explored the information conveyed by color vision, he developed a semantic information theory [11], which can be used for machine learning [12]. Recently, the author learned that

* international Commission on Illumination (CIE) recommended almost the same symmetric color model for color transformation [13];

* the human or macaque temporal cortex can decode and reconstruct colors [14-21].

These facts impel the author to introduce his further studies on color vision. The purpose of this paper is to help readers understand opponent-process, color evolution, color blindness, color decoding and reconstruction, and color recognition better.

The decoding model should be helpful to researchers who build the color vision mechanism of robots, and it may enlighten someone to build a neural network with the fuzzy logic [22] that is compatible with Boolean algebra for applications.

To the best of the author's knowledge, no other researchers use fuzzy logic that is compatible with Boolean Algebra for color models or neural networks, not to mention explaining color evolution, color blindness, and color decoding and reconstruction.

The main contributions of this paper are

* It provides more explicit explanations of color evolution and colorblindness than the popular zone models.

* It shows a simple algorithm for decoding and reconstructing colors in the human temporal cortex.

* It proposes the decoding neural network for potential applications.

The following sections first introduce the decoding model and the color transform from the RGB system to the HSV system, then use this model to explain the opponent-process, color evolution, color blindness, and color decoding and reconstruction in the visual cortex. Discussions include topics about the reasonability of the decoding model and the similarity between the decoding model and the neural network.

# 2 The Decoding Model of Color Vision

## 2.1 The Fuzzy Logic compatible with Boolean Algebra

The 3-8 decoder is frequently used in computers or numerical circuits for selecting one register or memory from eight. If $B$, $G$, and $R$ are binary switching variables taking values from set $\{0,1\}$ as three inputs to a 3-8 decoder, then eight outputs will be $[\overline{BGR}]$, $[\overline{BGR}]$, $[\overline{BGR}]$, $[\overline{BGR}]$, $[\overline{BGR}]$, $[\overline{BGR}]$, and $[\overline{BGR}]$ ([...] Denotes a logical expression). For example, if $B=G=0$ and $R=1$, then $[\overline{BGR}]=1$, otherwise $[\overline{BGR}]=0$.

Now, suppose that $B$, $G$, and $R$ are continuous switching variables, i.e. $B$, $G$, and $R$ take continuous values from set $[0, 1]$. With the special continuous-valued logic or fuzzy logic [19], we can extend the binary 3-8 decoding into the fuzzy 3-8 decoding [9,10]. The values of output codes are illustrated in Fig.1.
<figure><img src="image_2.png" /><figcaption>Fig. 1. Relationship between three inputs B, G, and R and eight outputs of a fuzzy 3-8 decoder. When B>G>R, only four outputs are not zero.</figcaption></figure>

Let function max(a, b) be the maximum of a and b, min(a, b) be the minimum of a and b, and so on. Hence

$$ [\overline{BGR}] = 1 - \max(B, G, R), \quad [\overline{BGR}] = \max(0, R - \max(B, G)) \quad (1) $$

$$ [\overline{BGR}] = \max(0, \min(G, R) - B), \quad [BGR] = \min(B, G, R) $$

The others can be calculated in like manner.

## 2.2 Transform from the RGB System to the HSV System

Let B, G, and R be tri-stimulus valves from cones. How do we simulate the visual system to obtain H (hue), S (saturation), and V (brightness) from B, G, and R? For any given color denoted by (B, G, R), there is

$$ (B, G, R) = [\overline{BGR}](0, 0, 1) + [\overline{BGR}](0, 1, 1) + [\overline{BGR}](0, 1, 0) + [BGR](1, 1, 0) \\ + [BGR](1, 0, 0) + ([\overline{BGR}](1, 0, 1) + [BGR](1, 1, 1)) \quad (2) $$

which means that any color can be decomposed into the combination of white and six unique colors in different ratios. In the above equation, (0, 0, 1) stands for the most saturated red, i.e., unique red, and the coefficient $[\overline{BGR}]$ is the redness, and so on.

It is coincident that only three items on the right of Eq. (2) may be non-zero for a given color, and the three cardinal vectors with 0 and 1 or unique colors must be at the three vertexes of one of six sectors in Fig.2. Hence Eq. (2) can be changed into

$$ (B, G, R) = m_1 e_1 + m_2 e_2 + [BGR](1,1,1) \quad (3) $$

where $e_1$ and $e_2$ are two cardinal vectors or unique colors, and $m_1$ and $m_2$ are corresponding coefficients or output codes' values.

Suppose the angles at which $e_1$ and $e_2$ are located (see Fig.2) are $H_1$ and $H_2$. Let
$$ 
\begin{aligned}
H &= (m_1 H_1 + m_2 H_2) / (m_1 + m_2), & C &= m_1 + m_2, \\
V &= m_1 + m_2 + [BGR] = \max(B, G, R), & S &= C / V.
\end{aligned} 
\quad (4) $$

<figure><img src="image_3.png" /><figcaption>Fig. 2. The Decomposition of color (B, G, R).</figcaption></figure>

Then *H*, *C*, *V*, and *S* will represent hue, colorfulness, brightness, and saturation of (B, G, R) properly if *B*, *G*, and *R* are obtained from appropriate linear and nonlinear transforms of spectral tri-stimulus values *X*, *Y*, and *Z* [10]. According to the decoding model, the relationship between brightness, colorfulness, whiteness, blackness, and *B*, *G* and *R* is shown in Fig.3, where med(*B*, *G*, *R*) is the median or second one of *B*, *G* and *R*. For example, med(1, 3, 5)=3, med(1, 2, 5)=2, med(1, 5, 5)=5, med(1, 1, 5)=1.

<figure><img src="image_4.png" /><figcaption>Fig. 3. Relationship between *B*, *G*, *R* and brightness, colorfulness, whiteness, and blackness.</figcaption></figure>

Later the author found the above transform is the same as that proposed by Smith earlier [23]. The differences are:

* *B*, *G*, and *R* in Smith's transform are the signals of three primary colors lightening a pixel of a display instead of the tri-stimulus values of visual cones;

* Smith uses “if-then” programming language rather than logical operations;
* The logical expressions for the opponent-process only exist in the decoding model.

## 2.3 Opponent-Process

We use Venn's Diagram to show the essence of the opponent-process. Let $∩, U$, and $^c$ denote three set operations: intersection, union, and complement respectively, and let $B, G$, and $R$ represent the three circular fields respectively (see Fig.4). For convenience, we also use "-" for complement operation and omit $∩$. Then, the eight fields can be represented by $[\overline{BGR}], [\overline{BGR}], [\overline{BGR}], [\overline{BGR}], [BGR], [\overline{BGR}], [BGR]$, and $[BGR]$.

From $B, G$, and $R$, we can first obtain

$$M = BG \cup BR \cup GR, \qquad (5)$$

which represents the trefoil (the intersecting fields of two or three circles).

<figure><img src="image_5.png" /><figcaption>Fig. 4. Venn's diagram showing the logic operations of the opponent-process.</figcaption></figure>

Then, we have

$$ \begin{aligned} \overline{BM} &= B(\overline{BG} \cup \overline{BR} \cup \overline{GR}) = B(\overline{B} \cup \overline{G})(\overline{B} \cup \overline{R})(\overline{G} \cup \overline{R}) \\ &= B(\overline{B} \cup \overline{BG} \cup \overline{BR} \cup \overline{GR}) = \overline{BGR}, \end{aligned} \qquad (6) $$

where DeMorgan Law is used. Similarly, there are $\overline{BM} = \overline{BGR}$ (yellow area), $\overline{GM} = \overline{BGR}$ (green area), $\overline{GM} = \overline{BGR}$ (magenta area), $\overline{RM} = \overline{BGR}$ (red area), $\overline{RM} = \overline{BGR}$ (red area), and $\overline{RM} = \overline{BGR}$ (cyan area).

Now let $B, G$, and $R$ denote three cones' outputs, which take values from [0, 1], and let the set operations be replaced by the fuzzy logic operations: $\vee, \wedge, \neg$ (can be omitted). First, we obtain the median value of $B, G$, and $R$ (see Fig. 5):

$$ \begin{aligned} M &= \text{med}(B, G, R) = [BG \vee BR \vee GR] \\ &= \max(\min(B, G), \min(B, R), \min(G, R)). \end{aligned} \qquad (12) $$
Then we obtain three opponent signals: blueness-yellowness ($M_{BY}$), greenness-magenta-ness ($M_{GM}$), and redness-cyanness ($M_{RC}$). The calculations are surprisingly simple:

$$M_{BY} = B - M = \begin{cases} +[\overline{BGR}], & B \ge M, \\ -[\overline{BGR}], & B < M; \end{cases} \qquad (13)$$

$$M_{GM} = G - M = \begin{cases} +[\overline{BGR}], & G \ge M, \\ -[\overline{BGR}], & G < M; \end{cases} \qquad (14)$$

$$M_{RC} = R - M = \begin{cases} +[\overline{BGR}], & R \ge M, \\ -[\overline{BGR}], & R < M. \end{cases} \qquad (15)$$

The opponent-process corresponding to different monochromatic lights is shown in Fig. 5, where the three response curves are assumed for convenience. We can also consider the left-upper part of Fig. 5 as a Venn's diagram.

<figure><img src="image_6.png" /><figcaption>Fig. 5. The opponent-process corresponding to different monochromatic lights.</figcaption></figure>
There are eight divided fields. The length of the part of a vertical line on a field is just the magnitude of the corresponding unique color signal. These fields can illustrate the changes in the color perception caused by different monochromatic lights well.

# 3 Further Investigations

## 3.1 To Explain Color Evolution

According to the decoding model, we can easily explain the evolution of color vision by splitting the sensitivity curves of visual cones (see Fig. 6). We may imagine that curves $R(\lambda)$ and $G(\lambda)$ gradually approach one another to become one curve named $Y(\lambda)$. Then we would see the fields representing red, green, cyan, and magenta disappear gradually. Further, let curves $B(\lambda)$ and $Y(\lambda)$ approach one another gradually to become one curve named $W(\lambda)$. Then we would see the fields representing blue and yellow disappear gradually, and only the black and white fields remain. Now, we can imagine that color vision was evolving in the opposite procedure. First, there was only one kind of visual cones in the human retina, and only two totally different colors (black and white) could be discerned. Then, with the evolution of color vision, the cones split into two kinds that had different spectral sensitivities so that blue and yellow were also perceived. After that, the cones split into three kinds so that more colors were perceived.

We may conclude that $n$ different kinds of cones can produce $2^n$ totally different color perceptions for $n=1, 2, 3$. When $n=4$, the conclusion seems also true. A symmetrical model of four primary colors for robots can be seen in [24]. The model has 14 "unique colors", which can be symmetrically put on the surface of a ball, besides "white" (1, 1, 1, 1) and "black" (0, 0, 0, 0). We can obtain a "color" ball that has many properties very similar to those in the Newton color wheel.

The evolution of color vision might have come in a somewhat different way. For example (see Fig. 6, deuteranopia-2), the curve $W(\lambda)$ first split into $R(\lambda)$ and $C(\lambda)$ related to cyan, instead of $B(\lambda)$ and $Y(\lambda)$, then $C(\lambda)$ split into $B(\lambda)$ and $G(\lambda)$.
<figure><img src="image_7.png" /><figcaption>**Fig. 6.** The Evolution of color vision illustrated by splitting sensitivity curves.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>**Fig. 7.** Different kinds of color blindness illustrated by incomplete separations of three sensitive curves.</figcaption></figure>

## 3.2 To Explain Color Blindness

Color blindness has been discussed by many researchers [25,26]. It can also be easily explained by the sensitivity curves of cones that are too close to each other (see Fig. 7). For example, monochromatism can be explained under the assumption that the sensitivity curves $B(\lambda)$, $G(\lambda)$ and $R(\lambda)$ have not yet separated from one curve; red-green blindness can be explained under the assumption that the curves $G(\lambda)$ and $R(\lambda)$ have not yet separated from one curve.

According to the decoding model, some red-green blindness can be identified as protanopia or deuteranopia only because the peak of $Y(\lambda)$ has shorter or longer wavelength. Tritanopia and tetartanopia can be illustrated by the assumption that the $B(\lambda)$ and $G(\lambda)$ (or $B(\lambda)$ and $R(\lambda)$) have not yet separated so that each kind of color blindness can only perceive two chromatic colors: red and cyan (or green and magenta).

## 3.3 Decoding and Reconstructing Colors in the Human Visual Cortex

Many neurons in the visual cortex are color selective, and color-opponent responses are evident throughout the visual cortex [14,15, ]. The human visual cortex can decode and reconstruct colors from the voxel responses of cones [16,17]. Many different
neurons in the visual cortex respond to many specific colors [18,21]. However, the relation between voxel responses and decoding outputs in the visual cortex is still unclear. Reference [16] uses a matrix to represent the decoding algorithm from voxel responses to decoding outputs; reference [21] uses Poisson distributions to describe the decoding algorithm from colors to decoding outputs. However, what we need is to explain the decoding algorithm from opponent-color signals as the outputs of ganglion cells to decoding outputs in visual cortex. Using the decoding model, we can build a two-layer neural network, with which we can easily explain the decoding algorithm.

The decoding model can be regarded as a single-layer neural network with three inputs and eight outputs. If the three outputs at every sector, such as signals of white, red, and yellow, are used as the inputs of another fuzzy 3-8 decoder (see Fig. 8), then we have a two-layer neural network. We call this neural network the Decoding Neural Network (DNN). Using a two-layer DNN, we can explain how six unique colors plus white are further decoded into more specific colors that are reconstructed in the human visual cortex, as shown in Fig. 8. For example, one output represents an orange or pink color in the visual cortex.

<figure><img src="image_9.png" /><figcaption>**Fig. 8.** A two-layer DNN can be used to explain the decoding algorithm from voxel responses in cones to decoding outputs in the visual cortex.</figcaption></figure>

We can also use the DNN for color recognition, not only for three primary colors, but also for four primary colors perceived by robots [24].

# 4 Discussions

## 4.1 About the Reasonability of the Decoding Model

Several reasons make the decoding model convincing:

* The model is concise, symmetrical, and without any coefficient.
* It can be used to explain the opponent-process, color evolution, color blindness more naturally than the popular zone models.

* It can be used to explain the decoding and reconstruction of colors in the visual cortex more conveniently.

* In the popular zone model, red plus green at the zone of visual cones become yellow; yet, red plus green at the zone of visual nerves become white. Therefore, the meanings of “red” and “green” in the popular zone model are easily misunderstood. However, the decoding model does not have this problem.

* The decoding model is more compatible with the laws of color mixture that are used for the color transform of computer graphics.

* We can also use the decoding model to explain the phenomenon of negative afterimage conveniently. For example, when the sensitivity of the R-cone falls down, $[BGR]$ for cyanness will be over zero when white (1,1,1) is perceived so that white looks cyan.

There is a possible problem with the decoding model. In the popular zone model, there are only two pairs, instead of three pairs, of opponent colors. It seems that psychological and physiological experiments support the affirmation that only two pairs of opponent colors exist. However, we can argue that the “red and green” in the popular zone model is, in fact, a pair of opponent colors between red-cyan and green-magenta. More than four unique colors were also affirmed by others [27].

According to the decoding model, we can make two predictions. One is that there should be some fuzzy logic gates, which execute the operations of maximum, minimum, and even median, in the human retina. Another is that there should be some chromatic opponent units in visual nerves, whose response curves have a horizontal line, instead of a neutral point, between positive and negative parts (see the right part of Fig. 5). These logic gates and opponent units have not been mentioned yet (which is another problem with the decoding model) either because most experiments were made with animals whose color vision is not complete as ours, or because the guidance from appropriate theory was absent. For example, a widely used method for identifying a chromatic opponent unit is to find its neutral point [28]; however, this method is not suitable for identifying the opponent unit suggested above. The author believes that the predicted logic gates and the opponent units will be discovered soon by physiologists who pay attention to them.

## 4.2 About the Potential Applications of the Decoding Neural Network

Fig. 8 indicates that we can use several or many 3-8 or $n-2^n$ decoders to construct a two-layer or multi-layer Decoding Neural Network (DNN). Compared with the popular neural network, the DNN has different characteristics:

* It uses fuzzy logic without parameters.

* Every nerve cell in the next layer has inputs that are selected from one sector of the previous layer.

* The number of non-zero outputs is equal to the number of inputs.
Using biologic feature vectors instead of color vectors, we may use a DNN to rec-
ognize some different living things. It is possible to combine the DNN with existing
neural networks to improve machine learning in some cases. We need further studies
on the DNN.

5 Conclusions

This paper has introduced the decoding model of color vision, including the fuzzy logic and the opponent-process for it. This model can unify Young and Helmholtz's tri-pigment theory and Hering's opponent theory more naturally than the popular zone models. And, this model is symmetrical and compatible with the popular color transform method, and hence, is also practical. The paper has used the decoding model to explain color evolution and color blindness pictorially. It has also used the DNN, which consists of two-layer fuzzy 3-8 decoders, to explain the decoding and reconstruction of colors in the visual cortex. These explanations indicate the reasonability of the decoding model. The DNN that consists of some fuzzy n-2ⁿ decoders has some desirable characteristics for machine learning and needs further studies.

References

1. Young, T.: Bakerian Lecture: On the Theory of Light and Colours. Phil. Trans. R. Soc. Lond. 92, 12–48(1802). doi: 10.1098/rstl.1802.0004

2. Solomon, R. L., Corbit, J. D.: An Opponent-Process Theory of Motivation: I. Temporal Dynamics of Affect. Psychological Review 81(2), 119–145 (1974).

3. De Monasterio, F. M., Gouras P. and Tollhurst D J.: Trichromatic color opponency in ganglion cells of therhesus monkey retina. J. Physiol. 252, 197-216(1975).

4. Judd, D. B., Response functions for types of vision according to the Muller theory, J. Res. Natl. Bur. Std. 42. (1949).

5. Hurvich, L. M., Jameson D.: An opponent-process theory of color vision, Psychological Review 64(6), 384-404(1957).

6. Walraven, P. L.: On the Bezold-Brucke phenomenon. J. Optical Society of America 51(10), 1113-1116 (1961).

7. Hunt, R. W. G.: A model of color vision for predicting color appearance. Color Res. Appl. 7(2), 95-112(1982).

8. De Valois R. L., De Valois K. K.: A multi-stage color model. Vision Research 33(8), 1053-1065(1993).

9. Lu, C.: New Theory of color vision and simulation of mechanism, Developments in Psychology (in China) 14(2), 36-45(1986).

10. Lu, C.: Decoding model of color vision and its verification, ACTA OPTICA SINICA, 9(2), 158-163(1989).

11. Lu, C.: A generalization of Shannon's information theory. Int. J. of General Systems 28 (6): 453-490(1999).

12. Lu, C.: Semantic information G theory and logical Bayesian inference for machine learning. Information 10(8), 261(2019).

13. CIELAB, Symmetric colour vision model, CIELAB and colour information technology, 2006. http://130.149.60.45/~farbmetrik/A/FI06E.PDF
14. Kleinschmidt, A., Lee, B. B., Requardt, M. et al.: Functional mapping of color processing by magnetic resonance imaging of responses to selective P- and M-pathway stimulation. *Exp Brain Res* 110, 279-288 (1996).

15. Engel, S., Zhang, X., Wandell, B.: Color tuning in human visual cortex measured with functional magnetic resonance imaging. Nature 388(6637),68-71(1997).

16. Brouwer, G. J., Heeger, D. J.: Decoding and reconstructing color from responses in human visual cortex. J. Neurosci 29(44):13992-14003 (2009)

17. Brouwer, G. J., Heeger, D. J.: Categorical clustering of the neural representation of color, J. Neurosci, 33(39), 15454-15465(2013).

18. Conway, B. R., Moeller, S., Tsao, D. Y.: Specialized color modules in macaque extrastriate cortex. *Neuron*, **56**(3), 560-573(2007).

19. Bohon K.S., Hermann, K. L., et al.: Representation of perceptual color space in macaque posterior inferior temporal cortex (the V4 complex). eNeuro 2016, 3(4), 1-28(2016).

20. Zaidi, Q., Marshall, J., Thoen, H., Conway, B. R.: Evolution of neural computations: man-tis shrimp and human color decoding, i-Perception, 5 (6), 492-496(2014).

21. Zaidi, Q., Conway, B.: Steps towards neural decoding of colors, Current Opinion in Behavioral Sciences 30, 169-177(2019).

22. Lu, C.: B-Fuzzy set algebra and generalized mutual information formula, Fuzzy Systems and Mathematics, 5(1), 76-80(1991).

23. Smith, A. R.: Color Gamut Transform Pairs, Computer Graphics 12(3), 12-19(1978).

24. Lu, C.: Models of color vision for robots. Robot( A Journal of Chinese Society of Automation) 1(6),39-46 (1987).

25. Marmor, M. F.: Vision, eye disease, and art: 2015 Keeler Lecture. Eye. 30 (2), 287-303(2016).

26. Chan, X., Goh, S., Tan, N.: Subjects with colour vision deficiency in the community: what do primary care physicians need to know?. Asia Pacific Family Medicine 13 (1): 10 (2014). doi:10.1186/s12930-014-0010-3.

27. Hardin, C. L.: The resemblances of colors, Philosophical Studies. 48(7), 35-47 (1985).

28. De Valois, R. L., Abramov, I., Jacobs, G. H.: Analysis of response patterns of LGN cells, Journal of the Optical Society of America 56(7), 966-977 (1966).