BACKGROUND
how a three-dimensional scene can be “faithfully” expressed in a  plane , that is to say, how it can be “faithfully” represented using a planar cartesian coordinate system, and what the differences are between the stereoscopic perception of an actual scene and its two-dimensional image are important issues in visual information processing research, neural computation, psychophysics, and neuroscience.
at the cellular level, previous studies have shown that in the v <dig> cortex, only complex cells are able to respond to absolute parallax  <cit> .
in the v <dig> cortex, there are some cortical neurons that respond to relative parallax  <cit>  and parallax-sensitive neurons can be described by specific and generalized energy models  <cit> .
studies have been carried out both in the ventral and dorsal streams of the visual cortex, mainly to detect neurons that can respond to depth perception through specific signal stimuli.
the binocular visual system is able to perceive depth information using binocular disparity, and one of the founders of the computational theory of vision, marr, proposed a classic reconstruction algorithm for three-dimensional images  <cit> .
julesz’s experiments on random dot stereograms  led to a psychophysical study on the binocular disparity that forms stereoscopic vision.
its purpose was to show how the human brain deals with depth information  <cit> .
in other words, the task was to explore how human vision extracted stereoscopic information from a visual scene contained in a cartesian coordinate system and depicted on a two-dimensional imaging plane.
a three-dimensional scene “faithfully” represented in a plane seems to be commonplace phenomenon, yet the mechanism for this has never been explored.
it is, however, a basic theoretical problem and is worthy of study in depth, not only because it concerns the geometric and physical properties of planes and space and is closely related to the three-dimensional perception of human vision, but also because it is closely related to the problem of stereoscopic perception in computer vision, robotics navigation, and visual cognitive psychology.
in fact, there are many similar phenomena, such as optical illusions generated using optics, geometry, physiology, psychology, and other means.
optical illusions are largely due to the uncertainty caused by the bimodal graphics in a two-dimensional plane and uncertainty during visual information processing in the brain.
the illusions, such as bimodal images for instance  and additional file  <dig> disappear when the images are placed in a real three-dimensional space.
additional files  <dig>   <dig>   <dig>  and  <dig> show the lifelike effect of three-dimensional perception, can be more intuitively reflect the meaning f this article.
marr pointed out that the essence of visual information processing is to discover what and where objects are in space  <cit> .
f. crick also stated that visual information processing is a construction process  <cit> .
in their book seeing, frisby and stone defined how “seeing” is a particularly difficult task.
they analyzed research from computational vision, psychophysics, neurobiology, neuroanatomy, brain imaging, modeling methods, image statistics, multiple representations, active vision, bayesian theory, and the philosophy of visual information processing.
the understanding of “seeing” among these fields is not the same, each focuses on different aspects of “seeing”, and each has their own understanding of the “the essence of seeing” .
as is known, any point in space can be represented by a cartesian coordinate system  at a certain moment, t, and an object at this point can be expressed using light intensity v
x
, v
y
, v
z
and color-related wavelength λ. in this way, one can define a function p
w
, pw = pw, that completely represents an object, and is also a good description of the objective external world.
when human vision processes an object, the optical axis of the eyeball is consistent with the z axis , such that the visual imaging plane is perpendicular to the optical axis.
this reduces one variable from the function, p
w
, and leaves only seven variables that form the plenoptic function proposed by adelson and bergen in the study of human primary visual information processing  <cit> .
the intensity of each ray can be described as a function of the spatial viewing angle; that is, the wavelength, time, and light intensity of the observation position  in spherical coordinates and pv = pv in cartesian coordinates) captures all that the human eye or optical device may “see”, including ambient light.
therefore, the plenoptic function and full holographic representation of the visible world are equivalent.
as for the different definitions of the plenoptic function and its mathematical expression, we will discuss this in some detail in the discussion  <cit> .
we should note that the plenoptic function not only reveals how humans “see” the external world, but also intuitively and concisely describes the information processing that occurs between the retina and the primary visual cortex.
marr pointed out that the true nature of information processing in “seeing” is to discover where and what is in space.
“where” in space can be located by a cartesian rectangular coordinate system .
“what” is in this position may be perceived through the emitted or reflected structure of the light ray from the “object” to the viewer’s eyes, these correspond to the intensity v
x
, v
y
, v
z
and wavelength λ of light at that location that carry information about the contour, shape, and color of the object.
thus, it can be seen that the plenoptic function is a good description of the external world.
when adelson and bergen proposed the plenoptic function, their intentions were to solve the problem related to the corresponding points in computer vision.
it was not expected that the study would promote the birth and development of the new discipline of computational photography  <cit> .
to adapt to the needs of different disciplines, there are two basic formulae for the plenoptic function, one describes an object p
w
= p and the other describes the viewer’s perception of the object.
in such a case, the optical axis  of human vision and the coordinate axis z are consistent, thereby eliminating the need for coordinate axis z, namely: pv = p. “seeing” is the association between the observer and the object, where the coordinates of an observer’s position are x, y, and z, and the light intensities that an object emits or reflects to the observer’s eye are v
ox
, v
oy
, v
oz
, representing the light intensity information of the object itself.
the intensity of light is related to the number of excited photo-sensitive cells in the retina and their activity levels.
as long as the angles of the incident light θ and φ are recorded, the plenoptic function can be simplified as pv = p such that a dual-plane  and  parameterization becomes possible.
this parameterization is used in this paper and is important for processing the visual information of an image to reveal its deep meaning.
an interesting and important question concerns the difference between the functions pw and pv.
it is generally considered that pw differs from pv in the number of dimensions; i.e., the coordinates are reduced from  to .
however, in practice, when the visual system perceives an external scene, the optical axis  is consistent with the z axis.
the imaging plane is perpendicular to the optical axis  and this is an inherent characteristic of the optical imaging system of vision.
at a certain distance in front of and behind the focal plane of a visual image , the visual system is able to form a clear visual image.
the diameter δ of this region  is very small  and gives us the depth of focus   <cit> .
according to the conjugate relationship between the image and object points of the viewed object, there is a similar situation.
when a light spot is formed at a certain small distance before and after the object, the depth of field is formed jointly by the near and far points.
the human visual system perceives the depth of field in the surrounding world through its optical system.
the Δl depth of field of imaging on the retina is jointly determined by δ, f, f, l, Δl <dig>  and Δl <dig>  where δ is the diameter of the permissible circle, f is the focal length of the lens, f is the size of the pupil, l is the focusing distance, Δl <dig> is the front depth of field, and Δl <dig> is the back depth of field.
then, the formula for the depth of field Δl can be expressed as follows:
Δl=Δl1+Δl2=2f2fδl2f4-f2δ2l <dig>
and the focal length f can be calculated by
1lo+1li=1forf=lolilo+li
where lo object is distance and li is image distance, as shown in figure  <dig>  therefore, the visual image in the retina contains information about the depth of field that is not lost when the three-dimensional objective world is represented in the two-dimensional retina of the visual system.
this is mainly because the optical axis is coincident with the coordinate axis z; that is to say, l and z are equivalent in formula  and thus they can replace each other.
therefore, formula  can be rewritten as follows
Δlz=Δl1+Δl2=2f2fδz2f4-f2δ2z <dig>
in formula , z is the distance of the z axis, reflecting depth information.
it is thus clear that the coincidence of the optical axis with the coordinate axis z is a very effective constraint.
it is not imposed artificially, but is determined by the optics of the visual system.
as one gazes into the distance, the depth of field may extend to infinity.
one familiar phenomenon occurs when we look at a distant railway or highway and the tracks or road edges gradually converge to a single point in the distance , as shown in figure  <dig>  the image in the imaging plane is just a visual image on the retina.
this is a result of an affine transformation of the visual optical system, but is also the objective reality or physical truth of human vision when observing the world and is a basic characteristic of visual image processing.
the fact that the visual system perceives the railroad tracks converging at one point in the distance demonstrates that there is not a corresponding point in the cartesian coordinate system.
however, it is easy to solve this problem by supplementing a new coordinate point  with a homogeneous coordinate, thereby establishing the mapping relationship between the cartesian coordinate system r
n
and an affine coordinate system p
n
as
rn→pn:x <dig> x <dig> ⋯,xnt→tx <dig> x <dig> ⋯,xn,0t→t,a→0x <dig> x <dig> ⋯,xn,at→t,a→ <dig>
where the infinity point t is just the limit of t under a →  <dig>  <cit> .
therefore, the infinity point represents the vanishing point of the vision range.
it is critical for depth perception of the surrounding scenery that the perspective direction and number of vanishing points are considered when displaying a three-dimensional scene in a two-dimensional plane  <cit> .
when human eyes look into the distance, the fixation point can change in position, and this forms a horizontal vanishing line, as shown in figure  <dig>  this line is known as the infinity line and is composed of countless vanishing points  <cit> .
similarly, it is also an objective phenomenon that occurs in the visual perception of the external world.
it occurs at the intersection of the sky and ground, and provides a broader perspective.
RESULTS
mapping between the scene and visual image
the above brief description of previous research aims to introduce the problem of how a three-dimensional cartesian coordinate system converted into a two-dimensional plane is able to express a real three-dimensional scene.
this also explains why visual images in the retina can provide three-dimensional scene information to an observer.
however, how the cartesian coordinate system in a two-dimensional plane can “faithfully” represent a three-dimensional scene is not known, even though the problem seems trivial.
the difference between the stereoscopic perception of actual scenes and a scene in a two-dimensional plane is an important issue in visual information processing, neural computation, psychophysics, and neuroscience, and is also a main research topic in image processing, three-dimensional display methods, and computer vision.
figure  <dig> shows a cartesian coordinate system.
we cannot draw the z axis in the x–y plane such that the included angle between all axes is 90° and the z axis is truly perpendicular to the x–y plane.
one approach for drawing the z-axis is to introduce an angle α between the external incident light and the optical axis of vision, where α ≠  <dig>  for visual purposes, we can only receive incident light from the front; that is, 0 < α is not possible.
however, when creating a three-dimensional cartesian coordinate system to express a stereoscopic image, all quadrants of space should be discussed.
hence, a more general assumption is |α| ≠  <dig>  for the sake of simplicity, we discuss the second quadrant .
the cases for the other quadrants are the same.
when the angle is within the range 90° < θ < 180°, we can express three-dimensional stereoscopic structure in a plane, as shown in figure  <dig>  according to the statistical results obtained in a psychological experiment that we conducted, when α ≈ 30° , the visual stereoscopic perception is strong and the image structure is stable.
it can now be envisaged that to have a true z′ axis perpendicular to the x–y plane , the projection of the z axis onto the z′ axis is z cos = z cos α; that is, the projection of the z axis onto the z′ axis is simply z
p
, which is equivalent to the value along the z axis in real three-dimensional space.
for example, if α = 30°, then
zp=zcosα=zcos30∘=3/2z= <dig> z
the actual loss of depth information along the z-axis, or the information loss of visual depth perception, is zloss = z - z
p
= z -  <dig> z = z =  <dig> z. naturally, α can have different values and indicate different depths.
this is consistent with our experience of visual perception, although we usually pay no attention to it.
as already pointed out, there is a conjugate relation  between the object point and its image point.
when an observer sees a three-dimensional scene iwr = pw in the external world, a corresponding visual image i¯rw = pvx,y,z′;vxvyvz′;λ,t forms on the retina that is more than two-dimensional but less than three- dimensional.
in turn, if there is a visual image i¯rw on the retina, then the observer perceives the scene of the external world iwr according to i¯rw.
hence, the scene and image have the mutually conjugate mapping
iwr⇄pvx,y,z′;vx,vy,vz′;λ,tpwx,y,z;vx,vy,vz;λ,ti¯rw
iwr⇄z′zi¯rw
z′↔zcosθ-90∘=zcosα
that is, the actual scene pw is transformed by z cos α and forms the visual image pvx,y,z′;vxvyvz′;λ,t on the retina.
it is important to note that, with this transformation relationship, human vision using a two-dimensional image on the retina can perceive an actual three-dimensional scene.
according to the above discussion, it is clear that the scene pvx,y,z′;vxvyvz′;λ,t can be drawn on a two-dimensional plane, and it can be expresses as a stereoscopic image on the retina and provide stereoscopic perception.
the basic concept of this information processing is more clearly expressed as
pwx,y,z;vxvyvz;λ,t↔pv=pvx,y,zcosα;vxvyvzcosα;λ,t=pvx,y,zp;vxvyvzp;λ,t
that is, through the plenoptic function pw, iwr forms a visual image i¯rw on the retina.
conversely, the visual image i¯rw matches the external world through the plenoptic function pv, and the loss of image information between iwr and i¯rw is approximately z cos α.
of course, this is largely a proof of principle, but this discussion demonstrates that it can be used for studies in visual information processing.
it has been confirmed in many eye tracker tests, including psychophysical experiments that the visual system can adjust with eye movements to find a suitable viewing angle and orientation so that the loss of information is minimal  <cit> .
this is a fundamental property of the visual system and means that forming visual images on the retina and in the v <dig> cortex does not require inversion and reconstruction, possibly because the computational cost is too high to solve its inverse, an ill-posed problem without a unique solution.
loss of information due to the introduction of a three-dimensional cartesian coordinate system in the plane
figure  <dig> shows three groups of three-dimensional cartesian coordinate systems introduced in a two-dimensional plane.
the main differences are the different angles between the x- and z-coordinates; i.e., orientations of the z-axis relative to the x-y plane are different and thus, the stereoscopic visual perception is also different.
in the example of figure 4a , when angle θ is 120°, the perpendicular relationship among the three axes x, y, and z is most obvious.
in figure 4a , when angle θ is 90°, there is still a perpendicular relationship among all three axes x, y, and z in the actual space.
however, it is not possible to draw a real vertical line perpendicular to the x-y plane itself.
it is instead projected points of this vertical line at the coordinate origin by a perpendicular projection into the x-y plane.
on the contrary, it can be seen in figure 4a , when angle θ is 120°, that the z-axis and the x-y plane and the z-axis and x-coordinate axis have an included angle of 120°.
therefore, it can be considered that, in actual space, its projection in the z axis is cosα=cosθ‒90∘=cos30∘=3/2= <dig> .
in other words, the information loss of depth perception is approximately  <dig> .
in contrast, the cartesian coordinates can be introduced in the plane and can provide visual stereoscopic perception.
for figures  <dig> and  <dig>  the situation is similar.
role of the vanishing point in stereoscopic visual perception
figure  <dig> further illustrates the important role of the vanishing point when introducing a three-dimensional cartesian coordinate system into a two-dimensional plane to represent a stereoscopic scene.
figure  <dig> is the convergence of projection in a single direction with only a single vanishing point.
figures  <dig> and  <dig> show the convergence of two and three projective directions with two and three vanishing points, respectively.
please note that the blue lines are the x, y, and z coordinates of the cartesian three-dimensional rectangular coordinate system that can be found in figures  <dig>   <dig>  and  <dig>
the existence of the vanishing point is the fundamental reason why a cartesian three-dimensional rectangular coordinate system can be drawn in a two-dimensional plane.
as mentioned above, it can be easily seen that the formation of vanishing points underlies the optical system of human vision .
it is also the basis of an affine transformation by which the human visual system is able to perceive the three-dimensional external world, as illustrated in the case of railroad tracks that converge to a single point, forming of a vanishing point .
dual-plane parameterization of the plenoptic function for neural computation of early vision
we know that each pixel of a two-dimensional digital image is a record of the intensity of all light that reaches this point, but does not distinguish between the directions of the light rays.
it is just a projection of the light field of the three-dimensional structure, with lost information about phase and direction.
unlike this, the light field refers to the collection of light from any point in space in an arbitrary direction.
it comprises all light from different angles that makes a contribution to each pixel.
if it takes into account how the angle of light changes with time , it is a dynamic light field.
the plenoptic function is a good mathematical description of the dynamic light field.
however, questions remain regarding how the human visual system perceives and processes the structural information of the dynamic light field as well as how it receives three-dimensional information from the image on the retina.
studies by zeki, livingstone et al.
have indicated that in the human visual system color information is transmitted in a separate channel in the cerebral cortex  <cit> .
therefore, wavelength λ can be separated from the plenoptic function.
in addition, position, direction, and orientation information can also be separated.
in this way, without considering time variation and separating dimensions, the seven-dimensional plenoptic function pv = pv can describe and reconstruct plenoptic images, or visual information of the objective world with different combinations of variables.
when the viewer’s eyes are looking at a point in any scene, emitted or reflected light rays from this point will enter the eye.
the intensity information of the incident light ray carried in vx, vy, vz is received by the eye.
since the optical and the coordinate z axes are the same, the light intensity of the stimulus is converted into the strength of photosensitive cell activity.
therefore, only angles θ andφ of the light need to be recorded.
for this reason, the plenoptic function can be parameterized using a dual-plane representation formed from p and p, as shown in figure  <dig>  a light ray intersects with the position plane p and angle plane p at  and , respectively.
the coordinates of the points of intersection  and  can be used to describe this plenoptic function.
the form of a two-plane parameterization is very simple and intuitive.
hence researchers have used this method for visualization of light field data, namely, using double nested coordinates to arrange the data of a four-dimensional light field into a two-dimensional plane, forming two symmetrical representations.
figure  <dig> is an example in which p is the inner angle plane and p is the outer position plane.
as can be seen, a light ray at different angles corresponds to different viewing angles in the imaging plane.
therefore, such representation of the light field may be closely related to a neural representation of the retina and primary visual cortex of the human visual system.
many experiments in neurobiology have shown there are topological mappings with a one-to-one correspondence between the retina and the v <dig> cortex that is established through projections from the ganglion cells via the lateral geniculate nucleus to the primary visual cortex.
through photosensitive cells, the retina records the position information of the incident light ray, while the v <dig> cortex processes the orientation information through simple and complex cells, as well as the orientation function columns  <cit> .
therefore, for early visual information processing, this is a viable solution.
it minimizes the loss of information as much as possible without making the algorithm too complex.
of course, to do this is not an easy matter, and whether the human visual system employs this strategy needs further study.
three-dimensional visual perceptions of images in a two-dimensional plane
we know that if an image of a scene on a plane does not contain depth information, the human visual system has no way of perceiving the scene three-dimensionally.
when observing the external world, human vision has characteristics of perceptual constancy .
this constancy is the basis of an affine transformation, which depends on vanishing points and vanishing lines in visual perception and is determined by the characteristics of the optical system of the visual pathway.
as rock pointed out, the height of an object in the base plane is an important depth cue.
it can be calculated according to  <cit>
s=δad
where s is the height of the object on the fundamental plane , δ is the viewing angle of the camera, d is the distance  between the photographer and the object, or the depth information, and a is the scaling factor of the retina.
formula  is used to reconstruct a three-dimensional scene from an image in a two-dimensional plane.
figure  <dig> is an optical model of the affine transformation of the retina.
for the sake of simplicity, we analyze only the example  of one vanishing point, as shown in figures  <dig> and  <dig>
the main purpose of the calculation example is to show that we can use the vanishing point, size constancy and affine transformation model in figure  <dig> to calculate the depth value in a picture taken of an actual scene.
a comparison of the calculation results with actual measurements reveals that the vanishing point reflecting the basic characteristics of the optical system of human vision and size constancy reflecting cognitive psychological characteristics are important in accessing depth information in a two-dimensional picture.
the example focuses on the absolute depth perception of white markers, edges on the ground and nine trees .
comparisons with measurements are listed in figure  <dig>
specific calculations are carried out employing two methods.
the first method employs psychological methods based on formulae  and , and the second method employs an affine transformation based on an optical model of vision .
known parameters required for the calculation are the height of the camera from the ground  and the horizontal distance between the photographer and first white line on the ground  .
the camera is a nikon-e3700ccd, and the image size is 2048 × 1536 pixels.
the calculation includes the vanishing point, the vanishing line, the height of the tree, and the line whose change in depth value is fastest on the ground portion of the image plane.
specific calculations are found in the literature  <cit> .
naturally, algorithms of computer vision can also be used  <cit> .
the results of both calculation methods are consistent with actual measurement results, showing that the calculation methods are reasonable and reflect the consistency between visual psychology and the optical system of visual pathways in the depth perception of an actual scene.
more importantly, the results show that a two-dimensional image can contain rich three-dimensional information that is perceived by the visual system itself.
we know that when looking at an image or a scene from different angles, the perceived depth of field changes.
to show depth information provided by constancy and the affine transformation in a two-dimensional image plane , formula  is corrected according to equation , such that the image height of the object may be calculated according to
s=δcosαad=δcosθ-90∘ad,
where α =  is the included angle between the z-axis and z′ -axis  when looking at the image.
hence, we use formula  to correct the result of the depth information given by method  <dig>  and these corrected values are also given in figure  <dig>  after taking into consideration information loss, the corrected value roughly reflects the visual depth perception obtained from the image .
the proposed method is completely different from three-dimensional image reconstruction that uses binocular disparity and corresponding points in the field of visual computational theory, or three-dimensional reconstruction using corresponding points in two images taken by two cameras in the field of computer vision.
the processing method of visual perception has advantages  <cit>  such as efficiency, robustness, and low computational complexity.
it is therefore worthy of study by researchers in the fields of computer vision and visual neural computational theory.
in appendix  <dig>  according to figure  <dig>  figure  <dig>  the formulae ,  and  we will make some predictions about stereoscopic perception of the image on a two-dimensional plane, including:  <dig> the picture, in which there is no vanishing point;  <dig> alternating process of cartesian coordinate system and affine coordinate system;  <dig> the moon illusion, and  <dig> the inversion reconstruction of visual image.
discussion
this article explores how the human vision system extracts depth information from an image of a scene in a cartesian rectangular coordinate system on a two-dimensional plane.
we introduced the concepts of a plenoptic function in the optical system of the visual pathway.
in the section of methods “computational approach in visual cortex v1”, we proposed an algorithm of coincidence test, in which an image primitive r
u,v
transferred by ganglion cells from retina to visual cortex v <dig> will coincide with neurons’ receptive field [b
θ,φ
]
Θ × Φ
in cortical columns.
note that, all of neurons in the columns simultaneously carry out compliance testing operations in parallel manner, neuron of [b
θ,φ
]
Θ × Φ
, which most consistent with the image primitives r
u,v
, is activated and its firing rate is strongest, so that each image primitive r
u,v
can be detected.
because it is distributed and parallel processing , the mathematic operation of coincidence test is very simple, robust, fast and completely consistent with the pattern of stimulating → firing → response of neurons.
based on the biological function and structure of the visual pathway and the primary visual cortex, we proposed the dual-parameterized method, which can be expressed as p ⊗ p, and is mathematically equivalent to the formula pv = [r
u,v
]
u × v
⊗ [b
θ,φ
]
Θ × Φ
, or to formula  <dig>  as described as follows.
in this paper, we have raised an issue “in the two-dimensional plane, why can three-dimensional structure of a picture be expressed by adopting cartesian coordinate system?”, its importance is to study the information processing from 2d retinal image to three-dimensional visual perception.
based on neural computation of visual cortex v <dig>  and taking into account the affine transformation processing of visual image information and size constancy of visual perception, and also considered the findings of psychophysics.
however, formula  and figure  <dig> show that the psychology of visual perception can explain how the human vision perceives a three-dimensional scene from a two-dimensional retina.
because of a structured light field that densely fills the surroundings, human vision processes information according to formulae  and .
the information loss from the three-dimensional scene in the external world to a visual image in the two-dimensional retina is small, and hence the visual image on the retina contains the rich information of the three-dimensional scene.
therefore, we may consider the visual system as a causal system, meaning that the scene has a one-to-one correspondence with the visual image.
the scene produces a visual image in the retina, and conversely, if a visual image is formed in the retina, then a viewer perceives the external scene that produced that visual image in the retina.
we know the reconstruction of visual image is just a hard inverse problem as a major topic of research in computer vision, its concern is how to use binocular disparity information  to find a stable and efficient reconstruction algorithms; it is also an issues concerned by current 3d display technical, its focal point is that this kind of research will able to provide an effective method for better 3d display technology; of course, it is also hard problem to trouble the research of biological vision, vision research mainly is to start from unified basic viewpoint of the biological function and structure of the vision and then explore how to achieve the following information processing by human visual system, namely : from retinal images of three-dimensional scenes to → 2d visual image, and to → 3d visual perception.
in the first section “mapping between the scene and visual image ” of this paper, this issue has been discussed in more detail, in which the formulas  and  had shown that there is no specific reconstruction algorithm from 2d retinal images to three-dimensional scene.
at present, to an image, the processing time of the brain has been determined by using an approach of rapid serial visual presentation of image series and cognitive psychological method, it is just 13 ms  <cit> .
so fast processing speed shows that human vision may not be obtained three-dimensional depth perception by using reconstruction method based on the corresponding point, because this method and related algorithms are too complicated, the computational cost is also too high, for this reason, it is impossible to implement such a reconstruction algorithms by using the neurons, neural circuits and partial network.
this paper studies how to obtain stereoscopic visual perception, when viewing pictures on the plane, obviously, this issue has important significance for vision information processing; of course, it is also the same for computer vision.
according to figures  <dig> and  <dig>  the formulae ,  and  we may make some predictions about stereoscopic perception of the image on a two-dimensional plane, including:
<dig>  the picture, in which there is no vanishing point;
<dig>  alternating process of cartesian coordinate system and affine coordinate system;  <dig>  the moon illusion .
we have reason to believe that rough outline of theory about three-dimensional visual perception of visual pathway is generally clear.
CONCLUSIONS
we know that there are many monocular depth cues  that can also form depth perception.
however, in this paper, we study how to express stereoscopic visual perception in a two-dimensional plane and only use the parameterized method of a dual plane of the plenoptic function to process the visual information of an image.
according to the principle of graceful degradation proposed by marr  <cit> , if the visual system calculates a rough two-dimensional description from an image, it will be able to calculate a rough three-dimensional description represented by this image.
in other words, human vision can perceive the real three-dimensional description from stereoscopic images on a two-dimensional plane.
marr posed the problem in this way: “the contours of the image are two-dimensional, but we often come to understand these contours from the perspective of three dimensions.
therefore, the key question is how do we make a three-dimensional interpretation of the two-dimensional contour?
why can we make this explanation?”
we have studied this issue, and to answer marr’s question, this paper presents a preliminary explanation.
the main results are as follows:
<dig>  two different plenoptic functions to describe the objective world were introduced.
the difference between these two functions pw and pv regarding the external scene obtained by visual perception were analyzed, and their specific applications in visual perception were discussed.
<dig>  the main results were how the processing of visual depth information perceived in stereoscopic scenes can be displayed in a two-dimensional plane.
constraints for the coordinates and an algorithm implementation were also provided, in particular, a method used to separate the plenoptic function and a transformation from the retina to the visual cortex.
a dual-plane parameterized method and its features in neural computing from the visual pathway to visual cortex v <dig> were discussed.
numerical experiments showed that the advantages of this method are efficiency, simplicity, and robustness.
<dig>  size constancy, a vanishing point, and vanishing line form the psychophysiological basis for visual perception of the external world, as well as the introduction of the three-dimensional cartesian rectangular coordinate system into a two-dimensional plane.
this study revealed the corresponding relationship between perceptual constancy, the optical system of vision, and the mapping of the vanishing point and line in the visual image on the retina.
the main results of this paper are a preliminary explanation as to why and how the cartesian rectangular coordinate system can be introduced into a two- dimensional plane, and how a three-dimensional scene can be perceived in a two-dimensional plane.
the results of this study are of significance in visual depth perception and possibly in applications of computational vision.

 how a three-dimensional scene can be “faithfully” expressed in a plane, that is to say, how it can be “faithfully” represented using a planar cartesian coordinate system, and what the differences are between the stereoscopic perception of an actual scene and its two-dimensional image are important issues in visual information processing research, neural computation, psychophysics, and neuroscience. at the cellular level, previous studies have shown that in the v cortex, only complex cells are able to respond to absolute parallax. in the v cortex, there are some cortical neurons that respond to relative parallax and parallax-sensitive neurons can be described by specific and generalized energy models. studies have been carried out both in the ventral and dorsal streams of the visual cortex, mainly to detect neurons that can respond to depth perception through specific signal stimuli. the binocular visual system is able to perceive depth information using binocular disparity, and one of the founders of the computational theory of vision, marr, proposed a classic reconstruction algorithm for three-dimensional images. julesz’s experiments on random dot stereograms led to a psychophysical study on the binocular disparity that forms stereoscopic vision. its purpose was to show how the human brain deals with depth information. in other words, the task was to explore how human vision extracted stereoscopic information from a visual scene contained in a cartesian coordinate system and depicted on a two-dimensional imaging plane. a three-dimensional scene “faithfully” represented in a plane seems to be commonplace phenomenon, yet the mechanism for this has never been explored. it is, however, a basic theoretical problem and is worthy of study in depth, not only because it concerns the geometric and physical properties of planes and space and is closely related to the three-dimensional perception of human vision, but also because it is closely related to the problem of stereoscopic perception in computer vision, robotics navigation, and visual cognitive psychology. in fact, there are many similar phenomena, such as optical illusions generated using optics, geometry, physiology, psychology, and other means. optical illusions are largely due to the uncertainty caused by the bimodal graphics in a two-dimensional plane and uncertainty during visual information processing in the brain. the illusions, such as bimodal images for instance and additional file disappear when the images are placed in a real three-dimensional space. additional files and show the lifelike effect of three-dimensional perception, can be more intuitively reflect the meaning f this article. marr pointed out that the essence of visual information processing is to discover what and where objects are in space. f. crick also stated that visual information processing is a construction process. in their book seeing, frisby and stone defined how “seeing” is a particularly difficult task. they analyzed research from computational vision, psychophysics, neurobiology, neuroanatomy, brain imaging, modeling methods, image statistics, multiple representations, active vision, bayesian theory, and the philosophy of visual information processing. the understanding of “seeing” among these fields is not the same, each focuses on different aspects of “seeing”, and each has their own understanding of the “the essence of seeing”. as is known, any point in space can be represented by a cartesian coordinate system at a certain moment, t, and an object at this point can be expressed using light intensity v x, v y, v z and color-related wavelength λ. in this way, one can define a function p w, pw = pw, that completely represents an object, and is also a good description of the objective external world. when human vision processes an object, the optical axis of the eyeball is consistent with the z axis, such that the visual imaging plane is perpendicular to the optical axis. this reduces one variable from the function, p w, and leaves only seven variables that form the plenoptic function proposed by adelson and bergen in the study of human primary visual information processing. the intensity of each ray can be described as a function of the spatial viewing angle; that is, the wavelength, time, and light intensity of the observation position in spherical coordinates and pv = pv in cartesian coordinates) captures all that the human eye or optical device may “see”, including ambient light. therefore, the plenoptic function and full holographic representation of the visible world are equivalent. as for the different definitions of the plenoptic function and its mathematical expression, we will discuss this in some detail in the discussion. we should note that the plenoptic function not only reveals how humans “see” the external world, but also intuitively and concisely describes the information processing that occurs between the retina and the primary visual cortex. marr pointed out that the true nature of information processing in “seeing” is to discover where and what is in space. “where” in space can be located by a cartesian rectangular coordinate system. “what” is in this position may be perceived through the emitted or reflected structure of the light ray from the “object” to the viewer’s eyes, these correspond to the intensity v x, v y, v z and wavelength λ of light at that location that carry information about the contour, shape, and color of the object. thus, it can be seen that the plenoptic function is a good description of the external world. when adelson and bergen proposed the plenoptic function, their intentions were to solve the problem related to the corresponding points in computer vision. it was not expected that the study would promote the birth and development of the new discipline of computational photography. to adapt to the needs of different disciplines, there are two basic formulae for the plenoptic function, one describes an object p w  = p and the other describes the viewer’s perception of the object. in such a case, the optical axis of human vision and the coordinate axis z are consistent, thereby eliminating the need for coordinate axis z, namely: pv = p. “seeing” is the association between the observer and the object, where the coordinates of an observer’s position are x, y, and z, and the light intensities that an object emits or reflects to the observer’s eye are v ox, v oy, v oz, representing the light intensity information of the object itself. the intensity of light is related to the number of excited photo-sensitive cells in the retina and their activity levels. as long as the angles of the incident light θ and φ are recorded, the plenoptic function can be simplified as pv = p such that a dual-plane and parameterization becomes possible. this parameterization is used in this paper and is important for processing the visual information of an image to reveal its deep meaning. an interesting and important question concerns the difference between the functions pw and pv. it is generally considered that pw differs from pv in the number of dimensions; i.e., the coordinates are reduced from to. however, in practice, when the visual system perceives an external scene, the optical axis is consistent with the z axis. the imaging plane is perpendicular to the optical axis and this is an inherent characteristic of the optical imaging system of vision. at a certain distance in front of and behind the focal plane of a visual image, the visual system is able to form a clear visual image. the diameter δ of this region is very small and gives us the depth of focus. according to the conjugate relationship between the image and object points of the viewed object, there is a similar situation. when a light spot is formed at a certain small distance before and after the object, the depth of field is formed jointly by the near and far points. the human visual system perceives the depth of field in the surrounding world through its optical system. the Δl depth of field of imaging on the retina is jointly determined by δ, f, f, l, Δl and Δl where δ is the diameter of the permissible circle, f is the focal length of the lens, f is the size of the pupil, l is the focusing distance, Δl is the front depth of field, and Δl is the back depth of field. then, the formula for the depth of field Δl can be expressed as follows: Δl=Δl1+Δl2=2f2fδl2f4-f2δ2l and the focal length f can be calculated by 1lo+1li=1forf=lolilo+li where lo object is distance and li is image distance, as shown in figure  therefore, the visual image in the retina contains information about the depth of field that is not lost when the three-dimensional objective world is represented in the two-dimensional retina of the visual system. this is mainly because the optical axis is coincident with the coordinate axis z; that is to say, l and z are equivalent in formula and thus they can replace each other. therefore, formula can be rewritten as follows Δlz=Δl1+Δl2=2f2fδz2f4-f2δ2z in formula, z is the distance of the z axis, reflecting depth information. it is thus clear that the coincidence of the optical axis with the coordinate axis z is a very effective constraint. it is not imposed artificially, but is determined by the optics of the visual system. as one gazes into the distance, the depth of field may extend to infinity. one familiar phenomenon occurs when we look at a distant railway or highway and the tracks or road edges gradually converge to a single point in the distance, as shown in figure  the image in the imaging plane is just a visual image on the retina. this is a result of an affine transformation of the visual optical system, but is also the objective reality or physical truth of human vision when observing the world and is a basic characteristic of visual image processing. the fact that the visual system perceives the railroad tracks converging at one point in the distance demonstrates that there is not a corresponding point in the cartesian coordinate system. however, it is easy to solve this problem by supplementing a new coordinate point with a homogeneous coordinate, thereby establishing the mapping relationship between the cartesian coordinate system r n and an affine coordinate system p n as rn→pn:x x ⋯,xnt→tx x ⋯,xn,0t→t,a→0x x ⋯,xn,at→t,a→ where the infinity point t is just the limit of t under a → . therefore, the infinity point represents the vanishing point of the vision range. it is critical for depth perception of the surrounding scenery that the perspective direction and number of vanishing points are considered when displaying a three-dimensional scene in a two-dimensional plane. when human eyes look into the distance, the fixation point can change in position, and this forms a horizontal vanishing line, as shown in figure  this line is known as the infinity line and is composed of countless vanishing points. similarly, it is also an objective phenomenon that occurs in the visual perception of the external world. it occurs at the intersection of the sky and ground, and provides a broader perspective.  mapping between the scene and visual image the above brief description of previous research aims to introduce the problem of how a three-dimensional cartesian coordinate system converted into a two-dimensional plane is able to express a real three-dimensional scene. this also explains why visual images in the retina can provide three-dimensional scene information to an observer. however, how the cartesian coordinate system in a two-dimensional plane can “faithfully” represent a three-dimensional scene is not known, even though the problem seems trivial. the difference between the stereoscopic perception of actual scenes and a scene in a two-dimensional plane is an important issue in visual information processing, neural computation, psychophysics, and neuroscience, and is also a main research topic in image processing, three-dimensional display methods, and computer vision. figure  shows a cartesian coordinate system. we cannot draw the z axis in the x–y plane such that the included angle between all axes is 90° and the z axis is truly perpendicular to the x–y plane. one approach for drawing the z-axis is to introduce an angle α between the external incident light and the optical axis of vision, where α ≠  for visual purposes, we can only receive incident light from the front; that is, 0 < α is not possible. however, when creating a three-dimensional cartesian coordinate system to express a stereoscopic image, all quadrants of space should be discussed. hence, a more general assumption is |α| ≠  for the sake of simplicity, we discuss the second quadrant. the cases for the other quadrants are the same. when the angle is within the range 90° < θ < 180°, we can express three-dimensional stereoscopic structure in a plane, as shown in figure  according to the statistical results obtained in a psychological experiment that we conducted, when α ≈ 30°, the visual stereoscopic perception is strong and the image structure is stable. it can now be envisaged that to have a true z′ axis perpendicular to the x–y plane, the projection of the z axis onto the z′ axis is z cos = z cos α; that is, the projection of the z axis onto the z′ axis is simply z p, which is equivalent to the value along the z axis in real three-dimensional space. for example, if α = 30°, then zp=zcosα=zcos30∘=3/2z= z the actual loss of depth information along the z-axis, or the information loss of visual depth perception, is zloss = z - z p  = z -  z = z =  z. naturally, α can have different values and indicate different depths. this is consistent with our experience of visual perception, although we usually pay no attention to it. as already pointed out, there is a conjugate relation between the object point and its image point. when an observer sees a three-dimensional scene iwr = pw in the external world, a corresponding visual image i¯rw = pvx,y,z′;vxvyvz′;λ,t forms on the retina that is more than two-dimensional but less than three- dimensional. in turn, if there is a visual image i¯rw on the retina, then the observer perceives the scene of the external world iwr according to i¯rw. hence, the scene and image have the mutually conjugate mapping iwr⇄pvx,y,z′;vx,vy,vz′;λ,tpwx,y,z;vx,vy,vz;λ,ti¯rw iwr⇄z′zi¯rw z′↔zcosθ-90∘=zcosα that is, the actual scene pw is transformed by z cos α and forms the visual image pvx,y,z′;vxvyvz′;λ,t on the retina. it is important to note that, with this transformation relationship, human vision using a two-dimensional image on the retina can perceive an actual three-dimensional scene. according to the above discussion, it is clear that the scene pvx,y,z′;vxvyvz′;λ,t can be drawn on a two-dimensional plane, and it can be expresses as a stereoscopic image on the retina and provide stereoscopic perception. the basic concept of this information processing is more clearly expressed as pwx,y,z;vxvyvz;λ,t↔pv=pvx,y,zcosα;vxvyvzcosα;λ,t=pvx,y,zp;vxvyvzp;λ,t that is, through the plenoptic function pw, iwr forms a visual image i¯rw on the retina. conversely, the visual image i¯rw matches the external world through the plenoptic function pv, and the loss of image information between iwr and i¯rw is approximately z cos α. of course, this is largely a proof of principle, but this discussion demonstrates that it can be used for studies in visual information processing. it has been confirmed in many eye tracker tests, including psychophysical experiments that the visual system can adjust with eye movements to find a suitable viewing angle and orientation so that the loss of information is minimal. this is a fundamental property of the visual system and means that forming visual images on the retina and in the v cortex does not require inversion and reconstruction, possibly because the computational cost is too high to solve its inverse, an ill-posed problem without a unique solution. loss of information due to the introduction of a three-dimensional cartesian coordinate system in the plane figure  shows three groups of three-dimensional cartesian coordinate systems introduced in a two-dimensional plane. the main differences are the different angles between the x- and z-coordinates; i.e., orientations of the z-axis relative to the x-y plane are different and thus, the stereoscopic visual perception is also different. in the example of figure 4a, when angle θ is 120°, the perpendicular relationship among the three axes x, y, and z is most obvious. in figure 4a, when angle θ is 90°, there is still a perpendicular relationship among all three axes x, y, and z in the actual space. however, it is not possible to draw a real vertical line perpendicular to the x-y plane itself. it is instead projected points of this vertical line at the coordinate origin by a perpendicular projection into the x-y plane. on the contrary, it can be seen in figure 4a, when angle θ is 120°, that the z-axis and the x-y plane and the z-axis and x-coordinate axis have an included angle of 120°. therefore, it can be considered that, in actual space, its projection in the z axis is cosα=cosθ‒90∘=cos30∘=3/2=. in other words, the information loss of depth perception is approximately. in contrast, the cartesian coordinates can be introduced in the plane and can provide visual stereoscopic perception. for figures  and the situation is similar. role of the vanishing point in stereoscopic visual perception figure  further illustrates the important role of the vanishing point when introducing a three-dimensional cartesian coordinate system into a two-dimensional plane to represent a stereoscopic scene. figure  is the convergence of projection in a single direction with only a single vanishing point. figures  and show the convergence of two and three projective directions with two and three vanishing points, respectively. please note that the blue lines are the x, y, and z coordinates of the cartesian three-dimensional rectangular coordinate system that can be found in figures  and the existence of the vanishing point is the fundamental reason why a cartesian three-dimensional rectangular coordinate system can be drawn in a two-dimensional plane. as mentioned above, it can be easily seen that the formation of vanishing points underlies the optical system of human vision. it is also the basis of an affine transformation by which the human visual system is able to perceive the three-dimensional external world, as illustrated in the case of railroad tracks that converge to a single point, forming of a vanishing point. dual-plane parameterization of the plenoptic function for neural computation of early vision we know that each pixel of a two-dimensional digital image is a record of the intensity of all light that reaches this point, but does not distinguish between the directions of the light rays. it is just a projection of the light field of the three-dimensional structure, with lost information about phase and direction. unlike this, the light field refers to the collection of light from any point in space in an arbitrary direction. it comprises all light from different angles that makes a contribution to each pixel. if it takes into account how the angle of light changes with time, it is a dynamic light field. the plenoptic function is a good mathematical description of the dynamic light field. however, questions remain regarding how the human visual system perceives and processes the structural information of the dynamic light field as well as how it receives three-dimensional information from the image on the retina. studies by zeki, livingstone et al. have indicated that in the human visual system color information is transmitted in a separate channel in the cerebral cortex. therefore, wavelength λ can be separated from the plenoptic function. in addition, position, direction, and orientation information can also be separated. in this way, without considering time variation and separating dimensions, the seven-dimensional plenoptic function pv = pv can describe and reconstruct plenoptic images, or visual information of the objective world with different combinations of variables. when the viewer’s eyes are looking at a point in any scene, emitted or reflected light rays from this point will enter the eye. the intensity information of the incident light ray carried in vx, vy, vz is received by the eye. since the optical and the coordinate z axes are the same, the light intensity of the stimulus is converted into the strength of photosensitive cell activity. therefore, only angles θ andφ of the light need to be recorded. for this reason, the plenoptic function can be parameterized using a dual-plane representation formed from p and p, as shown in figure  a light ray intersects with the position plane p and angle plane p at and, respectively. the coordinates of the points of intersection and can be used to describe this plenoptic function. the form of a two-plane parameterization is very simple and intuitive. hence researchers have used this method for visualization of light field data, namely, using double nested coordinates to arrange the data of a four-dimensional light field into a two-dimensional plane, forming two symmetrical representations. figure  is an example in which p is the inner angle plane and p is the outer position plane. as can be seen, a light ray at different angles corresponds to different viewing angles in the imaging plane. therefore, such representation of the light field may be closely related to a neural representation of the retina and primary visual cortex of the human visual system. many experiments in neurobiology have shown there are topological mappings with a one-to-one correspondence between the retina and the v cortex that is established through projections from the ganglion cells via the lateral geniculate nucleus to the primary visual cortex. through photosensitive cells, the retina records the position information of the incident light ray, while the v cortex processes the orientation information through simple and complex cells, as well as the orientation function columns. therefore, for early visual information processing, this is a viable solution. it minimizes the loss of information as much as possible without making the algorithm too complex. of course, to do this is not an easy matter, and whether the human visual system employs this strategy needs further study. three-dimensional visual perceptions of images in a two-dimensional plane we know that if an image of a scene on a plane does not contain depth information, the human visual system has no way of perceiving the scene three-dimensionally. when observing the external world, human vision has characteristics of perceptual constancy. this constancy is the basis of an affine transformation, which depends on vanishing points and vanishing lines in visual perception and is determined by the characteristics of the optical system of the visual pathway. as rock pointed out, the height of an object in the base plane is an important depth cue. it can be calculated according to s=δad where s is the height of the object on the fundamental plane, δ is the viewing angle of the camera, d is the distance between the photographer and the object, or the depth information, and a is the scaling factor of the retina. formula is used to reconstruct a three-dimensional scene from an image in a two-dimensional plane. figure  is an optical model of the affine transformation of the retina. for the sake of simplicity, we analyze only the example of one vanishing point, as shown in figures  and the main purpose of the calculation example is to show that we can use the vanishing point, size constancy and affine transformation model in figure  to calculate the depth value in a picture taken of an actual scene. a comparison of the calculation results with actual measurements reveals that the vanishing point reflecting the basic characteristics of the optical system of human vision and size constancy reflecting cognitive psychological characteristics are important in accessing depth information in a two-dimensional picture. the example focuses on the absolute depth perception of white markers, edges on the ground and nine trees. comparisons with measurements are listed in figure  specific calculations are carried out employing two methods. the first method employs psychological methods based on formulae and, and the second method employs an affine transformation based on an optical model of vision. known parameters required for the calculation are the height of the camera from the ground and the horizontal distance between the photographer and first white line on the ground. the camera is a nikon-e3700ccd, and the image size is 2048 × 1536 pixels. the calculation includes the vanishing point, the vanishing line, the height of the tree, and the line whose change in depth value is fastest on the ground portion of the image plane. specific calculations are found in the literature. naturally, algorithms of computer vision can also be used. the results of both calculation methods are consistent with actual measurement results, showing that the calculation methods are reasonable and reflect the consistency between visual psychology and the optical system of visual pathways in the depth perception of an actual scene. more importantly, the results show that a two-dimensional image can contain rich three-dimensional information that is perceived by the visual system itself. we know that when looking at an image or a scene from different angles, the perceived depth of field changes. to show depth information provided by constancy and the affine transformation in a two-dimensional image plane, formula is corrected according to equation, such that the image height of the object may be calculated according to s=δcosαad=δcosθ-90∘ad, where α =  is the included angle between the z-axis and z′ -axis when looking at the image. hence, we use formula to correct the result of the depth information given by method and these corrected values are also given in figure  after taking into consideration information loss, the corrected value roughly reflects the visual depth perception obtained from the image. the proposed method is completely different from three-dimensional image reconstruction that uses binocular disparity and corresponding points in the field of visual computational theory, or three-dimensional reconstruction using corresponding points in two images taken by two cameras in the field of computer vision. the processing method of visual perception has advantages such as efficiency, robustness, and low computational complexity. it is therefore worthy of study by researchers in the fields of computer vision and visual neural computational theory. in appendix according to figure  figure  the formulae, and we will make some predictions about stereoscopic perception of the image on a two-dimensional plane, including: the picture, in which there is no vanishing point; alternating process of cartesian coordinate system and affine coordinate system; the moon illusion, and the inversion reconstruction of visual image. discussion this article explores how the human vision system extracts depth information from an image of a scene in a cartesian rectangular coordinate system on a two-dimensional plane. we introduced the concepts of a plenoptic function in the optical system of the visual pathway. in the section of methods “computational approach in visual cortex v1”, we proposed an algorithm of coincidence test, in which an image primitive r u,v transferred by ganglion cells from retina to visual cortex v will coincide with neurons’ receptive field [b θ,φ ] Θ × Φ in cortical columns. note that, all of neurons in the columns simultaneously carry out compliance testing operations in parallel manner, neuron of [b θ,φ ] Θ × Φ, which most consistent with the image primitives r u,v, is activated and its firing rate is strongest, so that each image primitive r u,v can be detected. because it is distributed and parallel processing, the mathematic operation of coincidence test is very simple, robust, fast and completely consistent with the pattern of stimulating → firing → response of neurons. based on the biological function and structure of the visual pathway and the primary visual cortex, we proposed the dual-parameterized method, which can be expressed as p ⊗ p, and is mathematically equivalent to the formula pv = [r u,v ] u × v  ⊗ [b θ,φ ] Θ × Φ, or to formula  as described as follows. in this paper, we have raised an issue “in the two-dimensional plane, why can three-dimensional structure of a picture be expressed by adopting cartesian coordinate system?”, its importance is to study the information processing from 2d retinal image to three-dimensional visual perception. based on neural computation of visual cortex v and taking into account the affine transformation processing of visual image information and size constancy of visual perception, and also considered the findings of psychophysics. however, formula and figure  show that the psychology of visual perception can explain how the human vision perceives a three-dimensional scene from a two-dimensional retina. because of a structured light field that densely fills the surroundings, human vision processes information according to formulae and. the information loss from the three-dimensional scene in the external world to a visual image in the two-dimensional retina is small, and hence the visual image on the retina contains the rich information of the three-dimensional scene. therefore, we may consider the visual system as a causal system, meaning that the scene has a one-to-one correspondence with the visual image. the scene produces a visual image in the retina, and conversely, if a visual image is formed in the retina, then a viewer perceives the external scene that produced that visual image in the retina. we know the reconstruction of visual image is just a hard inverse problem as a major topic of research in computer vision, its concern is how to use binocular disparity information to find a stable and efficient reconstruction algorithms; it is also an issues concerned by current 3d display technical, its focal point is that this kind of research will able to provide an effective method for better 3d display technology; of course, it is also hard problem to trouble the research of biological vision, vision research mainly is to start from unified basic viewpoint of the biological function and structure of the vision and then explore how to achieve the following information processing by human visual system, namely: from retinal images of three-dimensional scenes to → 2d visual image, and to → 3d visual perception. in the first section “mapping between the scene and visual image ” of this paper, this issue has been discussed in more detail, in which the formulas and had shown that there is no specific reconstruction algorithm from 2d retinal images to three-dimensional scene. at present, to an image, the processing time of the brain has been determined by using an approach of rapid serial visual presentation of image series and cognitive psychological method, it is just 13 ms. so fast processing speed shows that human vision may not be obtained three-dimensional depth perception by using reconstruction method based on the corresponding point, because this method and related algorithms are too complicated, the computational cost is also too high, for this reason, it is impossible to implement such a reconstruction algorithms by using the neurons, neural circuits and partial network. this paper studies how to obtain stereoscopic visual perception, when viewing pictures on the plane, obviously, this issue has important significance for vision information processing; of course, it is also the same for computer vision. according to figures  and the formulae, and we may make some predictions about stereoscopic perception of the image on a two-dimensional plane, including: the picture, in which there is no vanishing point; alternating process of cartesian coordinate system and affine coordinate system; the moon illusion. we have reason to believe that rough outline of theory about three-dimensional visual perception of visual pathway is generally clear.  we know that there are many monocular depth cues that can also form depth perception. however, in this paper, we study how to express stereoscopic visual perception in a two-dimensional plane and only use the parameterized method of a dual plane of the plenoptic function to process the visual information of an image. according to the principle of graceful degradation proposed by marr, if the visual system calculates a rough two-dimensional description from an image, it will be able to calculate a rough three-dimensional description represented by this image. in other words, human vision can perceive the real three-dimensional description from stereoscopic images on a two-dimensional plane. marr posed the problem in this way: “the contours of the image are two-dimensional, but we often come to understand these contours from the perspective of three dimensions. therefore, the key question is how do we make a three-dimensional interpretation of the two-dimensional contour? why can we make this explanation?” we have studied this issue, and to answer marr’s question, this paper presents a preliminary explanation. the main results are as follows: two different plenoptic functions to describe the objective world were introduced. the difference between these two functions pw and pv regarding the external scene obtained by visual perception were analyzed, and their specific applications in visual perception were discussed. the main results were how the processing of visual depth information perceived in stereoscopic scenes can be displayed in a two-dimensional plane. constraints for the coordinates and an algorithm implementation were also provided, in particular, a method used to separate the plenoptic function and a transformation from the retina to the visual cortex. a dual-plane parameterized method and its features in neural computing from the visual pathway to visual cortex v were discussed. numerical experiments showed that the advantages of this method are efficiency, simplicity, and robustness. size constancy, a vanishing point, and vanishing line form the psychophysiological basis for visual perception of the external world, as well as the introduction of the three-dimensional cartesian rectangular coordinate system into a two-dimensional plane. this study revealed the corresponding relationship between perceptual constancy, the optical system of vision, and the mapping of the vanishing point and line in the visual image on the retina. the main results of this paper are a preliminary explanation as to why and how the cartesian rectangular coordinate system can be introduced into a two- dimensional plane, and how a three-dimensional scene can be perceived in a two-dimensional plane. the results of this study are of significance in visual depth perception and possibly in applications of computational vision. 
the results of this study show that the use of plenoptic functions and their dual plane parameterizations can not only explain the nature of information processing from the retina to the primary visual cortex and, in particular, the characteristics of the visual pathway’s optical system and its affine transformation, but they can also clarify the reason why the vanishing point and line exist in a visual image. the main results include a basic method for introducing a three-dimensional cartesian coordinate system into a two-dimensional plane to express the depth of a scene, its constraints, and algorithmic implementation. plenoptic functionvisual perceptionthree-dimensional scenevanishing pointretinaprimary visual cortexneural computationaffine transformation we introduce two different mathematical expressions of the plenoptic functions, p in particular, we include a method to separate the plenoptic function and proceed with the corresponding transformation in the retina and visual cortex. we propose that size constancy, the vanishing point, and vanishing line form the basis of visual perception of the outside world, and that the introduction of a three-dimensional cartesian coordinate system into a two dimensional plane reveals a corresponding mapping between a retinal image and the vanishing point and line. that can describe the objective world. in addition, they can better explain the reasons why a three-dimensional cartesian coordinate system can be introduced into the two-dimensional plane to express a real three-dimensional scene. we also analyze the differences between these two functions when describing visual depth perception, that is, the difference between how these two functions obtain the depth information of an external scene. how it is possible to “faithfully” represent a three-dimensional stereoscopic scene using cartesian coordinates on a plane, and how three-dimensional perceptions differ between an actual scene and an image of the same scene are questions that have not yet been explored in depth. they seem like commonplace phenomena, but in fact, they are important and difficult issues for visual information processing, neural computation, physics, psychology, cognitive psychology, and neuroscience.
 how it is possible to “faithfully” represent a three-dimensional stereoscopic scene using cartesian coordinates on a plane, and how three-dimensional perceptions differ between an actual scene and an image of the same scene are questions that have not yet been explored in depth. they seem like commonplace phenomena, but in fact, they are important and difficult issues for visual information processing, neural computation, physics, psychology, cognitive psychology, and neuroscience.  the results of this study show that the use of plenoptic functions and their dual plane parameterizations can not only explain the nature of information processing from the retina to the primary visual cortex and, in particular, the characteristics of the visual pathway’s optical system and its affine transformation, but they can also clarify the reason why the vanishing point and line exist in a visual image. in addition, they can better explain the reasons why a three-dimensional cartesian coordinate system can be introduced into the two-dimensional plane to express a real three-dimensional scene.  we introduce two different mathematical expressions of the plenoptic functions, p w and p v that can describe the objective world. we also analyze the differences between these two functions when describing visual depth perception, that is, the difference between how these two functions obtain the depth information of an external scene. the main results include a basic method for introducing a three-dimensional cartesian coordinate system into a two-dimensional plane to express the depth of a scene, its constraints, and algorithmic implementation. in particular, we include a method to separate the plenoptic function and proceed with the corresponding transformation in the retina and visual cortex. we propose that size constancy, the vanishing point, and vanishing line form the basis of visual perception of the outside world, and that the introduction of a three-dimensional cartesian coordinate system into a two dimensional plane reveals a corresponding mapping between a retinal image and the vanishing point and line. plenoptic functionvisual perceptionthree-dimensional scenevanishing pointretinaprimary visual cortexneural computationaffine transformation 