Patent Publication Number: US-11663704-B2

Title: Distortion correction via modified analytical projection

Description:
BACKGROUND 
     Cameras are used for communication in many different settings. For example, cameras are commonly used in conference rooms to allow video image data of meetings to be sent to remote locations for others to participate in the meetings. In this manner, people at remote locations can enjoy a more collaborative experience than with audio alone. 
     SUMMARY 
     This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure. 
     Examples are disclosed that relate to correcting distortion in an image by applying an analytical geometric projection that has been modified by an amplitude function. One example provides a computing device comprising a logic subsystem and a storage subsystem holding instructions executable by the logic subsystem to receive an image of a scene as acquired by an image sensor, apply a mapping to the image of the scene that maps pixels of the image to projected pixels on an analytical projection that is modified by an amplitude function such that the analytical projection achieves a higher zoom effect on pixels closer to a center of the image compared to pixels closer to an edge of the image, thereby obtaining a corrected image, and output the corrected image. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG.  1    schematically shows an example camera suitable for use in a videoconferencing system. 
         FIG.  2    schematically shows an example camera controller configured to generate distortion-corrected images by applying a projection mapping comprising an amplitude function. 
         FIG.  3    shows an example conference room comprising a camera arranged with a downward pitch angle. 
         FIG.  4    shows an example rectilinear projection based in part on a tilt angle. 
         FIG.  5    shows superimposed images of a scene including a first image obtained using a rectilinear projection and a second image obtained using a projection modified by an amplitude function. 
         FIG.  6    shows a comparison of various projection profiles for a horizontal field-of-view of 136°. 
         FIG.  7    shows a comparison of super-Gaussian projection profiles for various zoom amplitude values. 
         FIG.  8    shows a comparison of super-Gaussian profile sag functions for various zoom amplitude values. 
         FIG.  9    shows a comparison of super-Gaussian profiles that have been normalized for zoom amplitude and width. 
         FIG.  10    shows a comparison of relative pixel position factors for super-Gaussian sag profiles that have been normalized, offset-corrected, and inverted. 
         FIG.  11    graphically shows pixel positions shifted based on horizontal position for an example projection depth profile, within the confines of a rectilinear projection plane. 
         FIGS.  12 A- 12 H  show example field angle maps and example rectilinear projections modified with an amplitude function. 
         FIG.  13    shows an example image that has been processed by applying a tilted rectilinear projection mapping. 
         FIG.  14    shows an image of the scene of  FIG.  13    that has been processed by applying a tilted rectilinear projection mapping that has been modified by a super-Gaussian-based amplitude function. 
         FIG.  15    is a flow diagram depicting an example method for correcting an image by applying an analytical projection mapping that has been modified by an amplitude function. 
         FIG.  16    schematically shows an example computing system. 
     
    
    
     DETAILED DESCRIPTION 
     A camera captures an image of a scene by directing light onto a flat image sensor. In many settings, a wide-angle camera may be used to image multiple subjects of interest that are spread out in a scene. Such cameras employ a wide-angle lens to capture light from a wide angular field-of-view (FOV), but suffer from various types of optical distortion. Standard low geometric distortion lenses may have a horizontal FOV between 40° and 70° and have mild distortion, whereas wide-angle angle (70°-90° FOV) and ultra-wide-angle (&gt;90° FOV) lenses may have more significant distortion at the edges and corners of the image frame. Furthermore, in some settings, particularly indoors such as a conference room, some subjects of interest may be located at different distances from the camera lens, leading to discrepancies in apparent size of the subjects when imaged. The unnatural appearance of people and other objects in the images may provide for an unsatisfactory experience for remote videoconference participants. Wide angle cameras equipped with low geometric distortion lenses may suffer from trapezoidal stretching at higher field angles, while wide angle cameras equipped with fisheye lenses may suffer from curvature of straight-line objects, such as walls of a room, at higher field angles. 
     In some distortion correction approaches, a raw image is mapped to an analytical geometric projection that yields a corrected image. The analytical geometric projection may approximate the arrangement of people and/or other objects of interest in the imaged space, such as a cylindrical projection for people arranged in a semi-circle. In some examples, the mapping may be based upon an optical center of a lends of the camera. Likewise, the projection also may be based on a known or approximate distortion function of the lens. In the corrected image, vertically oriented objects in the scene, such as people standing near the sides of the camera FOV, may appear straight and vertically oriented instead of warped and/or leaning, and thus have a more natural appearance. As used herein, the term “raw image” means an image that is generated without any distortion correction and may include monochrome images, color images, and images that have been at least partially processed. 
     However, some camera placements and arrangements of subjects in the imaged scene may pose challenges to distortion correction. For instance, a videoconferencing camera may be placed at one end of a room, for example high on a wall above, or attached to the top of, a videoconference display, and tilted downwards to capture subjects seated around a conference table. In this example, subjects at a far end of the conference table opposite the camera may be located a relatively far distance from the camera near the center of the raw image, while subjects at the near end of the conference table adjacent to the camera may be located relatively close to the camera near the edges of the raw image. As subjects close to the camera may be located at extreme viewing angles, this may lead to foreshortening effects and/or trapezoidal distortion for these subjects. At the same time, subjects farther from the camera may appear small compared to other subjects. As a result, using an ultra-wide-angle lens in a conference room setting, for example, people in an image standing close to the camera near the sides of the FOV may appear to be warped, standing at non-vertical angles, and/or appear to be unnaturally large, among other distortions. As such, the scene may appear unbalanced to a user. Furthermore, size disparities and various types of distortion may cause faces to be inconsistent across a field of view. This also may pose challenges for facial recognition, gesture recognition, scene understanding, and/or other applications. 
     Accordingly, examples are disclosed that relate to correcting distortion an image by using a projection mapping that may achieve a more consistent face appearance across a FOV than other corrections. The disclosed examples apply an analytical projection that is modified with an amplitude function such that the projection achieves a higher zoom effect for pixels located near the center of the image compared to pixels located at the edges of the image. The modified analytical projection mapping may preserve aspect ratios on a local scale. The zoom effect and distortion correction provided by the modified projection further may help balance a scene across a FOV, and thus maintain face consistency across the FOV. 
     In some examples, the analytical geometric projection that is modified with an amplitude function comprises a rectilinear projection. In such a correction, the projection mapping may correct for distortion while maintaining a rectilinear-like appearance. That is, vertical-line objects may appear straight while curvature may be relatively small, locally, for horizontal-line objects near the top or bottom of the corrected image. Additionally, in some examples, the projection mapping further may be based on a sensed camera tilt angle. As such, the mapping may adjust for a vertical angle bias. 
     Prior to discussing the use of an amplitude function modified by an analytical projection to correct image distortion, an example camera is described with reference to  FIG.  1   . The camera  100  is configured to image a scene  102 . The camera  100  includes an image sensor  104  and a lens  106  positioned to direct object light  107  from the scene  102  onto the image sensor  104 . The lens  106  may take any suitable shape and may including any suitable optical material having any suitable optical characteristics. In some implementations, the lens  106  may be a system of two or more lenses or other optical elements. As a more specific example, the lens  106  may be an f-θ lens, which may comprise a known linear or near-linear, with angle, lens distortion function. The use of an f-θ lens in the camera may help to ensure a nearly linear relationship between the image sensor space and the incidence angle, which may be useful for imaging wide-angle FOV. The lens  106 , having lens elements held in lens barrel  108 , may be maintained in a center fixed position relative to the image sensor  104  via a holder mount structure  110 . In some examples, the holder mount structure  110  is a rigid holder structure that fixes the lens barrel  108 , and thus all elements in the lens  106  relative to the image sensor  104  along every axis in six degrees of freedom (e.g., x, y, z, tip, tilt, azimuth rotation). For example, a fixed-focus camera may have such an arrangement. In some examples, the holder mount structure  110  may allow movement of the lens barrel  108  relative to the image sensor along one or more axes (e.g., for image stabilization and/or focus, such as by placing an auto-focus voice-coil actuator between lens barrel  108  and holder mount structure  110 ). In such examples, the lens  106  is still center fixed relative to the image sensor  104  even though the position of the lens  106  may move along the z axis relative to the position of the image sensor  104 . 
     In the illustrated example camera, the lens barrel  108  is operatively coupled to the holder mount structure  110 . The holder mount structure  110  is mounted to a printed circuit board (PCB)  112 . In one example, the holder mount structure  110  is bonded to the PCB  112  via an adhesive. The image sensor  104  is mounted on the PCB  112  such that an optical axis  114  of the lens  106  is substantially aligned with a center of the image sensor  104 . In particular, the lens barrel  108 , the holder mount structure  110 , and the PCB  112  collectively maintain the lens  106  in optical alignment with the image sensor  104  (e.g., for the case of using a threaded lens barrel  108  and holder mount structure  110 , the holder mount structure  110  may be bonded in position relative to PCB  112  to fix x, y, z position and tip/tilt angle while threads may be substantially used to set the focus). Alternatively, as may be the case for using active alignment (AA), pre-focus position may be set by optically, or mechanically, fixing focus position between lens barrel  108  and holder mount structure  110 . Once fixed in this manner, the lens and holder assembly may be actively adjusted in all degrees of freedom and bonded with a gap bond between holder mount structure  110  and PCB  112  to fix x, y, final z focus, tip, tilt and azimuth rotation. Further still, a threadless lens holder may first be bonded to PCB, followed by AA positioning and bonding of a lens having threadless lens barrel. 
     The camera  100  further comprises a controller  116  configured to control the image sensor  104  to acquire images of the scene  102  as well as to perform other control operations of the camera  100  as discussed herein. The controller  116  includes a logic subsystem and a storage subsystem. The logic subsystem includes one or more physical devices configured to execute instructions held by the storage subsystem to enact any operation, algorithm, computation, or transformation disclosed herein. In some implementations, the logic subsystem may take the form of an application-specific integrated circuit (ASIC) or system-on-a-chip (SoC), in which some or all of the instructions are hardware- or firmware-encoded. The logic subsystem and the storage subsystem of the controller  116  are discussed in further detail with reference to  FIG.  16   . 
     During the process of manufacturing the camera  100 , manufacturing tolerances of the camera  100  may result in camera-to-camera variations in optical alignment of the image sensor  104  and the lens  106  that may result in a position of the image sensor being shifted from an ideal position that is aligned with the lens  106 . The image sensor  104 ′ is shown having a position that is shifted relative to the ideal position of the image sensor  104  that is aligned with the lens  106 . Moreover, manufacturing tolerances of the lens  106  itself may contribute to variations in optical alignment of the image sensor  104  and the lens  106 . As shown in the sidebar  122 , the optical axis  114  of the lens  106  is centered at an actual optical center  118  of the lens  106  relative to the image sensor  104  when the image sensor  104  is ideally aligned with the lens  106 . However, the actual optical center  118  of the lens  106  is offset from a center  120  of the image sensor  104 ′ when the image sensor  104 ′ has a position that is shifted relative to the ideal position of the image sensor  104 . The difference between the center  120  of the image sensor  104 ′ and the actual optical center  118  of the lens  106  may affect the image distortion attributed to the lens. 
     The actual optical center  118  may vary from camera to camera such that different cameras generate raw images having different distortion based on having different camera-specific optical centers. As such, in some examples, the application of an analytic projection distortion correction to an image may be based upon a determined optical center along with lens nominal distortion data. The optical center may be determined in any suitable manner. As one example, the optical center may be determined on a camera-by-camera basis during manufacturing by projecting a flat field of illumination through lens  106  onto the image sensor  104 , and measuring a centroid of an image intensity profile to determine the optical center. Note that if the camera includes lenses that have limited relative illumination roll-off, a live image based pointing measurement including a calibrated test setup having an optical target with positional reference to the lens optical axis, which may be repeatably mounted by use of kinematic mechanical mounting of lens barrel, may be expected to provide higher accuracy for optical center measurement. As another example, a laser beam aligned with the optical axis  114  may be used to measure the optical center  118 . In other examples, an analytic projection distortion correction may be based upon any other suitable calibration. 
       FIG.  2    shows a block diagram of the controller  116  in more detail. The controller  116  is configured to acquire a raw image  204  of a scene via the image sensor  104 . The controller  116  may be configured to load the raw image  204  in storage subsystem  202  of the camera  100 . The controller  116  is further configured to generate a distortion corrected image  214  from the raw image  204 , which may be based on the camera-specific optical center  118  of the camera  100  or other suitable calibration. The measured camera-specific optical center  118  or other calibration may be stored in the storage subsystem  202  (e.g., electrically erasable programmable read-only memory (EEPROM) or other nonvolatile memory) such that the camera-specific optical center  118  or other calibration may be made available for use in performing distortion correction operations. The measured camera-specific optical center  118  may be represented as (x, y) coordinates in a coordinate space of the image sensor  104 . Due to lens distortion and/or alignment variances, the measured camera-specific optical center  118  may be offset from an actual center position of the image sensor  104 . While use of optical center (OC) position data is not required to perform a distortion corrected image, the magnitude of effective distortion of the corrected image may increase with increase in OC deviation from the origin position used in the projection mapping. As an example, in some applications, small OC error, i.e. on order of &lt;5-10 pixels, may provide sufficient accuracy, while other applications may utilize higher accuracy OC data. Further, the optical center may be offset from the center of sensor in order to facilitate an angular bias in the FOV, e.g. offset of the sensor active area along the vertical dimension so as to bias the vertical FOV cone to image more downward in the scene for a camera mounted at a high level in a room. In such case, OC position data may indicate deviation from the target origin position used in the projection mapping. Further still, an offset region of interest, or image crop, from a smaller region within an oversized image sensor area may be used to achieve offset imaging. As an example, offset imaging may be combined with tilt correction, in order to achieve both VFOV bias angle in combination with tilt compensation. 
     The controller  116  includes a distortion correction engine  206  configured to translate pixel locations of pixels of the raw image  204  according to a distortion correction projection  212  comprising an amplitude function to generate the distortion corrected image  214 . In other examples distortion correction may be performed on another computing device, such as a computer receiving image data from the camera  100 , rather than on controller  116 . Note that the pixel locations of different pixels in the raw image may be translated and/or interpolated, such as by application of a mesh grid indicating mapping of each integer (x, y) pixel of a distortion corrected image to a floating-point position within the original input image (x′, y′), on an individual pixel basis based on the distortion correction projection. As such, in different instances, pixel locations of different pixels may be translated differently (e.g., different direction and/or distance of translation for different pixels), pixel locations of different pixels may be translated the same (e.g., same direction and/or distance of translation for different pixels), and/or pixel locations of some pixels may remain the same between the raw image  204  and the distortion corrected image  214 . Furthermore, distortion correction may include stretching and/or compressing portions of an image. 
     As described in more detail below, the distortion correction engine  206  is configured to perform distortion correction mapping according to a distortion correction projection  212  modified by an amplitude function. The distortion correction engine  206  optionally may utilize the measured camera-specific optical center  118 , image sensor parameters  208  and/or lens distortion parameters  210  as inputs. In one example, the image sensor parameters  208  may include a resolution of the image sensor  104  (e.g., a number of pixels included in the image sensor in both x and y dimensions) and a pixel size of pixels of the image sensor  104  (e.g., size of pixel in both x and y dimensions). In other examples, other image sensor parameters may be considered for the distortion correction projection  212 . In one example, the lens distortion parameters  210  may include distortion data, such as image real height versus field angle of the lens  106 . In other examples, other lens distortion parameters may be considered for the distortion correction projection  212 . In some examples, a plurality of distortion correction projections  212  may be stored, e.g., corresponding to different amplitude functions (e.g., conic, cylindrical, parabolic, super-Gaussian), different tilt angles and/or different types of projections (e.g. cylindrical, spherical, and/or rectilinear). 
     Further, in some examples, distortion correction engine  206  may, in response to an input, change from a current projection mapping to a different projection mapping. For example, controller  116  may receive an input for a zoom request and in response, control distortion correction engine  206  to select a distortion correction projection  212  comprising an amplitude function to apply for the selected level of zoom. As discussed below, a user may be able to select a distortion correction projection having an amplitude function with a different zoom amplitude scaling factor, thus controlling an apparent level of zoom. 
     In some examples, tilt angle data from a tilt sensor  213  may be used to select a distortion correction projection  212  to apply, such that the tilt angle of the projection corresponds to a tilt angle of the camera as determined from the tilt sensor data. In some examples, a partial tilt correction is applied which may correspond to approximately half of the camera tilt angle, e.g., an analytical projection comprising a tilt parameter of 8.5° may be used when a camera tilt angle is 17°. Applying a partial tilt correction may avoid foreshortening effects on subjects near the camera while still providing suitable distortion correction. The tilt sensor  213  may be incorporated into the camera or into a computing device comprising the camera, as examples. Further, tilt parameter may be provided by use of dedicated mechanical camera mount having a known tilt angle with reference to the lens optical axis. In yet other examples, the tilt angle may be provided as a user input, rather than determined from tilt sensor data. It should be noted that other geometric distortion correction projections not illustrated herein are envisioned to be compatible with tilted distortion correction, such as toroidal projection or hybrid projections such as upper portion cylindrical and lower portion toroidal or spherical, having transition at equatorial line. Further, the geometric distortion correction projection compatible with tilted distortion correction may include a parabolic profile along the vertical FOV which is radially swept, or curved, along the horizontal FOV. Other geometric distortion correction direct projections compatible with tilted distortion correction may include a projection surface which is linear along the vertical and curved along the horizontal, or curved sheet surface, whereas the curved dimension may include a conic profile, such as a cylindrical or parabolic or other conic profile, or a super-Gaussian profile. 
     The sensor parameters  208  and the lens distortion parameters  210  may be known a priori for the particular type of camera configuration that uses the lens  106  and the image sensor  104 . For example, the sensor parameters  208  and the lens distortion parameters  210  may be the same for every camera in a particular manufacturing lot, whereas each camera of the particular manufacturing lot may have a different measured camera-specific optical center due to variations in manufacturing of the cameras. In various implementations, sensor parameters  208  and lens distortion parameters  210  may be stored in storage subsystem  202 , and/or hard coded into the distortion correction algorithm(s). 
     The distortion correction projection  212  defines a relationship between the pixel locations of the raw image  204  and the translated pixel locations of the distortion corrected image  214  as an inverse function in which the sensor coordinates are mapped to projection plane and/or surface coordinates of the distortion correction projection  212 . As mentioned above, the distortion correction projection  212  comprises an amplitude function that achieves a higher zoom effect on pixels closer to the center of the image compared to pixels closer to an edge of the image. 
       FIG.  3    shows an example conference room  300  comprising a videoconferencing display and camera, in which camera  100  is placed to a side of display  302  with a downwardly tilted pitch angle. In  FIG.  3   , the camera  100  is located above the display  302 , but may be located to a different side of the display  302  in other examples, including left, right, or below in some examples. 
     As discussed above, a distortion correction can comprise applying a mapping that maps pixels of an image onto an analytical projection that has been modified by an amplitude function. The analytical projection may comprise a rectilinear, cylindrical, or spherical projection. 
       FIG.  4    shows a geometric representation of an example rectilinear projection  400  for pitch tilt angle ϕ (prior to modification by an amplitude function). The rectilinear projection maps sensor coordinates in image sensor space to projection plane coordinates of a tilted rectilinear projection plane. Prior to the mapping, the pixel grid of the raw image is parameterized into corresponding physical dimensions and optionally may be shifted based on the measured camera-specific optical center. 
     The image sensor coordinates are represented by (x s , y s ) for the raw image. Projection distance z p  may be used as a scaler for setting the field of view captured within the corrected image pixel positions. For example, for a real height H Re  of the horizontal edge of the image sensor, the corrected image may be set to subtend and include a target horizontal field of view (HFOV) by setting z p  as: 
               z   p     =       (     1     cos   ⁡   (   ϕ   )       )     ⁢       (       H     R   ⁢   e         tan   ⁡   (       (     π   180     )     ⁢     (       H   ⁢   F   ⁢   O   ⁢   V     2     )       )       )     .             
The rectilinear projection defines the relationship between the image sensor coordinates and the projection plane coordinates (coordinates on the plane represented by position (x p , y p ), all points which have been rotated by tilt angle ϕ about a point located a distance z p  along z axis, thus positions in the corrected image) as an inverse function in the form of, x s (f, x p , y p , z p , ϕ) &amp; y s (f, x p , y p , z p , ϕ).
 
     For a point P=(l, m, n) on the untilted plane, relative to the pivot point along z axis, l=x p , m=y p , and n=0. For a given tilt angle ϕ, the lateral position x o  remains constant. Including tilt ϕ of the plane, point P moves to P ϕ =(l ϕ , m ϕ , n ϕ ), where l ϕ =x p , m ϕ =n sin ϕ+m cos ϕ=y p  cos ϕ, and n ϕ =n cos ϕ−m sin ϕ=−y p  sin ϕ, thus coordinates with respect to XYZ origin become
 
 x   o   =l=x   p ,
 
 y   o   =m   ϕ   =m  cos ϕ= y   p  cos ϕ, and
 
 z   o   =z   p   −n   ϕ (= z   p −(− m  sin ϕ)= z   p   +y   p  sin ϕ.
 
Given (x o , y o , z o ) above, the Field Angles are expressed as:
 
             φ   =       sin     -   1       (           x   o   2     +     y   o   2               x   o   2     +     y   o   2     +     z   o   2           )           
while r p =√{square root over (x p   2 +y p   2 )} and z p =distance scaler to plane. For the projection plane coordinates (x p ,y p ):
 
 x   p   =r   p  cos(θ) where
 
               cos   ⁡   (   θ   )     =         x   p       r   p       =       x   p           x   p   2     +     y   p   2                     y   p   =r   p  sin(θ) where
 
     
       
         
           
             
               sin 
               ⁡ 
               ( 
               θ 
               ) 
             
             = 
             
               
                 
                   y 
                   p 
                 
                 
                   r 
                   p 
                 
               
               = 
               
                 
                   y 
                   p 
                 
                 
                   
                     
                       x 
                       p 
                       2 
                     
                     + 
                     
                       y 
                       p 
                       2 
                     
                   
                 
               
             
           
         
       
     
     The radius r s  on the image sensor from an angle to the image height may be represented as a function r s (φ). For example, this relationship may be determined via a distortion lookup table which may be used to interpolate a radial image height for any given field angle, a fit equation, or estimated with a parabolic percentage (p) distortion from a f-θ at Field Angle. The radius r s  may be determined as follows: 
                 r   s     (   φ   )     =       f   ⁢       φ   D     (   φ   )       ∼     f   ⁡   (     φ   +       p   ⁢     φ   3         φ   o   2         )             
For Field Angles φ in radians
 
             φ   =       sin     -   1       (           x   o   2     +     y   o   2               x   o   2     +     y   o   2     +     z   o   2           )           
The distortion lookup table may be defined as follows:
 
                 φ   D     (   φ   )     =       L   ⁢   U   ⁢     T   ⁡   (   φ   )       =     φ   ⁢         H     R   ⁢   e       (   φ   )         H   Ref     (   φ   )                 
For the sensor coordinates (x s , y s ), including rotation due to tilt angle ϕ:
 
     
       
         
           
             
               y 
               s 
             
             = 
             
               
                 f 
                 ⁢ 
                 
                   
                     φ 
                     D 
                   
                   ( 
                   φ 
                   ) 
                 
                 ⁢ 
                 
                   ( 
                   
                     
                       y 
                       o 
                     
                     
                       
                         
                           x 
                           o 
                           2 
                         
                         + 
                         
                           y 
                           o 
                           2 
                         
                       
                     
                   
                   ) 
                 
               
               ∼ 
               
                 
                   f 
                   ⁡ 
                   ( 
                   
                     φ 
                     + 
                     
                       
                         p 
                         ⁢ 
                         
                           φ 
                           3 
                         
                       
                       
                         φ 
                         o 
                         2 
                       
                     
                   
                   ) 
                 
                 ⁢ 
                 
                   ( 
                   
                     
                       y 
                       o 
                     
                     
                       
                         
                           x 
                           o 
                           2 
                         
                         + 
                         
                           y 
                           o 
                           2 
                         
                       
                     
                   
                   ) 
                 
               
             
           
         
       
       
         
           
             
               x 
               s 
             
             = 
             
               
                 f 
                 ⁢ 
                 
                   
                     φ 
                     D 
                   
                   ( 
                   φ 
                   ) 
                 
                 ⁢ 
                 
                   ( 
                   
                     
                       x 
                       o 
                     
                     
                       
                         
                           x 
                           o 
                           2 
                         
                         + 
                         
                           y 
                           o 
                           2 
                         
                       
                     
                   
                   ) 
                 
               
               ∼ 
               
                 
                   f 
                   ⁡ 
                   ( 
                   
                     φ 
                     + 
                     
                       
                         p 
                         ⁢ 
                         
                           φ 
                           3 
                         
                       
                       
                         φ 
                         o 
                         2 
                       
                     
                   
                   ) 
                 
                 ⁢ 
                 
                   ( 
                   
                     
                       x 
                       o 
                     
                     
                       
                         
                           x 
                           o 
                           2 
                         
                         + 
                         
                           y 
                           o 
                           2 
                         
                       
                     
                   
                   ) 
                 
               
             
           
         
       
     
     As mentioned above, such a geometric analytical projection may leave some subjects of interest warped and/or result in size disparities amongst various subjects of interest in some scenarios. Referring again to the conference room example, subjects near the center of the image may appear small compared to other subjects, while subjects near the edges of the image may appear to be leaning and/or suffer from trapezoidal distortion. To help correct these distortions, the amplitude function modifies the projection mapping by pulling some mapping points away from the edge towards an origin, and/or pushing other mapping points away from the origin towards an edge. The origin may be referred to as a central mapping point of reference, which may be a center of the raw image, a focal point in the image of the scene, or other reference point. In some examples, the origin may be vertically offset based on the height of the camera. In some examples the origin may be a user-selected central mapping point. In some examples, the origin may match the camera specific optical center. In some examples, the origin may be vertically offset from the camera specific optical center. 
       FIG.  5    illustrates an example modification of a rectilinear projection via an amplitude function. More particularly,  FIG.  5    shows two example images superimposed: a first image obtained using an unmodified rectilinear projection (or modified rectilinear projection with amplitude set to zero) and a second image obtained using a rectilinear projection that has been modified with an amplitude function. In  FIG.  5   , black lines are shown extending from an origin  502  to a selection of mapping points in the first image. For example, mapping point  504  is located at corner of a display.  FIG.  5    also shows white lines extending from origin  502  to updated projection points in the second image corresponding to the mapping points. For example, updated projection point  506  is located at a display corner in the second image corresponding to the display corner of mapping point  504 . As shown in  FIG.  5   , the amplitude function has the effect of pushing pixels away from origin  502 . Further, the type of lens distortion of lens used in the camera being corrected may be selected to provide improved resolve of pixels in the central region that has zoom appearance, e.g. for wide FOV applications, the post dewarp pixel density of central zoomed region may benefit from use of a wide angle fisheye lens (f-theta) over low TV distortion lens (gnomonic, or f-tan(theta)). 
     An example modification of a projection via an amplitude function will now be described with reference to the rectilinear projection, but any suitable geometric analytical projection may be used. First, consider a mapping point (x 1 , y 1 ) in the analytical projection and an origin (x 0 , y 0 ). The amplitude function modifies the horizontal position x 1  by a factor c to a modified position x 2 , where x 2 =cx 1 . Next, y 2  is chosen such that the slope of the line from (x 2 , y 2 ) to the origin matches the slope of the line from (x 1 , y 1 ) to the origin: 
                   y   2     -     y   0           x   2     -     x   0         =         y   1     -     y   0           x   1     -     x   0               
where (x 1 , y 1 ) is a point in the analytical projection, (x 2 , y 2 ) is an updated projection point, and (x 0 , y 0 ) is the origin (central mapping point of reference). For example, FIG.  5  shows the location of updated projection point  506  where the slope from updated projection point  506  to origin  502  (i.e., slope of white line) is the same as the slope from mapping point  504  to origin  502  (i.e., slope of black line). Rather than scaling each point by a constant factor c, a function c(x) is used, thus x 2 =c(x)x 1 . Solving the above for y 2  yields a solution for the updated projection point in terms of the function c(x).
 
                 x   2     =       c   ⁡   (   x   )     ⁢     x   1         ,                   y   2     =       y   0     +       (         c   ⁡   (   x   )     ⁢     x   1       -     x   0       )     ⁢     (         y   1     -     y   0           x   1     -     x   0         )           ,         
where c(x) is a non-linear function.
 
     The function c(x) determines the relative zoom effect for pixels in the projection mapping. For example, in  FIG.  5    near the center of the image, c(x) may be relatively large as updated projection point  506  is almost twice as far from origin  502  as mapping point  504 . However, near the edge of the image, c(x) may be smaller as updated projection point  510  is relatively close to mapping point  508 . As such, a higher relative zoom effect may be achieved for pixels closer to the center of the image. The rectilinear projection discussed above ( FIG.  4   ) may be modified to include the non-linear re-mapping of the x-grid axis values, thus adding a mapping from (x 1 , y 1 ) to (x 2 , y 2 ), as illustrated in  FIG.  5   . As a result, a projection mapping that maps a sensor pixel to (x 1 , y 1 ) can be modified by an amplitude function to instead map the sensor pixel to (x 2 , y 2 ). The projection mapping may comprise a direct mapping or indirect mapping. In a direct mapping, the amplitude function modifies the projection surface, to which pixels are projected. In an indirect mapping, the amplitude function modifies the grid positions, or mapping, within the projection plane. 
     Determination of the amplitude function may be based on a depth profile, which may approximate subjects of interest in a scene. The depth profile may be a curved surface defined by a conic function (e.g., cylindrical, elliptical, parabolic), a super-Gaussian function, or other suitable function. As one example, a conic profile can be defined as: 
               z   ⁡   (     A   ,   v   ,   w   ,     k   o     ,   x     )     =     v   +     A   ⁢       (       w     1   +       -     k   o             -       x   2       w   ⁡   (     1   +       1   -         (     1   +     k   o       )     ⁢     x   2         w   2             )         )       (     w     1   +       -     k   o             )                 
where z is the profile depth, A is the sag, v is the edge height, w is the edge width, k o  is an effective conic constant, and x is the horizontal position across the x-dimension. The edge width and edge height are defined as:
 
             w   =     c   ⁢     R   o     ⁢     sin   ⁡   (       (     π   180     )     ⁢     (       H   ⁢   F   ⁢   O   ⁢   V     2     )       )                   v   =     c   ⁢     R   o     ⁢     cos   ⁡   (       (     π   180     )     ⁢     (       H   ⁢   F   ⁢   O   ⁢   V     2     )       )             
where R o  is a nominal radius, c is an edge placement factor, and HFOV is the horizontal field of view. The parameters w and v determine the edges of the field of view as defined by a curved analytical projection surface. The sag of the projection profile is:
 
             A   =       (       A   o     -     (     c   -   1     )       )     ⁢       R   o     (     1   -     cos   ⁡   (       (     π   180     )     ⁢     (       H   ⁢   F   ⁢   O   ⁢   V     2     )       )       )             
where A o  is a scaling factor which affects the zoom (magnification) amplitude, described in more detail below. In many of the examples presented herein, HFOV≈136°, but any camera with any suitable FOV may be used.
 
     The effective conic constant k o  may be set based on a desired profile shape and field-of-view. The effective conic constant can be defined as: 
                 k   o     (     k   ,     H   ⁢   F   ⁢   O   ⁢   V       )     =     -     (         cos   ⁡   (       (       H   ⁢   F   ⁢   O   ⁢   V     2     )     ⁢     (     π   180     )       )     2     -   k     )             
where k is a function parameter input controlling the conic shape, and HFOV is the horizontal field-of-view. If k is set to 0 the resulting profile will be cylindrical, whereas k=−1 will result in a parabolic profile. Other values may yield elliptical shapes of varying eccentricity. For example, for a cylindrical profile shape and 136° horizontal field of view, k o (0,136°)=−0.14033 and this value is used to generate the depth profile using the equations above. Likewise, for a parabolic shape, k o (−1,136°)=−1.14033 may be used to generate the depth profile.
 
     In other examples, the depth profile can be based on a super-Gaussian function. The super-Gaussian profile depth z can be defined as: 
               z   ⁡   (       A   o     ,     R   o     ,     T   fwhm     ,   m   ,     w   o     ,     x   edge     ,     z   edge     ,     v   o     ,   x     )     =       z   edge     +       A   o     ⁢     R   o     ⁢     e       -     (     1   2     )       ⁢       (         Abs   (       x   ⁢     w   o         x   edge       )     ⁢     (     2   ⁢       Log   (   2   )       (     1   /   2   ⁢   m     )         )         T   fwhm       )       2   ⁢   m             -     v   o             
where T fwhm  is the full-width at half-max, m is a parameter that determines the sharpness of the profile, x edge  is the x-position at the edge, z edge  is the z-position at the edge, w o  is the profile normalized width, v o  is the function height, and the other variables are as defined above. The x and z positions at the edge are:
 
               x   edge     =     c   ⁢     R   o     ⁢     sin   ⁡   (       (     π   180     )     ⁢     (       H   ⁢   F   ⁢   O   ⁢   V     2     )       )                     z   edge     =     c   ⁢     R   o     ⁢     cos   ⁡   (       (     π   180     )     ⁢     (       H   ⁢   F   ⁢   O   ⁢   V     2     )       )             
where R o  is the nominal radius, c is the edge placement factor, and HFOV is the horizontal field of view as above. The function height v o  is:
 
               v   o     =       A   o     ⁢     R   o     ⁢     e       -     (     1   2     )       ⁢       (         Abs   (     w   o     )     ⁢     (     2   ⁢       Log   (   2   )       (     1   /   2   ⁢   m     )         )         T   fwhm       )       2   ⁢   m                   
which is the height at the profile normalized width w o . Finally, the sag of the super-Gaussian projection profile is v sag =A o R o −v o . In other examples, other suitable depth profiles may be used.
 
       FIG.  6    shows example depth profiles comprising a super-Gaussian profile, a cylindrical profile, a parabolic profile, and a general conic (elliptical) profile. Such depth profiles are generated using the equations above and may be used in a direct mapping or indirect mapping. In the direct mapping case, a depth profile may be used directly as a projection surface for the analytical projection. For example, a rectilinear projection mapping that projects onto a plane  602  may be modified by a super-Gaussian amplitude function to instead project onto super-Gaussian profile  604 . In the indirect mapping case, a depth profile may be indirectly converted into a zoom amplitude function within a rectilinear plane. For example, super-Gaussian profile  604  may be converted into a zoom amplitude function within rectilinear plane  602 . 
     A distortion correction projection may be precomputed and stored as a pixel mapping function (a mesh file) that is applied to pixels in the raw image to produce the distortion corrected image. In some examples, the distortion correction projection may be precomputed for each of a plurality of pitch angles. In still further examples, other geometric analytical projections may be used, such as a cylindrical projection or a spherical projection. 
     In some examples, a camera may be configured to be used with multiple different pitch angles. For example, a camera may have a pivoting joint at which a camera body attaches to a mount. For a camera incorporated into the body of a device (e.g. a camera positioned in a bezel of a display), the device, or a mount for the device, may include a similar joint. Likewise, a device incorporating a camera, such as a display, may have different pitch angles when mounted to a stand compared to a wall. Thus, to allow appropriate tilted projections to be applied, the camera, device incorporating the camera, and/or a camera mount, may include a tilt sensor (e.g. an inertial motion sensor, rotational encoder incorporated into the pivot joint, etc.) to allow sensing of a pitch tilt angle. Based upon a sensed pitch tilt angle, a corresponding projection mapping may be applied to correct images. As discussed above, in some examples a projection mapping corresponding to a partial tilt correction may be applied. 
     In the examples shown, the field-of-view  606  comprises a HFOV=136°. The depth profiles intersect the field-of-view at different edge points (e.g., edge points  608 ,  610 ,  612 ,  614 ). The location of these edge points is dependent on the edge placement factor c, described above. The edge placement factor c, the effective conic constant k o , the zoom amplitude A o , and other parameters may be used to adjust the shape of the depth profile. In particular, zoom amplitude A o  may be adjusted to achieve different levels of zoom effect in the modified analytical projection. In some examples, the profile, parameters, and/or zoom amplitude may be optimized for a given HFOV. 
       FIG.  7    shows example super-Gaussian profiles for various zoom amplitude values A o . When A o =0 the profile is flat, which corresponds to a rectilinear projection. When A o =1 the profile extends to the nominal radius R o  at the center of the HFOV. The profile depth is correlated to the zoom effect. For horizontal positions near x=0, the profile is deeper and therefore the zoom effect is greater compared to horizontal positions close to the edges. Further, for larger values of A o , the depth profile has a greater depth at the center of the HFOV, while for smaller values of A o , the depth profile is relatively less deep. As such, pixels close to the center of the image may appear magnified to a greater degree when using a profile with a higher zoom amplitude A o  compared to the apparent magnification when using a profile with a smaller A o  value. While only five profiles are shown for five non-zero A 0  values, it will be understood that any suitable amplitude scaling factor A 0  may be used. In some examples, a super-Gaussian amplitude function is used having A 0  between 0.7 and 0.9, while in other examples any other suitable value can be used for A 0 . 
     In some examples, the amount of zoom amplitude may change, e.g., in response to a user input request. In other examples, the amount of zoom amplitude may be fixed. Since the amount of zoom amplitude and zoom function profile may be optimized for a given maximum HFOV, in some examples, the zoom amplitude is maintained and scaled such that the appearance may be consistent for a variety of changes in HFOV and/or tilt. As such, the zoom amplitude may change based on a change in HFOV. In still other examples, the zoom profile function may be used to purposely change the appearance of image content as if imaged from different z vantage points. For example, a zoom profile function may be chosen to make a wide-angle scene image appear as if captured from a greater distance away and with greater zoom magnification, although the image of scene is captured by a wide-angle camera and at closer distance. 
     As discussed above, the depth profiles may also be converted to a zoom function to be used within a rectilinear projection plane, thereby providing an indirect mapping that can be applied. The edge-to-edge sag functions of the depth profiles can be normalized and inverted to obtain relative position factor mappings that can be used to modify an analytical projection. The edge-to-edge sag function equation for the conic profile is: 
               z   ⁡   (     A   ,   w   ,     k   o     ,   x     )     =     A   ⁢       (       w     1   +       -     k   o             -       x   2       w   ⁡   (     1   +       1   -         (     1   +     k   o       )     ⁢     x   2         w   2             )         )       (     w     1   +       -     k   o             )               
where A is the sag, w is the edge width, k o  is an effective conic constant, and x is the horizontal position across the x-dimension. Similarly, edge-to-edge sag function equation for the super-Gaussian profile is:
 
               z   ⁡   (       A   o     ,     R   o     ,     T   fwhm     ,   m   ,     w   o     ,     x   edge     ,     v   o     ,   x     )     =         A   o     ⁢     R   o     ⁢     e       -     (     1   2     )       ⁢       (         Abs   (       x   ⁢     w   o         x   edge       )     ⁢     (     2   ⁢       Log   (   2   )       (     1   /   2   ⁢   m     )         )         T   fwhm       )       2   ⁢   m             -     v   o             
where the parameters are the same as defined above.
 
       FIG.  8    shows super-Gaussian profile sag for the various profiles of  FIG.  7    that may be obtained using the equations above. In the example shown, the HFOV is 136° and the nominal radius R o  is 2.5 mm. At the center of the field of view (x=0), the profile sag z is greater for sag functions with larger zoom amplitude values A o . 
     Next, the above equations for the sag profiles can be normalized with respect to amplitude and width. For the conic sag profile, one can normalize the edge width by either setting w=1 or by introducing w factor to x as x→wx. The normalized function for the conic sag profile is: 
               z   (       A   o     ,   w   ,     k   o     ,   x     )     =       A   o     ⁢       (       w     1   +       -     k   o             -         (     w   ⁢   x     )     2       w   ⁡   (     1   +       1   -         (     1   +     k   o       )     ⁢       (     w   ⁢   x     )     2         w   2             )         )       (     w     1   +       -     k   o             )               
where z is the normalized profile depth, A 0  is the amplitude scaling factor, and w is the width of the conic profile as defined above.
 
     Similarly, the super-Gaussian sag profile can be normalized with respect to amplitude and width by setting x edge →1 and R o →1. The normalized function for the super-Gaussian sag profile is: 
               z   ⁡   (       A   o     ,     T   fwhm     ,   m   ,     w   o     ,     v   o     ,     v   sag     ,   x     )     =         A   o     (         A   o     ⁢     e       -     (     1   2     )       ⁢       (         Abs   (     x   ⁢     w   o       )     ⁢     (     2   ⁢       Log   (   2   )       (     1   /   2   ⁢   m     )         )         T   fwhm       )       2   ⁢   m             -     v   o       )     /     v   sag             
for
 
               v   o     =       A   o     ⁢     e       -     (     1   2     )       ⁢       (         Abs   (     w   o     )     ⁢     (     2   ⁢       Log   (   2   )       (     1   /   2   ⁢   m     )         )         T   fwhm       )       2   ⁢   m                   
and v sag =A 0 −v o , where A o  is the zoom amplitude factor.  FIG.  9    shows the normalized super-Gaussian sag profiles for various amplitude scaling factors from 0 to 1.0.
 
     Next, the normalized sag profiles are inverted with a unity offset to generate a relative position factor mapping that can be applied within the rectilinear plane. The equation for the normalized, offset, inverted conic profile is: 
               c   ⁡   (   x   )     =       1   /     z   (       A   o     ,   w   ,     k   o     ,   x     )       =     1   /     (     1   +       A   o     ⁢       (       w     1   +       -     k   o             -         (     w   ⁢   x     )     2       w   ⁡   (     1   +       1   -         (     1   +     k   o       )     ⁢       (     w   ⁢   x     )     2         w   2             )         )       (     w     1   +       -     k   o             )           )               
where k o  can be adjusted to correspond to the desired conic profile as described above.
 
     The equation for the normalized, offset, inverted super-Gaussian profile is: 
               c   ⁡   (   x   )     =       1   /     z   ⁡   (       A   o     ,     T   fwhm     ,   m   ,     w   o     ,     v   o     ,     v   sag     ,   x     )       =     1   /     (     1   +         A   o     (         A   o     ⁢     e       -     (     1   2     )       ⁢       (         Abs   (     x   ⁢     w   o       )     ⁢     (     2   ⁢       Log   (   2   )       (     1   /   2   ⁢   m     )         )         T   fwhm       )       2   ⁢   m             -     v   o       )     /     v   sag         )               
where, as before, v sag =A 0 −v o  and
 
               v   o     =       A   o     ⁢       e       -     (     1   2     )       ⁢       (         Abs   (     w   o     )     ⁢     (     2   ⁢       Log   (   2   )       (     1   /   2   ⁢   m     )         )         T   fwhm       )       2   ⁢   m           .             
The other parameters are as defined above.
 
     From these relations, a mapping from x 1 →x 2  is generated and applied within the rectilinear projection plane, thereby providing an indirect mapping of a raw image (e.g., sensor pixels) into a corrected image. For example, as shown in  FIG.  10   , the inverted super-Gaussian profile gives the x 2  relative position factor for a given normalized horizontal axis position factor x 1 . The normalized, offset, inverted super-Gaussian factor profiles are shown for a HFOV of 136° at various zoom amplitude values A o . 
       FIG.  11    shows example profile and x 1 →x 2  mappings for a super-Gaussian profile with A o =0.8. Graph  1100  shows the projection surface Z profile versus pixel position x 1 . As the zoom amplitude value A o =0.8, the normalized depth profile extends 0.8× past the nominal radius distance (i.e., R o =1). Graph  1102  shows the x 2  position factor c(x 1 ) versus the x 1  pixel position. In this example, the x 2  position factor is smallest for pixel positions x 1  located near the center of the image. Graph  1104  shows the x 2  corrected position versus the x 1  pixel position. The wavy appearance of graph  1104  arises since the pixels at the center of the image (i.e., x 1 ˜2000) spread farther to their x 2  positions, hence, the smaller slope. The steeper slope near the edges indicate that the pixels become relatively closer. As a result, the zoom effect is greater for pixels near the center compared to pixels near an edge. Applying y 1 →y 2  mappings, the corrected y 2  positions are then determined through relation 
                 y   2     =       y   0     +       (         c   ⁡   (   x   )     ⁢     x   1       -     x   0       )     ⁢     (         y   1     -     y   0           x   1     -     x   0         )           ,         
where c(x) is a non-linear function, such that x 2 =c(x)x 1 .
 
       FIGS.  12 A- 12 H  shows example field angle maps of a raw input image and example rectilinear projections modified with a zoom amplitude function.  FIGS.  12 A- 12 B  are field angle maps of raw input images from image sensor or cropped regions of interest (ROI) from image sensor while  FIGS.  12 C- 12 H  are field angle maps of dewarped output frames which are distortion corrected from raw image input images in  FIGS.  12 A- 12 B . For the case of dewarp using zero amplitude (A o =0) and no tilt compensation (T=0°) the distortion correction is a rectilinear analytical projection, which is a direct projection, as shown in  FIG.  12 C . Adding tilt compensation, e.g.  FIG.  12 E , T=8.75°, the distortion correction becomes a tilted rectilinear analytical projection, also a direct projection. The dark regions  1202   a - f  above and below the field angle contoured regions of  FIGS.  12 C,  12 E,  12 G  represent angles imaged beyond the ROI input frame edges. To utilize such frames with a given aspect ratio, e.g. 16×9, the HFOV must be reduced to avoid clipping (substantially, as can be seen in  FIGS.  12 C,  12 E, and  12 G ). However, use of amplitude function (A o =0.8) as shown in  FIGS.  12 D,  12 F,  12 H  enables wider HFOV in the dewarped output frame, providing more efficient usage of the pixels. Efficient usage of pixels may help produce higher resolution output images when utilizing wide FOV camera lens optics, e.g. f-theta lens. 
     Further, in some examples, combinations of tilt compensation with offset imaging may provide efficient use of pixels from the input ROI to maximize and support wide HFOV, while also improving interaction with a large display for scenarios in which the camera is mounted high and above the large display.  FIG.  12 H  shows a field angle map where A o =0.8, T=8.75° and v off =120 pixels. Comparing  FIG.  12 H  (offset) to  FIG.  12 F  (no offset), it can be seen that including an offset may reduce or eliminate the size of the dark region outside the contour region. 
     In the illustrations in  FIGS.  12 A- 12 H , the radial contour lines represent clocking angles about the optical axis, e.g. −180° to +180°, while the curved contour lines show field angles. Solid lines are increments of 10° and dotted lines are 5° in between the 10° increment contours. While the input ROI has radial symmetry due to lens radial distortion, and on-axis rectilinear analytical projection also has radial symmetry, other dewarp cases exhibit non-rotationally symmetric distortion field angle maps, which enable correction based on scene geometry. 
       FIGS.  13 - 14    demonstrate the distortion correction and zoom effects that can be achieved using an analytical projection that has been modified with an amplitude function.  FIG.  13    depicts an example image  1300  of a conference room scene after applying a tilted rectilinear projection that has not been modified by an amplitude function. The conference scene was imaged using a camera comprising a HFOV of 136°, the camera tilted downwards towards the scene at a 17° tilt angle. The projection mapping applied to the raw image used a rectilinear projection comprising a tilt angle of 17° (i.e., full tilt correction). The rectilinear projection and tilted rectilinear projection preserve all straight-line objects as straight lines in the image. The rectilinear projection modified with amplitude function preserves all straight-line objects as straight lines in image for amplitude set to zero, and preserves all vertical straight line objects as straight lines in image for non-zero amplitudes. While standing persons and vertical line objects appear vertical, some subjects located at higher field angles may appear warped when a displayed frame is viewed at a distance such that the angular subtend of FOV as seen by viewer is not the same as the original camera angular subtend, which may appear as trapezoidal stretching in the case of an ideal rectilinear projection of a wide-angle scene. Display  1302  appears stretched due to trapezoidal distortion and the subjects on display  1302  also appear warped. Furthermore, the subjects on display  1304  appear very small compared to the subjects on display  1302 . The size disparities and trapezoidal distortion may make an unsatisfactory viewing experience. 
       FIG.  14    depicts a wide FOV image  1400  of a similar conference room scene after applying a rectilinear projection that has been modified by an amplitude function. In this example, a rectilinear projection modified by a super-Gaussian amplitude function was applied to the scene. The distortion correction provided by the modified projection mapping helps correct the warping and size disparities present in  FIG.  13   . The trapezoidal distortion of display  1302  is corrected. The subjects on display  1302  are no longer stretched and appear more natural. A higher zoom effect is achieved for pixels close to the center of the image compared to pixels near the edges, as evidenced by the relative size of the subjects in display  1304  compared to the size of subjects in display  1302 . Thus, the modified projection used to obtain image  1400  may achieve a more consistent sizing of subjects across a field of view. 
     In  FIG.  14   , the super-Gaussian amplitude function used to modify the rectilinear projection was obtained using the above equations and comprises zoom amplitude of A o =0.8. Additionally, the rectilinear projection comprises a partial tilt correction of 8.5°. Use of a partial tilt correction and/or a zoom amplitude values less than 1.0 may limit warping and foreshortening effects of subjects located close to the camera. For example, subject  1406  may appear compressed or warped if a zoom amplitude of 1.0 is used. Further, certain facial features may appear abnormally enlarged if a full tilt correction is used with the rectilinear projection, as both horizontal FOV and appearance of the apparent pose of subjects positioned at high field angles may be adjusted by zoom amplitude. On the other hand, a zoom amplitude of 1.0 may be appropriate to use in for a different camera FOV and arrangement of subjects, e.g., a radial scene where use of cylindrical projection is undesirable due to curved appearance of some straight-line objects in scene, such that use of amplitude-modified rectilinear projection provides improved appearance of straight-line objects in image. In other examples, any other suitable zoom amplitude and/or tilt parameter may be used. Further, zoom amplitude may be used to balance between a more radial appearance with angle, such as moving toward cylindrical projection appearance, and rectilinear appearance, which suffers from stretching at higher field angles. 
       FIG.  15    shows a flow diagram depicting an example method  1500  for correcting image distortion by mapping an image to an analytical projection that is modified by an amplitude function. For example, method  1500  may be performed by controller  116  of camera  100  shown in  FIG.  1   , or on a computing device that receive images data from camera  100 . At  1502 , method  1500  includes obtaining an image of a scene acquired via a camera. For example, the camera may comprise a wide-angle lens and may be tilted at a downward angle. In other examples, the camera may be substantially level and/or have a relatively narrower FOV. In some examples, at  1504 , method  1500  comprises cropping the image of the scene. 
     Method  1500  further comprises, at  1506 , applying a mapping to pixels of the image of the scene to map the image of the scene to an analytical projection that is modified by an amplitude function that achieves a higher zoom effect on pixels closer to a central mapping point of reference in the scene compared to pixels closer to an edge of the image, thereby obtaining a corrected image. In some examples, at  1508 , the mapping comprises a direct mapping. For example, the amplitude function may be used to modify the projection mapping surface to which the sensor pixels are projected. In other examples, at  1510 , the mapping comprises an indirect mapping. For example, the amplitude function may modify the analytical projection via a x 1 →x 2  mapping within the rectilinear projection plane. 
     Continuing with method  1500 , in some examples, at  1512 , the amplitude function comprises one of a super-Gaussian function or a conic function. For example, the conic function may comprise a cylindrical, parabolic, or elliptical profile. In some examples, at  1514 , the amplitude function comprises a zoom amplitude between 0.7 and 0.9. 
     Further, in some examples, at  1516 , the analytical projection comprises a tilt correction. In some such examples, the tilt correction is a partial tilt correction. For example, the tilt correction may comprise a tilt parameter that is 0.4 to 0.8 of the camera tilt angle, which may lessen foreshortening effects on some subjects of interest. Further, the tilt correction may be based upon a tilt sensor of the camera. In some examples, at  1518 , the analytical projection comprises one of a tilted rectilinear projection with indirect mapping using an amplitude modifying function including super-Gaussian or conic amplitude function, and an analytical projection with direct mapping including tilted super-Gaussian analytical projection or tilted conic analytical projection. In some examples, at  1519  the method comprises selecting an amplitude function based on a horizontal field of view. For example, in response to a change in the HFOV, an updated mapping may be selected and applied based on the updated amplitude function. Further, the updated mapping may include a relationship of parameters defining the amplitude function such that the dewarped appearance may be maintained for a change of HFOV, such that parameters forming the amplitude function are dependent on HFOV. In some examples, a reference HFOV, HFOV ref , may be used to maintain amplitude function with change in HFOV, by first forming a reference distance, 
                 z     p   ⁢   _   ⁢   ref       =       (     1     cos   ⁡   (   ϕ   )       )     ⁢     (       H     R   ⁢   e         tan   ⁡   (       (     π   180     )     ⁢     (       H   ⁢   F   ⁢   O   ⁢     V   ref       2     )       )       )         ,         
then forming a nonlinear scaling ratio,
 
               F   ⁢   O   ⁢     V   rat       =       (       H   ⁢   F   ⁢   O   ⁢   V       2   ⁢   H   ⁢   F   ⁢   O   ⁢     V   ref         )     +       (       H   ⁢   F   ⁢   O   ⁢     V   ref         2   ⁢   H   ⁢   F   ⁢   O   ⁢   V       )     ⁢       (       tan   ⁡   (       (     π   180     )     ⁢     (       H   ⁢   F   ⁢   O   ⁢   V     2     )       )       tan   ⁡   (       (     π   180     )     ⁢     (       H   ⁢   F   ⁢   O   ⁢     V   ref       2     )       )       )     .               
Now the function c(x) may be defined in terms including FOV rat  and x edge :
 
                 c   ⁡   (   x   )     =       1   /     z   ⁡   (       A   o     ,     T   fwhm     ,   m   ,     w   o     ,     FOV   rat     ,     x   edge     ,     v   o     ,     v   sag     ,   x     )       =     1   /     (     1   +         A   o     (         A   o     ⁢     e       -     (     1   2     )       ⁢       (         Abs   (         FOV   rat     ⁢   x   ⁢     w   o         x   edge       )     ⁢     (     2   ⁢       Log   (   2   )       (     1   /   2   ⁢   m     )         )         T   fwhm       )       2   ⁢   m             -     v   o       )     /     v   sag         )           ,         
and the mapping becomes x 2 =c(x)x 1 . In some examples, the analytical projection comprises offset imaging. In some examples, the analytical projection comprises a combination of offset imaging and tilt compensation.
 
     Continuing, at  1520  the method comprises outputting the corrected image. In some examples, at  1522 , method  1500  comprises receiving an input of a zoom, pixel offset, and/or tilt angle compensation request, and in response applying an updated mapping to the analytical projection. In some examples, the tilt compensation may be determined by the camera pointing angle, or fraction thereof, as measured by and received from an inertial measurement unit (IMU) onboard the camera. In some examples, HFOV and zoom amplitude and vertical pixel offset and tilt compensation angle used in mesh generation may be determined by camera tilt angle, thus enabling ability to maximize HFOV for a given set of mesh generation input parameters. In some examples, a predetermined look-up table (LUT) of correction mesh arrays may be stored onboard camera such that a mesh is selected based on input camera IMU tilt angle, wherein the correction meshes may be generated based on specific combinations of parameters including HFOV, zoom amplitude A o , vertical pixel offset v off , and tilt compensation angle. In some examples, tilt compensation angle may be pre-determined by mechanical mounting of the camera device to an interactive display device, and/or known parameters concerning a specific mount or mount location between camera device and interactive display device. In some examples, at  1524 , the corrected image is cropped prior to output. Further, in some examples, the corrected image is output to a remote computing device at  1528 . For example, the corrected image may be output to a videoconferencing system of a remote user, or to a distributed computing system hosting a videoconferencing service. In other examples, at  1530 , the corrected image is output to a local display. 
     In some examples, the methods and processes described herein may be tied to a computing system of one or more computing devices. In particular, such methods and processes may be implemented as a computer-application program or service, an application-programming interface (API), a library, and/or other computer-program product. 
       FIG.  16    schematically shows a block diagram of an example computing system  1600  that can enact one or more of the methods and processes described above. Computing system  1600  is shown in simplified form. Computing system  1600  may take the form of one or more personal computers, server computers, tablet computers, home-entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smart phone), and/or other computing devices. For example, computing system  1600  may represent controller  116 , camera  100 , a computing device incorporating camera  100  (e.g., a videoconferencing system having an integrated camera), or any other suitable computing system. 
     Computing system  1600  includes a logic subsystem  1602  and a storage subsystem  1604 . Computing system  1600  may optionally include a display subsystem  1606 , input subsystem  1608 , communication subsystem  1610 , and/or other components not shown in  FIG.  16   . 
     Logic subsystem  1602  includes one or more physical devices configured to execute instructions. For example, the logic subsystem may be configured to execute instructions that are part of one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions may be implemented to perform a task, implement a data type, transform the state of one or more components, achieve a technical effect, or otherwise arrive at a desired result. 
     The logic subsystem may include one or more processors configured to execute software instructions. Additionally or alternatively, the logic subsystem may include one or more hardware or firmware logic subsystems configured to execute hardware or firmware instructions. Processors of the logic subsystem may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel, and/or distributed processing. Individual components of the logic subsystem optionally may be distributed among two or more separate devices, which may be remotely located and/or configured for coordinated processing. Aspects of the logic subsystem may be virtualized and executed by remotely accessible, networked computing devices configured in a cloud-computing configuration. 
     Storage subsystem  1604  includes one or more physical devices configured to hold instructions executable by the logic subsystem to implement the methods and processes described herein. When such methods and processes are implemented, the state of storage subsystem  1604  may be transformed—e.g., to hold different data. For example, storage subsystem  1604  may store instruction executable to apply an analytical projection that has been modified by an amplitude function, as described above. Further, storage subsystem  1604  may store applications executable by logic subsystem  1602  such as a videoconferencing application  1612  that receives corrected images for transmission and/or display. 
     Storage subsystem  1604  may include removable and/or built-in devices. Storage subsystem  1604  may include optical memory (e.g., CD, DVD, HD-DVD, Blu-Ray Disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM, etc.), and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive, tape drive, MRAM, etc.), among others. Storage subsystem  1604  may include volatile, nonvolatile, dynamic, static, read/write, read-only, random-access, sequential-access, location-addressable, file-addressable, and/or content-addressable devices. 
     It will be appreciated that storage subsystem  1604  includes one or more physical devices. However, aspects of the instructions described herein alternatively may be propagated by a communication medium (e.g., an electromagnetic signal, an optical signal, etc.) that is not held by a physical device for a finite duration. 
     Aspects of logic subsystem  1602  and storage subsystem  1604  may be integrated together into one or more hardware-logic components. Such hardware-logic components may include field-programmable gate arrays (FPGAs), program- and application-specific integrated circuits (PASIC/ASICs), program- and application-specific standard products (PSSP/ASSPs), system-on-a-chip (SOC), and complex programmable logic devices (CPLDs), for example. 
     The terms “module,” “program,” and “engine” may be used to describe an aspect of computing system  1600  implemented to perform a particular function. In some cases, a module, program, or engine may be instantiated via logic subsystem  1602  executing instructions held by storage subsystem  1604 . It will be understood that different modules, programs, and/or engines may be instantiated from the same application, service, code block, object, library, routine, API, function, etc. Likewise, the same module, program, and/or engine may be instantiated by different applications, services, code blocks, objects, routines, APIs, functions, etc. The terms “module,” “program,” and “engine” may encompass individual or groups of executable files, data files, libraries, drivers, scripts, database records, etc. 
     It will be appreciated that a “service”, as used herein, is an application program executable across multiple user sessions. A service may be available to one or more system components, programs, and/or other services. In some implementations, a service may run on one or more server-computing devices. 
     When included, display subsystem  1606  may be used to present a visual representation of data held by storage subsystem  1604 . This visual representation may take the form of a graphical user interface (GUI). As the herein described methods and processes change the data held by the storage subsystem, and thus transform the state of the storage subsystem, the state of display subsystem  1606  may likewise be transformed to visually represent changes in the underlying data. Display subsystem  1606  may include one or more display devices utilizing virtually any type of technology. Such display devices may be combined with logic subsystem  1602  and/or storage subsystem  1604  in a shared enclosure, or such display devices may be peripheral display devices. 
     When included, input subsystem  1608  may comprise or interface with one or more user-input devices such as a keyboard, mouse, touch screen, or game controller. In some embodiments, input subsystem  1608  may comprise an image sensor such as camera  1614  from which images are obtained and corrected via an applied projection mapping, as described above. For example, camera  1614  may represent camera  100  and may comprise a wide-angle lens. In some embodiments, the input subsystem may comprise or interface with selected natural user input (NUI) componentry. Such componentry may be integrated or peripheral, and the transduction and/or processing of input actions may be handled on- or off-board. Example NUI componentry may include a microphone for speech and/or voice recognition; an infrared, color, stereoscopic, and/or depth camera for machine vision and/or gesture recognition; a head tracker, eye tracker, accelerometer, and/or gyroscope for motion detection and/or intent recognition; as well as electric-field sensing componentry for assessing brain activity. 
     When included, communication subsystem  1610  may be configured to communicatively couple computing system  1600  with one or more other computing devices. Communication subsystem  1610  may include wired and/or wireless communication devices compatible with one or more different communication protocols. As non-limiting examples, the communication subsystem may be configured for communication via a wireless telephone network, or a wired or wireless local- or wide-area network. In some embodiments, the communication subsystem may allow computing system  1600  to send and/or receive messages to and/or from other devices via a network such as the Internet. 
     Another example provides a computing device comprising a logic subsystem, and a storage subsystem holding instructions executable by the logic subsystem to receive an image of a scene as acquired by an image sensor, apply a mapping to the image of the scene that maps pixels of the image to projected pixels on an analytical projection that is modified by an amplitude function such that the analytical projection achieves a higher zoom effect on pixels closer to a center of the image compared to pixels closer to an edge of the image, thereby obtaining a corrected image, and output the corrected image. In some such examples, the analytical projection comprises a rectilinear projection. Additionally or alternatively, in some examples the amplitude function comprises one of a super-Gaussian function, a conic function, a parabolic function, or a cylindrical function. Additionally or alternatively, in some examples, the amplitude function comprises a super-Gaussian function comprising an amplitude scaling factor between 0.7 and 0.9. Additionally or alternatively, in some examples the mapping comprises a direct mapping. Additionally or alternatively, in some examples the mapping comprises an indirect mapping. Additionally or alternatively, in some examples the instructions are further executable to receive one or more of an input of a zoom request, an input of a pixel offset request, or an input of a tilt angle compensation request, and in response apply an updated mapping to the analytical projection. Additionally or alternatively, in some examples the computing device further comprises the image sensor. Additionally or alternatively, in some examples the image of the scene comprises a horizontal field of view between 700 and 145°. Additionally or alternatively, in some examples the analytical projection is modified by applying a nonlinear function c(x) to a grid that maps (x 1 , y 1 ) to (x 2 , y 2 ) by 
               (       x   2     ,     y   2       )     =     (         c   ⁡   (   x   )     ⁢     x   1       ,       y   0     +       (         c   ⁡   (   x   )     ⁢     x   1       -     x   0       )     ⁢     (         y   1     -     y   0           x   1     -     x   0         )           )           
wherein (x 1 , y 1 ) is a pixel position in the analytical projection, (x 2 , y 2 ) is a pixel position in the modified projection, (x 0 , y 0 ) is a central mapping point of reference in the image of the scene, and c(x) comprises the amplitude function.
 
     Another example provides a method enacted on a computing device, the method comprising obtaining an image of a scene acquired via a camera, applying a mapping to pixels of the image of the scene to map the image of the scene to an analytical projection that is modified by an amplitude function that achieves a higher zoom effect on pixels closer to a central mapping point of reference in the scene compared to pixels closer to an edge of the image, thereby obtaining a corrected image, and outputting the corrected image. In some such examples, the mapping comprises a direct mapping. Additionally or alternatively, in some examples the mapping comprises an indirect mapping. Additionally or alternatively, in some examples the method further comprises receiving an input of one or more of a zoom request, a pixel offset request, or a tilt angle compensation request, and in response applying an updated mapping to the analytical projection. Additionally or alternatively, in some examples the amplitude function comprises one of a super-Gaussian function or a conic function. Additionally or alternatively, in some examples the analytical projection comprises a tilt correction that is based upon a camera tilt angle at which the image of the scene was acquired. Additionally or alternatively, in some examples the method further comprises selecting an updated amplitude function based on a change in a horizontal field of view, and applying an updated mapping based on the updated amplitude function. 
     Another example provides a videoconferencing system comprising a camera, a logic subsystem, and a storage subsystem holding instructions executable by the logic subsystem to receive an image of a scene from the camera, apply a mapping to the image to map pixels of the image to projected pixels on a rectilinear projection that is modified by an amplitude function such that the rectilinear projection achieves a higher zoom effect on pixels closer to a center of the image compared to pixels closer to an edge of the image, thereby obtaining a corrected image, and output the corrected image for display. In some such examples the videoconferencing system further comprises a display. Additionally or alternatively, in some examples the rectilinear projection is based at least in part on a tilt angle of the camera relative to the scene. 
     It will be understood that the configurations and/or approaches described herein are exemplary in nature, and that these specific embodiments or examples are not to be considered in a limiting sense, because numerous variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. As such, various acts illustrated and/or described may be performed in the sequence illustrated and/or described, in other sequences, in parallel, or omitted. Likewise, the order of the above-described processes may be changed. 
     The subject matter of the present disclosure includes all novel and non-obvious combinations and sub-combinations of the various processes, systems and configurations, and other features, functions, acts, and/or properties disclosed herein, as well as any and all equivalents thereof.