Minimizing dead zones in panoramic images

Techniques to minimize problems with dead zones associated with panoramic cameras are described. A dead zone is an area about a zero degree boundary where a three hundred and sixty degree panoramic image is split so that a two-dimensional panorama can be rendered. Redirection of the zero degree boundary to a position where no object appears is described that prevents an object located in the dead zone from being split between margins of the panorama. Another technique involves reproducing one or more portions of the image that appears at one margin adjacent to an opposite margin so that an object in the dead zone that would normally be split is reproduced in whole. The described techniques may be implemented in a panoramic camera endpoint device or in a client device remote from a camera. The techniques may be applied to live video or to previously recorded video.

TECHNICAL FIELD

The following description relates generally to image processing. More particularly, the following description relates to panoramic camera systems.

BACKGROUND

Panoramic images are wide-angle camera images that span up to three hundred and sixty degrees (360°). Panoramic video camera devices are especially useful in a conference room scenario. A single panoramic video camera can capture conference participants over a wide span of the conference room so that a viewer can see most or all of the conference participants simultaneously. A panoramic video camera device that can capture a three hundred and sixty degree (360°) view of the conference room can image all conference participants.

But a two-dimensional three hundred and sixty degree (360°) panoramic image has a left margin and a right margin at that represent a 0°/360° boundary of an imaged area. If a person happens to be seated at this boundary, then a portion of an image of that person will appear at the left margin of the panoramic image and a portion of that person will appear at the right margin of the panoramic image. A small area about the boundary is referred to as a “dead zone” since any object in that area will be split in a resultant panoramic image.

DETAILED DESCRIPTION

Cameras and techniques are described herein for providing a three hundred and sixty degree (360°) panoramic image without splitting an object image at a left/right margin boundary of the panoramic image, i.e. without “dead zones.” As used herein, the term “dead zone” refers to an area in a space imaged by a three hundred and sixty degree (360°) camera where a resultant panoramic image is divided for presentation in a two-dimensional format. Typically, an object located along a boundary where such a division is made will appear in two parts—one portion of the object will appear at a left margin of the panoramic image and one portion of the object will appear at a right margin of the panoramic image.

Although the present description refers to examples of panoramic cameras that image a three hundred and sixty degree (360°) area, it is noted that one or more of the described techniques may be used in cameras that image less than a three hundred and sixty degree (360°) area. As long as an object located in a “dead zone” as described here can be split on a resultant image, the present techniques may be applied.

FIGS. 1aand1billustrate a dead zone and a problem typically encountered when an imaged object (i.e. a person) is located in the dead zone.FIG. 1ais a diagram depicting an exemplary conference room configuration100. The configuration shown includes a conference room table102, several conference room chairs104and a panoramic camera106situated in the center of the top of the table102.

A zero degree (0°) boundary108is shown emanating from the panoramic camera106. The zero degree (0°) boundary108indicates a point at which the imaged area is divided so that the imaged area can be displayed in a two-dimensional format. A dead zone110is shown about the zero degree (0°) boundary108. An object located within the dead zone110may be divided into two partial images upon rendering the panoramic image. For example, a person seated in conference room chair112would likely appear in two portions. One side of the person's face would appear adjacent to a left margin of a resultant panoramic image, and the other side of the person's face would appear adjacent to a right margin of the resultant panoramic image.

FIG. 1bis an exemplary panoramic image120where a person is seated in a dead zone. The exemplary panoramic image120has a left margin122and a right margin124. An image of a face of a person (situated as if they were seated in conference room chair112) is shown in a first half126located adjacent to the left margin122and a second half128located adjacent to the right margin124. This is an example of how a face of a person seated in the dead zone110appears in a resultant panoramic image.

Several techniques are discussed herein for minimizing dead zone problems in three hundred and sixty degree (360°) panoramic images. One technique involves physically rotating a camera or a portion of a camera so that a zero degree (0°) boundary (a location where a resultant image is split to render a two-dimensional format) is oriented toward an area that contains no object or an insignificant object.

Another way to minimize dead zone problems is to make a similar adjustment by way of computer instruction translation in software, hardware, firmware, or the like. A panoramic camera described herein utilizes several individual cameras to image a three hundred and sixty degree (360°) span (although the described techniques may be applied to a single panoramic camera spanning up to three hundred and sixty degrees (360°)). A pixel remapping function is used to remap pixels in each individual image to a panoramic image. If an object to be imaged is located in a dead zone, the remapping function can be altered (i.e. the panoramic image is rotated) so that no object appears in a dead zone.

In one or more of the described techniques, an optimum orientation of a zero degree (0°) boundary may be determined manually or automatically. In an automatic determination of the zero degree boundary (0°), a face tracking mechanism is used to determine an area in which no person's face appears. The zero degree (0°) boundary is then set to that area. As a result, no person is imaged in the dead zone that surrounds that boundary.

An additional technique that is shown and described herein determines a portion of an image that is adjacent to a zero degree (0°) boundary (i.e. a portion of an image that will be on a margin of a resultant panoramic image) and reproduces that portion of the image adjacent to the opposite margin. As a result, if a person in a dead zone is imaged, then instead of that person's image being split on the margins, a full image of that person appears adjacent to one of the margins.

All of these techniques are shown and described in greater detail below, with reference to the figures included herewith. It is noted that the specific implementation shown and described herein are shown by way of example and not by way of limitation. Other systems and methods may be implemented within the spirit and scope of the description provided herein and the claims appended hereto.

Exemplary Photographic Device

FIG. 2is a diagram of an exemplary panoramic photographic device200in accordance with the present description. Although a particular configuration is shown for the exemplary panoramic photographic device200, it is noted that the particular configuration is shown by way of example only and that other configurations of panoramic camera devices may be utilized in accordance with the present description.

The exemplary panoramic photographic device200includes an inverted pyramidal mirror assembly202and multiple cameras206. The inverted pyramidal mirror assembly202includes multiple mirror facets204, there being one mirror facet204corresponding to each of the multiple cameras206. The inverted pyramidal mirror202is positioned over the multiple cameras206by a column208.

The configuration of the exemplary panoramic photographic device200achieves a near center of projection that reduces parallax errors in a panoramic image formed by the panoramic photographic device200. Such a configuration is shown and described in one or more patent applications incorporated by reference herein.

Exemplary Camera Configuration

FIG. 3is a diagram of an exemplary panoramic camera300in a 360° camera configuration. The diagram is a rudimentary representation of the panoramic photographic device200shown inFIG. 2. It is noted that although an example of a multi-camera 360° panoramic photographic device is shown and discussed herein, the present description applies to any wide-angle photographic device which is prone to splitting images of objects that are located in an area where the image is split for two-dimensional representation. In the following discussion, continuing reference is made to elements and reference numerals shown and described inFIG. 2.

The diagram shows multiple cameras302-310arranged in a pentagonal configuration to form the panoramic camera300, corresponding with the pentagonal shape of the inverted pyramidal mirror202ofFIG. 2. Each camera302-310photographs a different area of the environment of the panoramic camera300.

Since the imaged area is a complete three hundred and sixty degrees (360°) and a resultant image must be rendered in a two-dimensional format, the imaged area must be divided at some point so that the resultant panoramic image can be “unrolled”. It is this division that can cause problems in the resultant image since an object located along such a division will be rendered along both side margins of the resultant image.

Exemplary Image Assembly Module

FIG. 4is a diagram of an exemplary image assembly module400that assembles individual images10-50to form a panoramic image402. The image assembly module400receives as input one image from each camera in the panoramic camera. For purposes of the following discussion, images taken by the cameras302-310shown inFIG. 3are identified as image “1”10, image “2”20, image “3”30, image “4”40and image “5”50, respectively. These individual images are assembled to form the single panoramic image402.

The image assembly module400utilizes a remapping function that maps pixels in individual image space to pixels in a panoramic image space. Pixels in individual images10-50are addressed according to a (u, v) coordinate grid401. Pixels in the panoramic image402are addressed according to an (x, y) coordinate grid403. The image assembly module400maps each pixel from a (u, v) coordinate to an (x, y) coordinate location until all pixels in individual images10-50are reproduced in the panoramic image402.

The remapping function is calibrated so that the individual images are stitched together seamlessly where the images are joined together. This is so that the panoramic image402appears seamless, i.e. no distracting artifacts occur between individual images.

It is noted that not all pixels in the individual images are necessarily used in the resultant panoramic image due to methods required to properly stitch the individual images so that seams between images are minimized in the panoramic image. In addition, one or more distortion correction algorithms may be applied to normalize a view of the panoramic image. Image stitching techniques are shown and described in one or more of the patent application incorporated herein by reference.

One technique that can be used to correct dead zone problems is to reproduce a portion of an image adjacent to one side margin and append the reproduced portion to the other side margin. InFIG. 4a, a delta portion δ of individual image is reproduced from one side of the panoramic image402and is appended (delta prime portion δ′) to an opposite side of the panoramic image to create a 360+δ panoramic image.

A width of δ′ can vary but in a video conferencing situation similar to that discussed here, δ′ is approximately the width of a face of a person. In a typical conference room panorama, the width of a person's face is typically about four (4) to fourteen (14) degrees, depending on how far the person is from the camera.

The width of δ can be determined by default, automatically or manually. When the width is determined manually, a user interacts with a user interface to set a desired width of δ. When the width is automatically determined (such as in combination with a face tracker), the automatic setting should be configured to occur at certain intervals. Meeting situations can change, with people coming and going at various times. If the width of width of δ is automatically adjusted whenever a person enters or leaves the dead zone, undesirable results may occur. In addition, an automatic implementation may include a user option to disable the automatic determination of width of δ at any time.

This technique is described in greater detail below, with respect to one or more subsequent figures. This technique may be used with or without first orienting the camera to a desired position as described below.

When the delta prime portion δ′ is appended to the panoramic image402, a width (t) of the panoramic image is increased to (t+δ′). It may be desirable to resize the panoramic image402so that an aspect ratio of a typical panoramic image is maintained. Other methods may be utilized to resize the panoramic image. For example, to maintain the same size and image aspect ratio, the resultant panoramic image402(t+δ′) could be padded with a horizontal strip appended to the top and/or bottom of the panoramic image402.

FIG. 4bdepicts a panoramic image410similar to the panoramic image402shown inFIG. 4a. The panoramic image410includes two (2) delta portions (δ), one on each end of the panoramic image410. The process for creating the panoramic image410is similar to the process described with respect toFIG. 4ain that an area greater than three hundred and sixty degree (360°) is reproduced.

Samples imaged by a panoramic camera that are adjacent to each side of a zero degree (0°) boundary are mapped to two locations—a first location in nominal panoramic space, and a second location on an opposite side of the panoramic image410.

Appending two delta portions (δ) to a panoramic image will result in an image of a person sitting in the dead zone to appear twice in the resultant panoramic image410—once near each margin. However, as will be discussed below, appending a reproduced portion of an image to the image may be combined with one or more other techniques to avoid duplication of a person's image in the panoramic image410.

The portion of an image that is reproduced (δ) can be determined at any stage during the image assembly process. In one implementation, the panoramic image402is created before δ is copied and δ′ is appended to the panoramic image402. Other implementations may include remapping tables configured to duplicate pixels in this manner during the remapping process. Any such technique may be utilized to accomplish the same end result described above.

FIG. 5adepicts an exemplary conference room setting500of table502, chairs504and panoramic camera506. In the example shown inFIG. 5a, a zero degree (0°) boundary508of the panoramic camera506is manually oriented toward a position of the table502where no person is seated. The location of the boundary508is indicated on the camera506by a boundary indicator509that is visible to a user. To orient the panoramic camera506, a user simply rotates the panoramic camera506to a point where the boundary indicator509is oriented toward a location in which a subject does not appear, as shown.

As a result, no person appears in a dead zone510that occurs about the zero degree (0°) boundary508. Therefore, each person in the conference room is fully imaged on a panoramic image captured by the panoramic camera506.

FIG. 5bis a representation of an exemplary panoramic image520of meeting participants522situated when the panoramic camera506is oriented to a location where no person is located. Each image522of a person is shown in full, i.e. no image of a person is split between margins of the panoramic image520.

The situation shown inFIG. 5aandFIG. 5bis frequently available under circumstances where there is an optimum location for the dead zone, such as in rooms where a projector screen is in use. Since nobody usually sits between the conference table and the projector screen, an optimum location to orient the zero degree (0°) boundary is toward the projector screen. However, not all situations are so optimally presented.

Computational Rotation to Establish Zero Degree Boundary

In addition to the technique described above, wherein the camera is physically re-oriented to direct the location of the zero degree (0°) boundary, the zero degree (0°) boundary may be manually indicated by a user and then the panoramic image is computationally rotated to orient the image with respect to the zero degree (0°) boundary.

To accomplish such rotation of the panoramic image, the remapping function (previously described with respect to the image assembly module400ofFIG. 4) computes the angle difference between a nominal zero degree (0°) boundary and the selected zero degree (0°) boundary. This difference is factored into the remapping function to rotate the panoramic image to a position wherein the margins of the panoramic image correspond to the selected zero degree (0°) boundary.

Rotation is accomplished by adding an offset equaling the difference between the nominal zero degree (0°) boundary and the selected zero degree (0°) boundary to the x coordinate in the (x, y) coordinate system of the panoramic image (403,FIG. 4a).

Setting the zero degree boundary (0°) can be accomplished in one of several ways. For example, a user interface unit (not shown) of a panoramic camera (not shown) may contain means for effecting a rotation of a boundary indicator, such as arrow buttons. A user would position the boundary indicator (e.g. on a liquid crystal display (LCD)) by pressing right or left arrow buttons. Other techniques may be used to accomplish the same result.

Automatic Location of Zero Degree (0°) Boundary

FIG. 6depicts an exemplary conference room configuration700similar to that shown inFIG. 5a. The exemplary conference room configuration600includes a conference room table602, several conference room chairs604and a panoramic camera606. In the present example, there is a chair604present along both sides and at both ends of the conference room table602. Therefore, there is no clear position in which to orient the camera606.

InFIG. 6, the panoramic camera606is shown with a zero degree (0°) boundary oriented toward a corner of the table602, between two chairs604. A dead zone610associated with the zero degree (0°) boundary608falls in between two conference participants so as to avoid splitting an image of a participant.

The zero degree (0°) boundary608may be manually oriented as previously described, or it may be automatically oriented toward an optimal location using a face tracking mechanism. Face tracking mechanisms, or “face trackers,” are known in the art and can be used to determine a best location for the zero degree (0°) boundary608.

Typically, an optimum location as determined using a face tracker is a center of the largest detected angle between faces (or other objects). A face tracking mechanism locates each face in the imaged area and determines an angle between each two adjacent faces. The largest of these angles is used to orient the zero degree (0°) boundary608.

A representation of the boundary selection mechanism described above is:

When the optimum boundary has been determined, the remapping function (see discussion ofFIG. 4aabove) is adjusted to rotate the panoramic image so that the desired boundary is achieved.

This and other previously described techniques for adjusting the zero degree (0°) boundary are described in greater detail below, with respect to subsequent figures.

Exemplary Camera

FIG. 7is a block diagram of an exemplary panoramic camera700in accordance with the implementations described herein. In the following example, the exemplary panoramic camera is more or less a complete video conferencing endpoint that includes a panorama camera as well as other functionality that could be separated into one or more other devices in one or more alternative implementations. One such alternative implementation is shown and described inFIG. 8below. Other specific configurations may be implemented in accordance with the present description and claims.

In the following discussion, continuing reference is made to elements and reference numerals shown and described in previous figures. Although specific elements are shown and particular functionality is attributed to specific elements, it is noted that the described functions may be allocated to alternate elements in alternative configurations. Furthermore, although elements are shown as hardware or software elements, any element may be configured in hardware, software or firmware.

The exemplary panoramic camera700includes a processor702, memory704and at least one camera706. In the previous examples, there are five (5) cameras706included in one panoramic camera. The panoramic camera700also includes one or more sensors708typically found in cameras (e.g. charged coupled device (CCD), Complementary Metal-Oxide Semiconductor (CMOS), etc.) for sensing light and transmitting electronic information. Typically, there is one sensor708for each camera706.

The panoramic camera700may also include a microphone710(or microphone array) and/or one or more speakers712. In a video conference camera, for example, a microphone and speaker are necessary to conduct audio communications between remote conference participants.

An input/output (I/O) module714is included that consists of one or more units configured to receive external data and/or to transmit data. One example of a common I/O module is a Universal Serial Bus (USB) or other type of port, although the I/O module714is not limited thereto. The I/O module714may also include a unit configured to communicate with a panoramic camera/client device in a remote location. This includes, but is not limited to, network interface modules, telephone interface modules and the like.

A user interface716is also provided with the panoramic camera700and includes controls and displays (not shown) by which a user can interface with the panoramic camera700. Such controls may include buttons, touch pads, touch screens, pointing devices, and/or the like. One type of display that may be used includes, but is not limited to, a liquid crystal display (LCD) screen or the like.

A boundary indicator718may also be included in the panoramic camera700. Various types of boundary indicators718may be used to identify a nominal orientation of a zero degree (0°) boundary within the camera, i.e. an axis along which an image taken by the camera700is divided to render a two-dimensional image. As previously shown and described, a boundary indicator (509,FIG. 5) may be a marking on an external portion of the camera, such as an arrow, a dot, a line, etc. Any other type of indicator may be used (such as a light emitting diode (LED)) as long as a user can infer a direction of a zero degree (0°) boundary from the location of the indicator.

A power module720supplies electrical power to the panoramic camera700and the panoramic camera700includes other miscellaneous hardware722elements that may be required to carry out typical functionality of such a camera.

The memory704stores an operating system724that includes processor-executable instructions for carrying out operational functionality for the panoramic camera700and its components. Multiple images726detected by the cameras706are stored in the memory704.

One or more remapping tables728are also stored in the memory704and are utilized by a remapping module730to determine a correct mapping of individual images726from image space into a panoramic space to create a panoramic image732. Details of one or more remapping techniques are described in U.S. patent application Ser. No. 10/177,315 entitled “A System and Method for Camera Color Calibration and Image Stitching”, filed Jun. 21, 2002 assigned to the assignee of the present application. Said application has been incorporated by reference above.

A face tracker734is stored in the memory704and is used to automatically detect faces in, for example, a meeting situation. Any face tracking mechanism known in the art may be incorporated into the panoramic camera700for the purposes described herein. The face tracker734, in addition to locating faces, is also configured to compute an angle or distance between two adjacent faces.

An example of one or more face tracking mechanisms is described in greater detail in U.S. patent application Ser. No. 10/006,927 entitled “Automatic Detection and Tracking of Multiple Individuals Using Multiple Cues,” by Yong Rui and Yunqiang Chen, filed Dec. 3, 2001 and assigned to Microsoft Corp. Said application is incorporated herein by reference for all that is discloses and teaches.

An assembly module736is stored in the memory704and is configured to work in conjunction with the remapping module730and the remapping tables728to assemble the individual images726into the panoramic image732. One function that may be performed by the assembly module736is to determine a portion (δ) of an image726adjacent to one side of a zero degree (0°) boundary that can be reproduced adjacent to the other side of the zero degree (0°) boundary (as δ′) to minimize problems with a dead zone around the zero degree (0°) boundary.

Such functionality has been previously described and may or may not be included in a particular configuration of a panoramic camera700in accordance with the present description. Further discussion of the assembly module736and functionality thereof is discussed in greater detail below.

The memory704also stores a rotation module738. The rotation module738is configured to determine a distance between a default, or nominal, zero degree (0°) boundary of the panoramic camera700to an actual zero degree (0°) boundary (set automatically or manually by a user). The rotation module738determines an appropriate offset (x-axis offset) that should be included in a remapping function. By offsetting the zero degree (0°) boundary, a potential problems associated with a dead zone can be minimized as previously described.

A boundary module740is also included in the memory704of the panoramic camera700. The boundary module740handles functions related to determination of a nominal boundary742, an actual boundary744and a boundary difference746. The nominal boundary742will typically be a default boundary position set during the manufacturing process, either by storing an indication of the boundary in memory or in hardware. The nominal boundary742may then be retrieved and utilized in subsequent computations.

The actual boundary744is determined either from user input or from automatic calculations that identify an optimum location for a zero degree (0°) boundary. User input may be retrieved from the user interface716or from a discrete hardware element (such as a manual dial) that a user sets to point in a direction of a desired boundary.

Alternatively, the face tracker734may be used to determine locations of faces of meeting participants. The face tracker734(or some other module such as the boundary module740) may then use the face location information to determine an optimum location for a zero degree (0°) boundary, i.e. a location having a greatest distance between two faces. The boundary difference746is then determined in degrees, offset units or the like so that a remapping function can be configured to implement the appropriate offset.

Exemplary Client Device

FIG. 8is a block diagram of an exemplary client device800in accordance with the present description. The exemplary client device800may be a general computing device, such as a personal computer, or it may be a limited function device configured specifically for the purposes outlined herein. In the following discussion, continuing reference is made to previous figures and to elements and reference numerals associated therewith.

The client device800is configured to communicate with a panoramic camera such as the exemplary panoramic camera700shown inFIG. 7. It is noted, however, that the exemplary panoramic camera700may have fewer components when used in conjunction with the client device800, which could handle some of the process attributed to the exemplary panoramic camera inFIG. 7.

The client device800and panoramic camera700may also be configured to communicate with another client device (not shown) and/or panoramic camera in a remote location. This could be a situation in which two conference rooms, each with a client device and a remote panoramic camera, communicate with each over a communication line, such as a telephone line or a network connection.

The client device800includes a processor820, an I/O module822, a display824, a renderer825and a user interface826. The display824may be integrated with the client device800as shown or it may be separate from the client device800, such as in a configuration wherein the client device800is a personal computer and the display824is a monitor. The renderer825is a module—such as a video card—configured to render pixel representations in the panoramic image838on the display824.

The user interface826includes an image shifter827that allows a client device user to input a direction and magnitude of a rotation that the user wants to see in a viewed image. This feature is described in greater detail below.

The client device800also includes memory828which stores an operating system830and an RTP (Real-time Transport Protocol) Source module832. The RTP Source module832is configured to receive the panoramic image732from the panoramic camera700over a communication channel (not shown) and store the received panoramic image as panoramic image838. In this particular example, the panoramic camera700creates the panoramic image732from individual images and transmits the panoramic image to the client device800.

Various other elements840-852similar to and corresponding respectively with elements734-746ofFIG. 7. Said elements function as previously described except as indicated below. The panoramic image838is stored and can be manipulated when the panoramic image838is being replayed to minimize any deleterious effects related to a dead zone. The process described below can be applied to the stored panoramic image838whether replayed in real time as the image is received from the panoramic camera700or at a later time than the panoramic image838is received and stored.

In operation, a user may be situated in a first conference room with the client device800while the panoramic camera700is in a second conference room802. In this first example, the panoramic camera700is streaming video images to the client device800. The panoramic camera700has set an initial zero degree boundary and has created the panoramic image, which it then transmits to the client device800.

After the panoramic image838has been received by the client device800, the panoramic image838may be rotated if there is a problem with a dead zone, i.e. if a person's face is split in the panoramic image838. This may be accomplished automatically using the face tracker830or manually using the image shifter827.

One way that the rotation may be accomplished manually is by a user clicking on a desired zero degree boundary with a pointing device (not shown) and performing a rotation function by selecting from a menu, dragging a cursor, etc. Other techniques for rotating the image manually are available and any technique known in the art for doing so may be implemented in the context of the present example.

In at least one implementation, the rotation may be accomplished by a face tracker but may then be manually overridden by a user if the user so desires. Such an implementation is described below, with respect toFIG. 11.

In the context of the present example, the image rotation module844performs an image transformation function that is known in the art. When a new zero degree boundary is set, the image is divided vertically at the indicated point of the new zero degree boundary, creating two rectangular areas in the image. The two rectangular areas are then swapped so that a line dividing the two rectangles then becomes the margins for the new panoramic image.

It is also noted that the rotation function may be performed in the renderer825if the renderer825is available and is configured to perform such a function. For example, some video graphics accelerators include sufficient processing power to handle such a function and it may be more efficient to perform the rotation function in such a unit.

When the image shifter827is manipulated to manually reset the zero degree boundary, the image rotation module844receives feedback that indicates a direction and magnitude of the indicated rotation. The boundary module846in the client device800stores the nominal boundary848that is either received from the panoramic camera700or set by the face tracker840. The actual boundary850is the location that is identified from the detected manipulation. The difference (boundary difference852) is the difference between the nominal boundary848and the actual boundary850. The boundary difference852is used to compute an offset by which image pixels are shifted to effect the rotation of the image.

Exemplary Methodological Implementation

FIG. 9is a flow diagram900that depicts an exemplary methodological implementation of process for eliminating dead zone implications from panoramic images. The specific process depicted in the flow diagram900relates toFIG. 7, previously described. Although a specific exemplary methodological implementation is shown in a specific order, it is noted that more or less steps may be performed to implement the described functionality and that the steps may be performed in an order other than that shown in the present example.

An optimum location for a zero degree (0°) boundary is determined at block904or block906depending on whether the determination is made automatically (“Yes” branch, block902) such as with a face tracker, or manually (“No” branch, block902.

A camera may include a manual selector by which a user can determine an optimum location and set the manual selector to that location to identify an offset between a nominal boundary location and an actual boundary location. The zero degree (0°) boundary is set accordingly at block908.

If performed automatically, a face tracking mechanism may be used to automatically detect location of faces of meeting participants that are seated around a conference room table. Any face tracking mechanism known in the art may be used to accomplish this operation. When the locations of the faces are recognized, a determination is made as to the largest area between faces. The actual zero degree (0°) boundary is then set to the center of this area at block908.

Once the actual zero degree (0°) boundary is determined, an offset is determined at block910. The offset is the difference between the actual zero degree (0°) boundary and the nominal zero degree boundary. The nominal zero degree boundary is a location where the three hundred and sixty degree image is split for purposes of rendering a two-dimensional panoramic image. This is predetermined as a function of hardware or through a software mechanism.

The offset is a distance that the panoramic image must be rotated to accommodate the newly acquired actual zero degree boundary. This may be determined in actual distance by degrees or in other units compatible with the remapping function according to the (x, y) coordinate system. Once the offset has been determined, the offset is incorporated into the remapping function.

At block912, the individual images726are received. At block914, the remapping module remaps the individual images726into the panoramic image732, taking the offset into account. The resulting image will not split the image of a person's face between side margins of the panoramic image.

It is noted that the process outlined above is a general procedure that may be altered to conform to any implementation described above or performed in accordance with the description provided herein. Specific steps shown inFIG. 9may be omitted or altered in any particular implementation. Furthermore, additional steps may be included with those shown inFIG. 9without departing from the scope of the present description and claims.

FIG. 10is a flow diagram1000that depicts an exemplary methodological implementation of process for eliminating dead zone implications from panoramic images by reproducing a portion the panoramic image. The process depicted in the flow diagram1000relates specifically to the block diagram shown inFIG. 7, previously described. Although a specific exemplary methodological implementation is shown in a specific order, it is noted that more or less steps may be performed to implement the described functionality and that the steps may be performed in an order other than that shown in the present example.

As previously described, one or more portions of one of the individual images726or of the panoramic image732that depicts an area adjacent to the zero degree (0°) boundary can be reproduced and appended to the panoramic image so that any object split by the zero degree boundary will appear in whole in the panoramic image (seeFIG. 4aandFIG. 4b). This procedure may be performed in addition to resetting an optimum zero degree boundary, but it may also be performed without first making such a determination.

Since an image in the dead zone will be reproduced, finding an optimum location for the dead zone is not necessarily required. However, if a person is seated in the dead zone and this technique is applied, a portion of that person's image will appear adjacent to one margin of a resultant panoramic image and a whole image of the person's face will appear adjacent to the other margin.

At block1002, an optimum location for the zero degree (0°) boundary is determined similar to the ways described above (FIG. 9,904-906). The individual images726are received at block1004and the assembly module736stitches the images726together to form the panoramic image732at block1006. The assembly module736then determines (via a predetermined amount or automatically according to an object in the image or via manual input) a portion (δ) to be reproduced (block1008).

The portion (δ) is reproduced as δ′ and the reproduced portion (δ′) is appended to the panoramic image732at block1008. As previously discussed, two portions (δ) may be reproduced and appended to the panoramic image732, one reproduced portion (δ) at each opposite margin of the panoramic image732.

If desired, the resultant panoramic image732is resized at block1010(optional) to the same size as the panoramic image would be if δ′ were not appended thereto. This may be done by shrinking the panoramic image732horizontally or by padding the top and/or bottom of the panoramic image732with dead space (e.g., black or gray bars).

As previously stated, the process described above is but one implementation of several that utilize the addition of one or more duplicated image portions to minimize problems encountered with dead zones. The specific recitation of steps listed above is provided by way of example only and not by limitation.

Exemplary Methodological Implementation: Client Device

FIG. 11is a flow diagram1100depicting a process for minimizing dead zones in a panoramic image by rotating the panoramic image after the panoramic image has been created and stored. In the following discussion, continuing reference is made to elements and reference numerals shown in previous figures. The process depicted by the flow diagram1100refers to the block diagram shown inFIG. 8, previously described. Although a specific exemplary methodological implementation is shown in a specific order, it is noted that more or less steps may be performed to implement the described functionality and that the steps may be performed in an order other than that shown in the present example.

At block1102, the stored panoramic image838is received by the client device800. The panoramic image838may be received from the panoramic camera700in real time or it may be retrieved from the memory828at some time after it has been stored. At this point, a nominal zero degree boundary848has been set in order to render the panoramic image in a two-dimensional format. However, the nominal zero degree boundary848may not be in an ideal location for any of several reasons.

If a face tracker is available (“Yes” branch, block1104), then the face tracker840scans the panoramic image838to locate faces in the panoramic image838(block1106). If a better location for the zero degree boundary is located, then the boundary module846identifies the location, the new boundary is set and the rotation module844rotates the panoramic image838accordingly at block1108. If a user accepts the rotated image (“Yes” branch, block1110), then the process terminates.

If no face tracker is available (“No” branch, block1104) or if a user does not like the new location of the zero degree boundary (“No” branch, block1110), then the user manually resets the zero degree boundary at block1112as described above. When the user resets the boundary, the panoramic image838is rotated accordingly (block1114) and the process terminates.

Thereafter, the user may alter the zero degree boundary as desired. For example, if a new participant joins the meeting and happens to sit in the dead zone, the user may wish to adjust the boundary so as to capture a complete image of the new participant.

CONCLUSION

While one or more exemplary implementations have been illustrated and described, it will be appreciated that various changes can be made therein without departing from the spirit and scope of the claims appended hereto.