Patent ID: 12225320

DETAILED DESCRIPTION

Plural cameras coordinate image capture to provide a consolidated field of view visual image to an information handling system. For purposes of this disclosure, an information handling system may include any instrumentality or aggregate of instrumentalities operable to compute, classify, process, transmit, receive, retrieve, originate, switch, store, display, manifest, detect, record, reproduce, handle, or utilize any form of information, intelligence, or data for business, scientific, control, or other purposes. For example, an information handling system may be a personal computer, a network storage device, or any other suitable device and may vary in size, shape, performance, functionality, and price. The information handling system may include random access memory (RAM), one or more processing resources such as a central processing unit (CPU) or hardware or software control logic, ROM, and/or other types of nonvolatile memory. Additional components of the information handling system may include one or more disk drives, one or more network ports for communicating with external devices as well as various input and output (I/O) devices, such as a keyboard, a mouse, and a video display. The information handling system may also include one or more buses operable to transmit communications between the various hardware components.

Referring now toFIG.1, a desktop information handling system10and a portable information handling system34interface with plural cameras that stitch plural fields of view into a consolidated field of view. In the example embodiment, desktop information handling system10processes information with processing components disposed in a housing11configured to operate at a fixed location. A central processing unit (CPU)12executes instructions that process information in cooperation with a random access memory (RAM)14that stores the instructions and information. A solid state drive (SSD)16includes non-transitory memory that persistently stores instructions and information during power of the system, such as an operating system and applications that are retrieved at power up by preboot instructions of an embedded controller (EC)18. Embedded controller18manages system operations on a physical level, such as application of power, maintenance of thermal constraints and interactions with peripheral devices. A wireless network interface controller (WNIC)20supports wireless communication through a radio, such as with WIFI or Bluetooth. A USB hub22has plural USB ports to communicate with external devices, such as peripheral devices, such as with USB Type A, B or C ports. A graphics processing unit (GPU)24further processes information to define visual images for presentation at a peripheral display28, such as by defining pixel values that are communicated through a display cable to peripheral display28. For example, display cable26is a USB Type C cable that support graphical and data communications through a plurality of serial links.

The example embodiment also depicts a portable information handling system32built in a portable housing34that supports mobile use with an integrated display38and keyboard36. Portable information handling system32includes the processing components described with respect to desktop information handling system10that cooperate to process information, such as a CPU and RAM that cooperate to execute instructions of an application. The example embodiment has a clamshell configuration of the portable housing34that rotates between open and closed positions; alternative embodiments might have a tablet configuration built in a planar portable housing. A camera30integrates in portable housing34to capture visual images of a field of view associated with an end user viewing integrated display38. Portable information handling system32may interface with peripheral display28to present visual images. In the example embodiment, peripheral display28also includes a camera that captures visual images of a field of view associated with an end user viewing peripheral display28. In this way, both portable information handling system32and peripheral display28may support a videoconference with images captured by the integrated camera30.

In the example embodiment, plural peripheral cameras30are disposed external to the information handling systems and peripheral display to support capture of visual images, such as to support a videoconference. In an alternative embodiment, a camera integrated in an information handling system or display may be used to generate a consolidated visual image, either with another integrated camera or a peripheral camera. The peripheral cameras30interface with an information handling system and with each other through wireless communication, such as WIFI or BLUETOOTH, or through a cable, such as a Type C USB cable. Peripheral cameras30cooperate to define overlaps in their respective fields of view so that a consolidated field of view provided by the peripheral cameras offers a wider field of view to use by the information handling systems with full camera resolution than would be available from each camera separately. The cooperation between peripheral camera30is coordinated with logic included in the cameras that communicates by wireless signals or through a daisy chain of communication cables, such as the Type C USB cable31that couples to a USB port33of each peripheral camera. In one embodiment, plural cameras cooperate by exchanging field of view information and defining a boundary of an overlap of the fields of view to use as a reference for stitching a consolidated field of view. Once the boundary intersection for the overlapping and non-overlapping portions are defined, the consolidated visual image is stitched from the separate camera visual images by reference to common objects in the visual images, such as with the You Only Look Once (YOLO) algorithm that identifies a user or object in both visual images. In addition, the stitching of the visual images may include fine tuning with edge detection algorithms, such as Sobel, Canny, or Fuzzy Logic. As an example, a typical YOLO algorithm operating on graphics processing resource can process 155 frames per second, sufficient for video.

Referring now toFIG.2, plural cameras coordinate visual image capture to provide a consolidated field of view62. Each camera has a lens50that captures light from a field of view40and directs the light to an image sensor52that captures the visual images with an array of pixels that provide a composite visual image of raw pixel values typically using megapixel resolution. An image sensor processor (ISP)54accepts the image sensor pixel information and offers a variety of processing techniques to enhance image quality, such adjusting for ambient light brightness and color. ISP54may report the raw pixel values for communication as a visual image or can perform compression, such as MP4 compatible video compression that reduces the size of the visual image. A system on chip (SOC) processing resource56manages communication of the visual images from each camera, such as with an integrated radio that communicates through WIFI or BLUETOOTH or by a wire interface such as USB Type C. In one embodiment, each camera communicates separately to an information handling system that separately processes the visual images with each field of view40to generate the consolidated field of view62. A disadvantage to this approach is that the information handling system interfaces with each camera separately and manages a compressed version of the visual image.

In the example embodiment, the cameras communicate directly with each other to coordinate generation of the consolidated visual image at one of the cameras so that only that camera needs to maintain communication with the information handling system. For instance, a USB cable interface provides rapid communication of raw pixel values from one camera to the other so that a precise image stitching may take place. Alternatively, a compressed visual image may be used. Once the cameras define the overlap boundary, the secondary camera may send only non-overlap visual images58to the primary camera so that the amount of communicated visual images is reduced and the primary camera need manage stitching only with inclusion of non-overlapping images rather than removing portions of the secondary camera that overlap. Once the primary camera receives the non-overlapping visual images58, a graphical processing resource at the primary camera stitches the non-overlapping visual images into the consolidated visual image60for communication to the information handling system10. For example ISP54or SOC56accepts raw pixel values, stitches the visual images based on a boundary defined at configuration and then compresses the visual image to an MP4 or similar format. Pre-configuration of the boundary for stitching the visual images provides more rapid processing to generate the consolidated visual image. During capture of visual images, the cameras are monitored for movement, such as with an accelerometer or movement relative to a fixed reference point, so that at detection of movement a re-calibration of the boundary between the camera fields of view may be commanded.

Referring now toFIG.3, an example depicts plural cameras30configured to overlap visual image fields of view40that coordinate to provide a consolidated field of view visual image44. In the example embodiment, a reference marker46is placed in each camera30field of view40to provide a reference in the visual images captured by each camera from which the boundary for stitching the visual images can be determined. For instance, reference marker46is a printed QR code that has an identifier and dimensions. A processing resource on each camera30reads the identifier and queries other local cameras to determine if the reference marker is in any other captured visual images42. If so, then the dimension information and the resolution of the visual images provides a relative distance of each camera to the reference marker so that the consolidated visual images44can align along the boundary. If a reference marker is not found, then a shared object may be used instead. In the example embodiment, the two camera visual images42both have the reference marker in an overlapping portion. The primary camera applies its entire captured visual image stitched with the non-overlapping portion of the visual image of the secondary camera. As a result, consolidated visual image44has a wider field of view than any individual camera can offer without sacrificing visual image quality or introducing distortions. In alternative embodiments, more than two cameras may be used so that a wide enough field of view is available from the consolidated visual image without relying on fisheye types of lens. In the example embodiment, one primary camera30is visible to information handling systems through a wireless access point having a WIFI interface while the other secondary camera30sends the non-overlapping portion of the field of view to the primary camera. This arrangement aids in simplified end user access in a conference room. Alternatively, both cameras may send visual images to an information handling system, such as server, that consolidates the visual image. During configuration, the cameras deduce their position relative to each other so that the area of overlap is generally known. Thus, for example, analyzing to locate the border from the side of the image having the overlap can reduce processing time and processing resource use.

Referring now toFIG.4, a flow diagram depicts a process for configuring plural cameras to provide a consolidated field of view. The process starts at step80with installation of a camera at a location, such as by powering up the camera and interfacing the camera with a network resource, such as WIFI, and/or an information handling system. At step82, the camera attempts to automatically pair with other cameras, such as through a USB cable interface or wireless interface. If only one compatible camera is available, the process continues to step84to enter a single camera mode of operation at step86and complete a single camera setup and calibration at step88. If auto pairing at step82detects a compatible information handling system platform or a compatible camera, the process continues to step90to setup coordination with the resource for capture of visual images with a consolidated field of view. At step92a determination is made of whether the camera is configured to allow a conference room setup and consolidated field of view visual image. If not, the process continues to step86to perform a single camera setup. If other compatible cameras or information handling systems are available, the process continues to step94to setup plural cameras to provide a consolidated visual image with at least some visual information of the consolidated visual image from each of the cameras.

At step94a first camera is placed in a location to support a desired field of view and setup is initiated, such as by powering up a BLUETOOTH SOC to advertise compatibility with a consolidated field of view visual image. At step96a second camera is placed in a location to support a desired field of view and setup is initiated, such as by powering up a BLUETOOTH SOC to advertise compatibility with a consolidated field of view visual image. At step98the first and second camera field of view image overlap is determined, such as by searching for a reference marker or common object in each field of view. The definition of the boundary may be performed locally at one of the cameras by sending the image of the other camera to it or may be performed at an information handling system interfaced with both cameras. If at step98the camera visual image overlap is not acceptable to provide stitching that supports a consolidated visual image, the process continues to step100and returns to step94to provide the end user with another opportunity to position the cameras. If at step98the overlap in fields of view is acceptable, the process continues to step102to define the field of view overlap. In one embodiment, the fixed location of the cameras allows a precise demarcation of the overlap boundary at the camera raw pixel value level. Alternatively, a compressed image may be used. In either case, the cameras coordinate to assign one camera as the primary that receives a non-overlap portion of the secondary camera visual information for local processing at the primary camera into a consolidated visual image. At step104a video calibration is performed and shared between the cameras and at step106a combination of both cameras' fields of view into a consolidated field of view is performed by including the non-overlapping portion of the secondary camera in the visual image captured by the primary camera. The video calibration can include an exchange of camera resolution information so that both cameras can command the highest compatible resolution available that will provide a comprehensive consolidated visual image. Calibration may also include adjustments for different sizes of the captured visual images such as due to camera lens configurations and camera distances relative to the boundary identification object. At step108the calibration is complete and the cameras monitor for a change of position that initiates a recalibration, such as by monitoring an accelerometer in each camera or changes in the captured visual image of a fixed object position.

Although the present invention has been described in detail, it should be understood that various changes, substitutions and alterations can be made hereto without departing from the spirit and scope of the invention as defined by the appended claims.