System, method, and software for image processing

According to certain embodiments, an image processing system includes a computing system that receives a video signal including data for one or more images and determines a region of interest in a particular image of the one or more images. Using the determined region of interest, the computing system instructs the camera to generate a peripheral region of one or more subsequent images having a reduced information intensity level relative to the information intensity level with which the region of interest of the one or more subsequent images are generated. The region of interest of the one or more subsequent images corresponds to the region of interest of the particular image. The peripheral region including portions of the image other than the region of interest.

TECHNICAL FIELD OF THE DISCLOSURE

This disclosure generally relates to imaging systems, and more particularly, to a system, method, and software for image processing.

BACKGROUND

Image processing systems generally process imagery from a camera into a form suitable for view on a display. Image processing systems may enhance the quality of imagery that may otherwise be difficult to view without imagery enhancement and/or provide other suitable functionality. Image processing systems are often implemented with a computer processor that processes video signals that have been digitized using an analog-to-digital (ADC) circuit or other similar type of digitizing circuitry. Image processing systems may perform various types of image processing on a digitized video signal to manipulate imagery represented by the video signal. For example, image processing systems may perform one or more fast Fourier transforms (FFTs) on the received video signal from which one or more filters may be applied to sharpen, change contrast, or change color hue of the imagery represented by the video signal.

SUMMARY

In accordance with the present disclosure, disadvantages and problems associated with previous and existing image processing systems may be reduced or eliminated.

According to certain embodiments, an image processing system includes a computing system that receives a video signal including data for one or more images and determines a region of interest in a particular image of the one or more images. Using the determined region of interest, the computing system instructs the camera to generate a peripheral region of one or more subsequent images having a reduced information intensity level relative to the information intensity level with which the region of interest of the one or more subsequent images are generated. The region of interest of the one or more subsequent images corresponds to the region of interest of the particular image. The peripheral region includes portions of the image other than the region of interest.

Certain embodiments of the present disclosure may provide one or more technical advantages. For example, certain embodiments of the image processing system may use less electrical power than conventional image processing systems. Cameras typically use electrical power at levels that in many cases, may make them generally unsuitable for use in handheld devices that are often powered by portable power sources, such as batteries. By selectively scanning certain regions of the image, the image processing system may be enabled for use in applications, such as handheld devices that may inherently have limited amounts of electrical power.

For example, focal plane array devices such as microbolometers may be particularly useful for acquiring imagery within the longwave infrared light spectrum. These devices, however, may consume relatively large levels of electrical power due to the manner in which light in the longwave infrared spectrum is acquired. Thus, selectively acquiring only certain regions of imagery may reduce power requirements for these devices and thus enable their usage in handheld devices in certain embodiments.

Additionally, selectively scanning certain regions of imagery may reduce the amount of data that is processed by the image processing system such that a further reduction in electrical power may be consumed by the image processing system. Although image processing systems may provide valuable enhancement of imagery, their use may be limited when used with portable equipment, such as those that rely upon battery power for their operation. Image processing techniques performed by known image processing systems typically require a relatively large number of computer operations to perform their tasks. These computer operations may draw power at levels that, in some cases, may preclude their use in certain portable devices whose source of power may have a limited capacity. Certain embodiments of the image processing system according to the teachings of the present disclosure may provide a solution to this problem by selectively reducing acquisition, by the camera, of portions of the video signal such that power drawn from the power source by the camera may be reduced.

Certain embodiments of the present disclosure may provide some, all, or none of these advantages. Certain embodiments may provide one or more other technical advantages, one or more of which may be readily apparent to those skilled in the art from the figures, descriptions, and claims included herein.

DESCRIPTION OF EXAMPLE EMBODIMENTS

FIG. 1illustrates an example image processing system10according to certain embodiments of the present disclosure. Image processing system10includes a camera12, a computing system14that comprises an image processor16, and a display18. Image processor16receives video signals from camera12, processes data comprising one or more images from the received video signals, and transmits the processed images to display18. As will be described in detail below, image processor16controls camera12to acquire video signals that have reduced information intensity level in certain regions such that power consumption of image processing system10may be reduced.

Camera12includes a lens20and a read-out integrated circuit22having an focal plane array device or other suitable image generating device and an associated drive circuit26. Lens20refracts or reflects light to form an image on focal plane array device24. Focal plane array device24converts light present on its surface28to a video signal representing a sequence of images formed by light from lens20. Camera12may be any type of camera that generates video signals from received light energy. In certain embodiments, camera12may be a longwave infrared camera that receives and processes light in the longwave infrared spectrum.

Operation of focal plane array device24is controlled by drive circuit26. Drive circuit26may include several components that control operation of focal plane array device24, such as power circuitry, timing circuitry, and/or synchronization circuitry. Drive circuit26may also include other components associated with operation of focal plane array device24, such as amplifying circuitry to amplify video signals from focal plane array device24, filter circuitry to manipulate video signals from focal plane array device, and/or an analog-to-digital (A/D) circuitry to digitize video signals generated by focal plane array device24.

Certain embodiments of focal plane array device24generate video signals by scanning its surface to detect light intensity levels over its surface28and generate images representing the detected variations in light levels. According to the teachings of the present disclosure, focal plane array device24may be controlled by image processor16to selectively scan portions of its surface such that its surface28is partially scanned during acquisition of imagery. That is, focal plane array device24may be controlled by image processor16to not scan certain regions of its surface28while other regions of its surface are fully scanned according to one or more regions of interest detected by image processor16.

Certain embodiments of image processing system10may use less electrical power than conventional image processing systems. Image sensors typically use electrical power at levels that in some cases, may make them generally unsuitable for use in handheld devices that are often powered by portable power sources, such as batteries. By selectively scanning certain regions of the surface28of focal plane array device24, image processing system10may enabled for use in applications, such as handheld devices that may inherently have limited amounts of electrical power.

For example, focal plane array devices such as microbolometers may be particularly useful for acquiring imagery within the longwave infrared light spectrum. These devices, however, may consume relatively large levels of electrical power due to the manner in which light in the longwave infrared spectrum is acquired. Thus, selectively acquiring only certain regions of imagery may reduce power requirements for these devices and thus enable their usage in handheld devices in certain embodiments.

Additionally, selectively scanning certain regions of imagery may reduce the amount of data that is processed by image processing system10such that a further reduction in electrical power may be consumed by image processing system10. Although image processing systems may provide valuable enhancement of imagery, their use may be limited when used with portable equipment, such as those that rely upon battery power for their operation. Image processing techniques performed by known image processing systems typically require a relatively large number of computer operations to perform their tasks. These computer operations may draw power at levels that, in some cases, may preclude their use in certain portable devices whose source of power may have a limited capacity. Certain embodiments of the image processing system according to the teachings of the present disclosure may provide a solution to this problem by selectively reducing acquisition, by the camera, of portions of the video signal such that power drawn from the power source by the camera may be reduced.

Image processor16processes data in video signals from read-out integrated circuit22into a form suitable for view on display18. Image processor16may process data from video signals in any suitable manner. For example, image processor16may perform sharpening algorithms on data in video signals received from camera12to sharpen imagery for view on display18. As another example, image processor16may include an image recognition algorithm that detects certain features, such as colors or outlines, that may not be easily detectable by the human eye.

Image processor16determines regions of interest, and instructs focal plane array device24to partially scan certain regions according to the determined regions of interest. Image processor16also uses this information to combine images from received video signal into a form suitable for view on display18. Video signals output from image processor16may be viewable on any suitable type of display, which may include, for example, those that display video signals conforming to a National Television System Committee (NTSC) protocol, Phase Alternating Line (PAL) protocol, or a sequential color with memory (SECAM) protocol.

Computer display18is any device that generates a viewable images from video signals processed by image processor16. In certain embodiments, display18is a head mounted display (HMD) that may be sufficiently small to be placed in relatively close proximity to a human eye. HMDs typically include relatively small computer displays that may be positioned in front of the user's eye. Thus, computer displays of this type may provide for use of image processing system10in optical devices, such as binoculars, telescopes, or head mounted displays.

Optical devices, such as these may be particularly useful as portable devices. The utility provided by certain optical devices, however, may be limited by their level of portability. For example, although binoculars may be generally useful to users who may be traveling on foot, their usefulness may be diminished if their size and weight creates an undue burden. It is known that image processors may be relatively small in size and weight, yet they may exhibit power requirements that are proportional to their processing load irrespective of their size and/or weight. Therefore, the weight and size of batteries that power image processing circuits may limit the portability of and thus utility of optical devices, such as binoculars or head mounted displays. Certain embodiment of image processing system10may provide a solution to this problem by reducing power requirements while maintaining a sufficient level of information intensity for regions of interest determined by image processor16.

Computing system14includes one or more processors32and one or more memory units34. A processor as described herein may include one or more microprocessors, controllers, or any other suitable computing devices or resources and may work, either alone or with other components of image processing system10, to provide a portion or all of the functionality of image processing system10described herein. A memory unit34as described herein may take the form of volatile and/or non-volatile memory including, without limitation, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), removable media, or any other suitable memory component. A portion or all of memory units34may be remote from computing system14, if appropriate.

Computing system14may include an image processor16that is operable to determine regions of interest in imagery generated by camera12. Image processor16may be implemented in any suitable combination of hardware, firmware, and software. Additionally, image processor16may be implemented in a single location or in a distributed fashion (e.g., such that certain functions described with respect to image processor16are provided on a system other than computing system14), according to particular needs.

Embodiments of image processor16may include logic contained within a medium. Logic may include hardware, software, and/or other logic. The medium in which the logic is encoded may include a tangible medium. The logic may perform operations when executed by processor32. Certain logic may include a computer program, software, computer executable instructions, and/or instructions capable being executed by computing system14. The logic may also be embedded within any other suitable medium without departing from the scope of the disclosure.

The components of computing system14may be implemented using any suitable combination of software, firmware, and hardware. These components may be implemented using one or more computer systems at one or more locations. Each computer system may include any appropriate input devices, output devices, mass storage media, processors, memory, or other suitable components for receiving, processing, storing, and communicating data. For example, each computer system may include a personal computer, workstation, network computer, kiosk, wireless data port, personal data assistant (PDA), one or more Internet Protocol (IP) telephones, one or more servers, a server pool, one or more processors within these or other devices, or any other suitable processing device.

Components of image processing system10may be communicatively coupled with other computing systems via a network. The network facilitates wireless or wireline communication, and may communicate, for example, IP packets, Frame Relay frames, Asynchronous Transfer Mode (ATM) cells, voice, video, data, and other suitable information between network addresses. The network may include one or more LANs, radio access networks (RANs), metropolitan area networks (MANs), WANs, all or a portion of the global computer network known as the Internet, and/or any other communication system or systems at one or more locations.

In operation, image processor16receives video signals representing one or more images from focal plane array device24, determines a region of interest in the received images, and controls focal plane array device24to reduce information intensity certain regions of the generated video signal. In this manner, power consumption of focal plane array device24may be reduced by turning off its image acquisition circuitry for at least a part of its operation. Additionally, reduced acquisition of video data may reduce the amount of processing load upon image processor16for further reduction in power consumption in certain embodiments.

FIG. 2illustrates an example image36that may be displayed on the display18by the image processing system ofFIG. 1. Image36includes a region of interest38and a peripheral region40comprising portions of the image other than the region of interest38. Region of interest38generally describes a portion of image36that is scanned by focal plane array device at its full scan rate, while peripheral region40generally describes a portion of image36that is partially scanned by focal plane array device24.

Partial scanning of peripheral region40may be accomplished in any suitable manner. In the particular example shown, image processor16controls focal plane array device24to not scan certain rows44within peripheral region40. A row44generally refers to a linear segment of information acquired by focal plane array device24and typically extends laterally across image36. Thus, focal plane array device24may provide a reduced information intensity level in peripheral region40by scanning only a portion of its rows44.

In certain embodiments, focal plane array device24may provide a reduced information intensity level by performing an interlaced scan technique in peripheral region40while performing a progressive scan technique in region of interest38. The interlaced scan technique generally describes a process whereby odd rows and even rows of image36are alternatively scanned to form the image. The interlace scan technique may, therefore, reduce information intensity by a factor of two due to scanning at half the overall rate that would by provided by the progressive scan technique that scans all rows44sequentially.

As shown, region of interest38includes an element that may be, for example, a moving object, or an object having a particular color scheme and/or shape that may be of interest to the user. Element42may be determined by image processor16, or element42may be determined manually by the user.

Region of interest38may have any size and incorporate any portion of the generated image36. The size and position of region of interest38may be determined by image processor16according to information included in image36. For example, image processor16may determine that a particular element in the upper, left corner of image36to be a region of interest38at one instant, and determine another element in the lower, right corner of image36to be another region of interest38at a later instant. Thus, the size and position of region of interest38may change during operation of image processing system10.

FIG. 3illustrates another example image36′ that may be displayed on the display18by the image processing system10ofFIG. 1. Like image36ofFIG. 2, image36′ includes a region of interest38′ and a peripheral region40′ having a reduced information intensity level relative to region of interest38′. Image36′ generated by image processor16differs, however, in that its peripheral region40′ comprises certain rows44′ and columns46of image36′ having reduced information intensity relative to region of interest38′.

Rows44′ of peripheral region40′ that are above and below region of interest38′ may be processed in a manner similar to the manner in which image processor16processes the peripheral region40of image36. Rows44′ coinciding with region of interest38′ may be processed by selectively turning off focal plane array device24while scanning in the peripheral region40′ and turning on focal plane array device24while scanning through region of interest38′.

In the particular example shown, region of interest38′ is a foveal region of image36′. The human eye has a foveal region that detects visual information with greater precision than other portions of the eye. The foveal region yields a portion of the eye's field-of-view that is more precise. In most cases, the foveal region is centrally located within the field-of-view. The centrally located region of interest38′ may therefore, provide enhanced visual information intensity for the foveal region, while the peripheral region40′ displays video information with reduced visual information intensity. Certain embodiments of region of interest38′ including a foveal region may be particularly useful for displays, such as HMDs that remain generally static relative to the user's eye.

FIG. 4illustrates an example method for image processing according to certain embodiments of the present disclosure. In act100the process in initiated.

In act102, image processor16receives a video signal comprising data for one or more images from focal plane array device24. Focal plane array device24may be any suitable type that generates video signals according to images formed by light. In certain embodiments, focal plane array device24includes a microbolometer that generates video signals according to light in the longwave infrared spectrum.

In act104, image processor16determines a region of interest38or38′ in the received image36or36′. In certain embodiments, image processor16may determine region of interest38or38′ according to features within image36or36′. For example, image processor16may store visual characteristics associated with a particular object in memory34, such as the object's color, contrast, and/or shape. Image processor16may compare the received image36or36′ with these stored visual characteristics to determine a particular region of interest38or38′.

In certain embodiments, image processor16may determine region of interest38or38′ according to a rate of change of information within image36or36′. For example, the received image36or36′ may include video information associated with a fugitive running across a field. Although other objects such as plants in the field may remain essentially stationary, imagery associated with the running fugitive may have a rate of change sufficient to trigger image processor16to determine a region of interest38or38′ that includes video information associated with the running fugitive.

In certain embodiments, image processor16may determine region of interest38or38′ according to manual input from its user. For example, image processing system10may include a mouse or other similar device that receives user input for moving a cursor across display18, and a selection button for selecting a desired region of interest38or38′ by image processor16.

In act106, image processor16controls focal plane array device24to generate a peripheral region40or40′ having reduced information intensity relative a region of interest38or38′ of image36or36′. Image processor16may control focal plane array device24to reduce the information intensity level of peripheral region40or40′ in any suitable manner. In certain embodiments, image processor16may instruct focal plane array device24to not acquire certain rows44or44′ of peripheral region40or40′ of image36or36′, respectively. In certain embodiments, image processor16may instruct focal plane array device24to not acquire certain rows44′ and/or columns46of peripheral region40′ of image36′.

The previously described process continues during operation of image processing system10. When operation of image processing system10is no longer needed or desired, the process ends in act108.

Modifications, additions, or omissions may be made to the previously described process without departing from the scope of the disclosure. The process may include more, fewer, or other acts. For example, image processor16may apply one or more filters to the received imagery for adjusting color hue, contrast, or other visual aspects of the generated imagery. As another example, determination of one or more regions of interest38or38′ within image36or36′ may be made according to one or more threshold values that may be manually selectable by the user.

Although the present disclosure has been described with several embodiments, a myriad of changes, variations, alterations, transformations, and modifications may be suggested to one skilled in the art, and it is intended that the present disclosure encompass such changes, variations, alterations, transformation, and modifications as they fall within the scope of the appended claims.