Patent ID: 12211465

LIST OF REFERENCE NUMBERS APPEARING IN THE FIGURES

2—in-situ display monitoring and calibration system4—display6—display control system8—coordinate system showing x-axis, y-axis, and z-axis8X—x-axis8X a,8Xb—first x-axis, second x-axis8Y—y-axis8Ya,8Yb—first y-axis, second y-axis8Z—z-axis10—square tile, which is a regular 4-sided polygon10a,10b, etc. —first square, second square, etc.11—pitch distance12—square tiling of the plane12v—representative vertex of the square tiling12s—representative side of the square tiling14—predetermined pattern corresponding to a tiling of the plane16—rectangular tiling of the plane20—actuateable linkage21—Communication network22—data processing means23—visual media data24—calibration pattern25—expected image26—calibration data set27—transformed visual media data28—visual media rendered on the viewing plane of the display29—configuration data30—image acquisition system30a,30b—first, second image acquisition system31—image acquisition system stored position33—image acquisition system deployed position34—plurality of light receiving elements35—relative illuminance36—imaging plane38—captured image40—normalized image41—normalization function50—incidence vector50a,50b,50c, . . . —first, second, third, etc. incidence vector70—display module70a,70b,70c, . . . —first, second, third, etc. display module71—light emitting element71a,71b, etc.—first, second, etc. light emitting element72—plurality of light emitting elements72a,72b, etc.—first light emitting element, second light emitting element, etc.74—display plane74a,74b—first display plane, second display plane75—display plane disposed at a first angle with respect to the viewing plane76—display module substrate78—display assembly78a,78b,78c, etc.—first, second, third, etc. display assembly80—viewing plane82—surface normal vector84—luminous output84a,84b, . . . —first, second, etc. luminous output86—viewpoint vector86a,86b, . . . —first, second, etc. viewpoint vector{i, j, k}—unit vectors in x, y, and z directions, respectively{x0, y0, z0}—location of the center of the imaging plane in 3 dimensions{xi, yi, zi}—location of the ithlight emitting element in 3 dimensions{ai, bi, ci}—direction cosines corresponding to the ithincidence vector, the incidence vector having the form: ai*i+bi*j+ci*kG1, G2, G3, G4—first, second, third, fourth geometric feature of the viewing planeF1, F2, F3, F4—first, second, third, fourth geometric feature identified in a captured image, in which F1corresponds to G1, F2corresponds to G2, F3corresponds to G3, and F4corresponds to G4.r0, r1, r2—first, second, third radial distances from a reference pointA0—area located at a distance of r0A1—projection of area A0at distance r1A2—projection of area A0at distance r2200—a process for rendering visual media on a viewing plane202—process step of receiving, in a display control system, a frame of visual media data204—process step of transforming a frame of visual media data in a display control system206—process step of displaying a transformed frame of visual media300—a process for creating a normalized image302—process step of positioning in a deployed position with respect to a display, an image acquisition system304—process step of triggering an image acquisition system to acquire a captured image306—process step of defining an incidence vector for each of a plurality of light receiving elements308—process step of associating with each of the light receiving elements comprising the imaging plane a normalization function310—process step of applying a normalization function to a captured image thereby producing a normalized image400—a process for calibrating a display402—process step of displaying a calibration pattern404—process step of associating an expected image with a calibration pattern406—process step of triggering an image acquisition system to acquire a captured image of a viewing plane408—process step of creating a normalized image from a captured image410—process step of forming a calibration data set comprising the color and brightness differences between an expected image and a normalized image412—process step of applying, in a display control system, a calibration data set to the rendering of visual media upon a viewing plane of a display such that the differences between a normalized image and an expected image are reduced.

DESCRIPTION

Uniformity in color, brightness, grayscale are fundamental visual performance goals for a large display. Any visual non-uniformity present on the viewing plane of the display is easily noticed by viewers due to the highly refined and discriminating qualities of the human visual system. It often happens that one or more light emitting elements or display modules must be replaced due to damage, aging, or acts of nature. A replacement light emitting element or display module often has a different grayscale, brightness and/or color response as the element or module, respectively, that the light emitting element or display module replaces. In-situ monitoring and calibration of a display is particularly effective for maintaining uniformity in color, brightness, and grayscale across the entire viewing plane of the display, even when replacement of light emitting elements becomes necessary.

In general terms, in-situ display monitoring and calibration uses an image acquisition system to capture images of the viewing plane of the display. Captured images may then be processed to characterize various visual performance characteristics of the display. When not in use capturing images of the display, the image acquisition system can be stored in a manner that protects it from environmental hazards such as dust, dirt, precipitation, direct sunlight, etc. In addition, images may be presented on the display that facilitate the calibration process. For example, a calibration image in which a plurality of light emitting elements is set to a particular color and intensity may be displayed, an image then captured, and then a difference between what was expected and what was captured may be developed for each light emitting element. Differences between captured images and expected images may be used to create a calibration data set which then may be used to adjust the display of further images upon the display.

The visual performance of a display may be referenced from a defined viewpoint, which is essentially a point in 3-dimensional space from which the viewing plane is viewed by a person. The image acquisition system has an imaging plane for capturing images that is generally not at the same location as the viewpoint. A captured image may be post processed to infer what the display looks like at the viewpoint of choice. Each light emitting element of the display has a predetermined position and orientation in space. Each light emitting element produces an outgoing illuminance that varies in both brightness and color depending on the distance to the viewer and on the angle between the viewer and the illuminance pattern produced by the light emitting element. Knowing the distance, angles, and illuminance pattern between a light emitting element and an image plane enables the system to capture images of the viewing plane on the imaging plane and then infer, by computations involving the know distance, angles, and illuminance pattern, what the viewing plane looks like when viewed from the viewpoint. Both monitoring of the display and calibration of the display are thereby enabled by the system and methods of the present disclosure.

To further facilitate the present description, it will be useful now to turn to the construction of a display according to various embodiments of the present disclosure. Tesselation of a planar surface is the tiling of the plane using one or more geometric shapes, called tiles, creating no gaps and no overlaps. A periodic tiling has a repeated geometric pattern. A regular tiling is a tiling in which all tiles are regular polygons having the same size and shape. Square, triangular, and hexagonal tilings are each an example of a regular, periodic tiling that can achieve a tesselation of a planar surface without gaps or overlaps. Tilings are of special interest in the construction of modular displays because their properties enable the construction of large displays with desirable properties. Assembling a plurality of smaller display modules in which each display module is configured to have a size, shape, and orientation corresponding to a predetermined tiling may produce a large display having no gaps and no overlaps between adjacent display modules.

Within a single display module, a plurality of light emitting elements may be arranged in a predetermined pattern derived from an appropriately configured tiling. A planar tiling of regular polygons consists of edges and vertexes. The set of vertexes of a regular polygon tiling can be seen to create a pattern with a high degree of regularity. A highly uniform visual effect may be produced by placing a light emitting element at or about each of the vertexes of a regular polygon tiling.

Light emitting elements of the present disclosure may each comprise a single light emitting device or multiple light emitting devices. A preferred light emitting element combines red, blue, and green light emitting devices within one light emitting element so as to provide full color spectrum display. Monochrome and other combinations of devices may be used still within the spirit and scope of this disclosure. In other embodiments a light emitting element may comprise white, red, blue, and green devices within a single light emitting element. In other embodiments a light emitting element may comprise red, green, blue, and cyan devices. In other embodiments a light emitting element may comprise red, green, blue, yellow, and cyan devices, or any combination of devices emitting at different colors within a single light emitting element. In other embodiments multiple devices emitting at substantially the same color may be used.

In still other embodiments of the present disclosure, light emitting elements may be replaced by light reflective elements. A light reflective element may receive a portion of incoming ambient or directed light and then reflect a portion of the light back to the viewer of a display. Modulating the reflective properties of the light reflective element allows control over the intensity of the reflected light. The portion of incoming ambient or directed light that is not reflected to a viewer may be absorbed, scattered, or otherwise redirected so that it is substantially attenuated with respect to a viewer of the display. A plurality of light reflective elements may be modulated so as to produce images upon a viewing plane. For a light source, a reflective display system may use ambient light, directed non-ambient light, or a combination of both ambient and directed non-ambient light in producing a display.

In creating a uniform visual effect, it is useful to consider a property called pitch distance, which is the distance between any light emitting element and its closest adjacent light emitting elements. It can be seen that a highly uniform visual effect is produced by maintaining a highly uniform pitch throughout a single display module and across a plurality of adjacent display modules. Preferred embodiments of the present disclosure use light emitting elements located at or about the vertexes of a regular polygon tiling. A regular square tiling is one such preferred tiling, producing a uniform visual effect by providing uniform spacing between both rows and columns of light emitting elements. The spacing between adjacent rows and between adjacent columns of a regular square tiling may be referred to as the pitch of that pattern. In such a square tiling, it can be seen that any light emitting element will have at least two closest adjacent neighboring elements that are spaced apart from each other by a distance close to or substantially equal to the pitch distance.

In addition to uniform pitch within a single display module, the spacing between display modules can be controlled so that uniform pitch of light emitting elements is maintained across a plurality of assembled display modules. A preferred embodiment is to provide a display module with a perimeter region, of a predetermined width, that contains no light emitting elements. The preferred width of the perimeter region is less than or about equal to one half of the pitch distance, when measured inward and along the edges of the regular polygon tiling defining the location of the plurality of the light emitting elements. When two display modules are assembled adjacent to one another, each module may provide a perimeter region width of about one half of the pitch, which cumulatively creates a pattern of uniform pitch spanning both modules. A plurality of display modules may thereby be assembled to create uniform pitch spanning the plurality of display modules.

A single display module may comprise a plurality of light emitting elements coupled to a substrate and arranged in a predetermined pattern corresponding to the vertexes of a regular polygon tiling. The display module has a perimeter. A plurality of display modules may be assembled such that a portion of the perimeter of each display module abuts a portion of the perimeter of at least one other display module, each module positioned to maintain uniform pitch spacing across the plurality of display modules.

A display system according to the present disclosure may be constructed by assembling a plurality of display modules onto a support frame, the support frame having been previously constructed.

Turning now toFIG.1A, shown is a regular four-sided polygon, also called a square10, consistent with the square tiling12of the two-dimensional plane shown inFIG.1B. A coordinate system8is indicated so as to make discussion of geometry features of the present disclosure clearer. Square tiling12is comprised of a plurality of square tiles, of which first square10aand second square10bare typical, arranged so that no gaps and no overlaps are produced. When arranged into the predetermined pattern shown inFIG.1B, the square tiling12can be seen to create a plurality of vertex12vand a plurality of side12s, in which every vertex12vis separated a distance of about12sfrom each of its closest neighboring vertexes.

FIG.1Cshows predetermined pattern corresponding to a tiling of the plane14according to a square tiling. Overlaid onto the predetermined pattern corresponding to a tiling of the plane14are x-axis8X and y-axis8Y, showing that a coordinate system can be overlaid onto the predetermined pattern to facilitate clear disclosure of the location and alignment of other features to be described. The enlarged section, denotedFIG.1D, shows that the square tiling of the plane gives rise to a highly uniform spacing of vertexes, which can be characterized as pitch distance11. Pitch distance11corresponding to the predetermined pattern14gives rise to uniform spacing between rows and columns when that predetermined pattern is based upon a square tiling. It can be seen that row spacing and column spacing are both about equal to the pitch distance11.

Turning now toFIG.1E, shown is a display module70having a plurality of light emitting elements72, of which first light emitting element71aand second light emitting element71bare individual members of the plurality. Plurality of light emitting elements72is shown arranged according to a predetermined pattern so as to create a highly uniform visual effect upon display plane74.FIG.1Fshows how predetermined pattern14according to a square tiling of the plane may be used to position individual light emitting elements71a,71b, and71caccording to the location of the vertexes of said predetermined pattern14. Superimposed upon the plurality of light emitting elements are x-axis8X and y-axis8Y. The display module70ofFIG.1Fcomprises a plurality of light emitting elements, each of which may be a single light emitting device or multiple light emitting devices. A preferred light emitting element combines red, blue, and green light emitting devices within one light emitting element so as to provide full color spectrum display. Monochrome and other combinations of devices may be used still within the spirit and scope of this disclosure. The display modules ofFIG.1EandFIG.1Feach have a region adjacent to their perimeter that is free from light emitting elements. This enables close spacing of adjacent modules as will be seen now.

FIG.1Gshows a first display module70aadjacent to a second display module70band disposed so that their display planes74aand74babut and their respective y-axes8Ya and8Yb are substantially aligned, thereby creating a highly uniform visual effect that spans the combined display modules. A pitch distance can be defined between adjacent light emitting elements between adjacent display modules that is substantially equal to the pitch distance between adjacent light emitting elements within a single display module.

FIG.1Hshows a first display module70aadjacent to a second display module70band disposed so that their respective display planes74aand74babut and their respective x-axes8Xa and8Xb are substantially aligned, thereby creating a highly uniform visual effect that spans the combined display modules. A pitch distance can be defined between adjacent light emitting elements between adjacent display modules that is substantially equal to the pitch distance between adjacent light emitting elements within a single display module. When abutted and aligned in the foregoing manner, two adjacent modules may be combined such that their combined plurality of light emitting elements are disposed upon a single predetermined pattern14defining a regular tiling of the plane.

FIG.1GandFIG.1Hmake it clear that a large display may be constructed from display modules designed according to the teaching ofFIG.1A-FIG.1H. Such a large display will tile the two-dimensional plane without gaps and without overlaps and produce a highly uniform visual effect. Any number of display modules may be combined in both x and y directions to make a large display that is substantially free from visual aberrations.

Turning now toFIG.2A, shown is a representative environment for using in-situ display monitoring and calibration system2. The figure shows a perspective view of a display4, controlled by display control system6, the display having a plurality of light emitting elements72disposed in a predetermined pattern collectively creating a viewing plane80. The plurality of light emitting elements may be formed in a predetermined pattern according to any of the teachings ofFIG.1A-FIG.1H. On the display is shown representative visual media28rendered on viewing plane80. Image acquisition system30is shown in a stored position31.

FIG.2Bshows the display4ofFIG.2Aand additionally shows image acquisition system30in a deployed position33with a calibration pattern24rendered on the viewing plane of display4. Associated with the display is display control system6, which is operative to control the presentation of visual media on the display as well as to control the presentation of calibration patterns24. The viewing plane80of display4inFIG.2AandFIG.2Bhas a predetermined geometric shape, geometric features G1, G2, G3, and G4being associated with that geometric shape. In the embodiment ofFIG.2AandFIG.2B, the geometric features identified as G1, G2, G3, and G4are corners of rectangular viewing plane80. Other embodiments may have a viewing plane having a different shape and consequently may have other identifiable geometric features that may be corners, edges, curved shapes, or other identifiable geometric features.

The deployed position33shown inFIG.2Bplaces image acquisition system30in a predetermined position and orientation with respect to the viewing plane. Said predetermined position and orientation is substantially repeatable each time system30is moved to deployed position33. Causing image acquisition system30to cycle from deployed to stored to deployed position again results in disposing image acquisition system in substantially the same position and orientation as in the previous deployed position.

Image acquisition system30is triggerable to capture one or more images when the system is in the deployed position. When triggered, an image may be captured, the image comprising at least a portion of the viewing plane. In preferred embodiments the captured image comprises the entire viewing plane. In other preferred embodiments the image acquisition system may comprise a plurality of imaging planes, each having a known position and orientation when in a deployed position, each operative to capture an image of at least a portion of the viewing plane, the plurality of imaging planes operative to capture, collectively, the entire viewing plane.

Turning now toFIG.3A, shown is a cross section view that was indicated inFIG.2B. In this view, image acquisition system30is shown in deployed position33, imaging plane36being positioned and oriented to capture one or more images of at least a portion of the viewing plane80of display4. In preferred embodiments, image acquisition system30may comprise optical and electronic components known in the art for the capture, storage, and transmission of high-resolution digital images suitable for photometric applications. The image acquisition system ofFIG.3Amay comprise an actuateable linkage coupled between display4and imaging plane36, the linkage being operable to move between the stored position ofFIG.2Aand deployed position33ofFIG.2BorFIG.3A. Imaging plane36comprises a plurality of image receiving elements arranged in a predetermined pattern. A jointed, powered, robotic arm is one of a number of feasible embodiments for actuateable linkage20consistent with the operational requirements previously described.

The stored position of the image acquisition system may be further characterized in that any electrical and optical components of the image acquisition system contributing to or responsible for capturing images are substantially protected from exposure to environmental contaminants including dust, dirt, moisture, direct sunlight, etc., that may detrimentally affect the operation of the image acquisition system.

Continuing withFIG.3A, shown are first incidence vector50aand first viewpoint vector86a, both originating from a first light emitting element71acomprising the viewing plane80. Incidence vector50aoriginates at first light emitting element71aand points to the region on imaging plane36that is operable to image first light emitting element71a. In similar fashion, second incidence vector50band second viewpoint vector86boriginate at second light emitting element71b, and third incidence vector50cand third viewpoint vector86coriginate at third light emitting element71c. Incidence vector50band50coriginate at second and third light emitting elements71band71c, respectively, and each points to the region on imaging plane36that is operable to image second and third light emitting elements71band71c, respectively. An incidence vector may be defined for each light emitting element comprising the viewing plane. It is evident inFIG.3Athat each incidence vector is influenced by the geometrical and optical aspects of the position and orientation of imaging plane36with respect to the position and orientation of viewing plane80.

A viewpoint may be defined anywhere in three-dimensional space from which the viewing plane is visible. The viewpoint represents a viewer located at that distance looking at the viewing plane. For any given, fixed viewpoint, at each light emitting element a viewpoint vector may be defined originating at the light emitting element and extending to the viewpoint. For any given, fixed viewpoint, each light emitting element may be expected to possess a unique viewpoint vector. It is evident from the geometry that a fixed viewpoint located far away from the viewing plane has the property that each viewpoint vector is essentially parallel to every other viewpoint vector. InFIG.3Afirst, second, and third viewpoint vectors86a,86b, and86c, respectively, are drawn consistent with a viewpoint that is located far enough away so that the viewpoint vectors are close to being parallel. In other embodiments the viewpoint may be close enough to the viewing plane that the viewpoint vectors are not close to being parallel. It is evident that, regardless of how far the viewpoint is from the viewing plane, all viewpoint vectors converge at the viewpoint.

Each light emitting element produces a luminous flux that radiates away from the light emitting element in 3-dimensional space. To facilitate the discussion, a first surface normal vector may be defined that originates at the location of the light emitting element and extends perpendicular to the local curvature of the viewing plane. In addition, a second surface normal vector may be defined originating at a light receiving element comprising the imaging plane and extending perpendicular to the imaging plane. The portion of a light emitting element's luminous flux that is received remotely from the light emitting element by a light receiving element having a given area is inversely proportional to the squared distance between emitter and receiver and is also a function not only of the brightness of the light emitting element but also of the angle between the first surface normal vector and the second surface normal vector. It is evident that for any predetermined position and orientation of the imaging plane, a unique incidence vector may be defined for each light emitting element comprising the viewing plane and that both angle and distance impact the light that is received on the imaging plane by any particular light emitting element.

An index i may be created for enumerating through each light emitting element comprising the viewing plane. Index i may be allowed to take the values from 1 to N, where N is the total number of light emitting elements comprising the display. An incidence vector may therefore be represented as: ai*i+bi*j+ci*k; where {ai, bi, ci} are direction cosines corresponding to the ithincidence vector, and {i, j, k} are unit vectors in x, y, and z directions, respectively. Furthermore, {xi, yi, zi} describes location of the ithlight emitting element in 3 dimensions, and {x0, y0, z0} describes the location of the center of the imaging plane in 3 dimensions. The distance from any particular light emitting element to the center of the imaging plane can be calculated as: Di=[(xi−x0)2+yi−y0)2+(zi−z0)2]1/2

Direction cosines {ai, bi, ci} are accordingly determined by the formulas:
ai=(xi−x0)/Di;bi=(yi−y0)/Di;ci=(zi−z0)/Di;

An even more exacting relationship can be described in which a unique coordinate {x0i, y0i, z0i} on the imaging plane is associated with each light emitting element that is imaged. In that case the distance be determined by the formula:
Di=[(xi−x0i)2+(yi−y0i)2+(zi−z0i)2]1/2
Direction cosines {ai, bi, ci} are then determined by computing:
ai=(xi−x0i)/Di;bi=(yi−y0i)/Di;ci=(zi−z0i)/Di;

FIG.3Bshows an enlarged view of the portion of a portion of image acquisition system in deployed position33, as shown in ofFIG.3A. Visible inFIG.3Bis a plurality of image receiving elements34which collectively form imaging plane36, and the dotted line paths of incoming incidence vectors corresponding to first, second, and third incidence vectors previously described in connection withFIG.3A. In the system ofFIG.3AandFIG.3B, each of the plurality of light emitting elements comprising the viewing plane80has associated with it its own incidence vector that points to the region of imaging plane that images the light received from each of the respective light emitting elements.

Shown now inFIG.4Ais a typical light emitting element71. An X-X axis is indicated as8X and a Y-Y axis is indicated as8Y. Light emitting element71produces a luminous output that diverges as it propagates away from the emitter at the speed of light.FIG.4Bis a schematic representation of the effect on luminous flux that distance from the emitter makes. Luminous output84propagates away from light emitting element71. The same luminous output passing through area A0, located at a distance of r0from the emitter, also passes through area A1, located a distance r1from area A0, and area A2, located a distance of r2from area A0. Given luminous output84produced by light emitting element71, the luminous flux measured by a light receiving element having a fixed size will diminish with distance according to an inverse square law with respect to distance from the emitter. A normalization operation can be performed that compensates the measured value of luminous flux for the distance dependence between the emitter and the receiver. Thus, the luminous output received from light emitting elements that are at different distances from the imaging plane can be directly compared after normalization.

FIG.4Cpresents a graph of relative illuminance35versus radiation angle for the representative light emitting element71ofFIG.4A. The reference designators X-X and Y-Y refer back to the light emitting element ofFIG.4A. 0 degrees on the graph corresponds to a direction that is perpendicular to the two-dimensional plane containing both X-X and Y-Y axes. On this graph the maximum illuminance has a value of 1.0, all other values being relative to this maximum. The graph indicates a representative way in which relative illuminance35will diminish as the angle with respect to either the X-X axis or the Y-Y axis moves away from 0 degrees. A normalization operation can be performed that compensates for the reduction in relative illuminance caused by angle of incidence between the incidence vector and relative illuminance. Thus, the luminous output received from light emitting elements that are at different angles with respect to the imaging plane can be directly compared after normalization. The angles of incidence with respect to x, y, and z axes can be determined using the direction cosines previously described.

It can be understood that the graph inFIG.4Cis an example of one specific emitter for the purposes of teaching in this disclosure. The exact pattern of radiation produced by any emitter is a multivariate function of the device or devices comprising emitter, the emitter's packaging and how the emitter is mounted with respect to the viewing plane. Embodiments of an in-situ display monitoring and calibration system according to the present disclosure may access configuration data29corresponding to one or more of the following: radiation pattern emitted by each emitter; position and orientation of each emitter; and position and orientation of the imaging plane.

Turning now toFIG.5A, shown is exemplary captured image38, captured by the image acquisition system. The image is of a rectangular viewing plane that has been captured with projection effects that are common for camera and imaging systems. Visible in captured image38are geometric features F1, F2, F3, and F4, which correspond to actual physical features present in the real world. In the embodiment ofFIG.5Afeatures F1, F2, F3, and F4present in the captured image are counterparts to real world features G1, G2, G3, and G4visible inFIG.2B. Real world features G1, G2, G3, and G4have associated with them {x, y, z} coordinates in three dimensions. The real-world coordinates of G1, G2, G3, and G4together with the image coordinates of F1, F2, F3, and F4, and calibration data from the imaging system allow an inverse projection transformation to be computed. The inverse projection transformation of the captured image38in combination with incidence vector and distance data from each light emitting element, enable the creation of a normalization function which can then be applied to a captured image to produce, as shown inFIG.5B, a normalized image40. In essence, normalized image40is an estimate, based on the captured image, of what the display looks like from a single viewpoint which may be distinct from position and orientation of the imaging plane. The normalized image may then be used as the basis for photometric operations such as monitoring and/or calibration.

While the embodiments ofFIG.2-FIG.5have been described with reference to a single image acquisition system disposed in a position with respect to the display, other embodiments are within the scope of the disclosure. In other embodiments, the image acquisition system may be moved through a sequence of different positions and/or orientations, capturing images from each different position and/or orientation in the sequence. Each different position or orientation of the image acquisition system may be effective for capturing one or more images of a different portion of the viewing plane. In some embodiments a sequence of overlapping images may be acquired. Overlapping image portions may be used to establish registration between adjacent images. Both captured images and overlapping captured images may be used to establish registration between a captured image and the portion of the viewing plane being imaged. A normalization function may be associated with each position and each orientation of the image acquisition system. Thus, a plurality of normalized images may be acquired that collectively creates a normalized composite image of the entire viewing plane. Monitoring and calibration may then proceed according to methods of the present disclosure.

FIG.6is a functional block diagram of an in-situ display monitoring and calibration system. The system2is shown comprising: an image acquisition system30which is trigerrable to capture one or more captured image38of the viewing plane of a display4; a display control system6having a data processing means22and a normalization function41, the display control system being operative to receive: visual media data23; one or more calibration pattern24; one or more expected image25each of which is associated individually with at least one of said one or more calibration patterns; one or more calibration data set26; a configuration data set29; one or more captured image38; display control system6being further operative to produce transformed visual media data27that is rendered via display4on to the viewing plane of said display as rendered visual media28; display control system6being further operative to: trigger the image acquisition system30to capture one or more captured image38of the viewing plane of display4; produce a normalized image40by transforming captured image38according to normalization function41; compare normalized image40to expected image25and produce one or more calibration data sets26; transform one or more frames of visual media data23according to calibration data set26and configuration data set29to produce transformed visual media data27. Visual media data23comprises brightness and color information for each of a plurality of pixels at one or more encoded resolutions. Transformed visual media data27comprises brightness and color information for each of a plurality of light emitting elements at a displayed resolution.

The apparatus ofFIG.6may include communications network21, which may comprise local and/or wide area networking components capable of transmitting or receiving commands and/or data to local or remote destinations. In conjunction with communications network21, display control system6may be further operable to send, receive, and do data processing operations on one or more of the following: visual media data, calibration pattern, expected image, calibration data set, configuration data, captured image, and normalized image.

FIG.7shows a flowchart for a process200of rendering visual media on the viewing plane of a display that may be carried out by the in-situ display monitoring and calibration system. Process200comprises the steps of:202receiving, in a display control system, a frame of visual media data23, the visual media data comprising brightness and color information at an encoded resolution for each picture element of a plurality of picture elements arranged in a pattern corresponding to said encoded resolution;204transforming said frame of visual media data, in said display control system, using a calibration data set26to produce a frame of transformed visual media data27at a displayed resolution for display on a plurality of light emitting elements collectively forming a viewing plane of a display, said calibration data set comprising adjustments to brightness and color for a plurality of light emitting elements comprising said display;206displaying said transformed frame of visual media28on at least a portion of said viewing plane of said display.

FIG.8shows a flowchart for a process300of creating a normalized image that may be carried out by the in-situ display monitoring and calibration system. Process300comprises the steps of:302positioning in a deployed position with respect to a display, an image acquisition system having a plurality of light receiving elements collectively forming an imaging plane36, the display comprising a plurality of light emitting elements arranged in a predetermined pattern collectively creating a viewing plane, such that said viewing plane is imageable upon said imaging plane;304triggering the image acquisition system to acquire a captured image38of said viewing plane;306defining an incidence vector50for each of said plurality of light receiving elements starting at each of said plurality of light emitting elements and directed toward the portion of the imaging plane that images each of said light emitting elements, each incidence vector having both a direction and a magnitude;308associating with each of said light receiving elements comprising the imaging plane a normalization function41that compensates brightness and/or color differences in said direction and said magnitude for each of said incidence vectors;310applying said normalization function41to the captured image38thereby producing a normalized image40.

FIG.9shows a flowchart of a process400of calibrating a display that may be carried out by the in-situ display monitoring and calibration system. Process400comprises the steps of:402displaying a calibration pattern24on the viewing plane of a display;404associating an expected image25with said calibration pattern, said expected image comprising brightness and color information for each light emitting element comprising the viewing plane;406triggering an image acquisition system to acquire a captured image38of said viewing plane;408creating a normalized image40from said captured image;410forming a calibration data set26comprising the color and brightness differences between said expected image and said normalized image;412producing visual media rendered on the viewing plane of the display28by applying, in a display control system, said calibration data set26to the rendering of visual media23upon the viewing plane of said display such that the differences between said normalized image40and said expected image25are reduced.

Turning now toFIG.10A, shown is a perspective view of an in-situ display monitoring and calibration system2with a calibration image24displayed on viewing plane80of display4. The system is shown comprising a first image acquisition system30aand a second image acquisition system30bwhich collectively image the entire viewing plane. Both image acquisition systems are shown in deployed position33. The display is shown having four geometric features G1, G2, G3, and G4.FIG.10Bshows a first captured image38awhich may be captured by first image acquisition system30aofFIG.10A. Geometric features F1and F4are visible in first captured image38aand they correspond to geometric features G1and G4, respectively, shown inFIG.10A.FIG.10Cshows a second captured image38bwhich may be captured by second image acquisition system30bofFIG.10A. Geometric features F2and F3are visible in second captured image38band they correspond to geometric features G2and G3, respectively, shown inFIG.10A. Display control system6is operable to: actuate both image acquisition systems between deployed33and stored positions; render one or more calibration patterns24on viewing plane80and trigger the capture of first and second image acquisition systems.

The display control system6ofFIG.10Amay also be operable to: associate a first expected image received by first image acquisition system30awith a portion of calibration pattern24, associate a second expected image received by second image acquisition system30bwith a portion of calibration pattern24; produce the first normalized image from first captured image; produce a second normalized image from the second capture image; produce a first calibration data set by comparing first normalized image to first expected image; produce a second calibration data set by comparing second normalized image to second expected image; combine first and second calibration data sets to creating a composite calibration data set that comprises brightness and color corrections for each light emitting element comprising the viewing plane.

Embodiments like that disclosed inFIG.10Amay use a plurality of image acquisition systems to capture images of different portions of a display screen, thereby collectively imaging the entire display. Other embodiments may use a single image capture system that is moved to different positions in order to capture images of different portions of the display, the overall effect being to collectively image the entire display while using just one image capture system.

Although the present invention has been described in considerable detail with reference to certain preferred versions thereof, other versions are possible. It may be desirable to combine features shown in various embodiments into a single embodiment. A different number and configuration of features may be used to construct embodiments of the apparatus and systems that are entirely within the spirit and scope of the present disclosure. Therefore, the spirit and scope of the appended claims should not be limited to the description of the preferred versions contained herein.

Any element in a claim that does not explicitly state “means for” performing a specified function, or “step for” performing a specific function, is not to be interpreted as a “means” or “step” clause as specified in 35 U.S.C. Section 112, Paragraph 6. In particular, the use of “step of” in the claims herein is not intended to invoke the provisions of 35 U.S.C. Section 112, Paragraph 6.