Billiard table lighting and game play monitor

A billiard table top lighting apparatus provides substantially uniform lighting across the surface of a billiard table surface. In one form, the lighting apparatus includes a frame supporting lights that are non-centrally placed such that no lights are supported above a middle portion of the billiard table surface. So configured, the frame can have an attractive profile and further mount additional items above the billiard table surface to enable a variety of other features. For example, the frame may support one or more cameras, one or more motion sensors, one or more microphones, and/or one or more computing devices to enable any of a variety of innovative features. Such features could include automatic game play recording from one or more perspectives, merged video track storage for replay, review, and analysis, automatic lighting and dimming control, control of the apparatus from any mobile device, and the like.

TECHNICAL FIELD

This invention relates generally to billiard table lighting and more particularly to providing substantially uniform billiard table lighting to facilitate automated game play monitoring including image capture and automated image and video merging and other associated features.

BACKGROUND

The game of billiards and related table top games have been known for many years. Such games involve movement of balls on a table top. Typically a ball is struck with an instrument such as a cue to move the ball around the table top surface for positioning, to strike and move other balls, and the like. In some variants, balls are struck into holes in or at edges of the table top called pockets. In other variants, balls are struck so as to contact in a particular way other balls on the table top and/or cushions that line the table top to keep the balls on the table top surface. These games have a variety of names including cue sports, billiards, pool, snooker, pocket billiards, among others. For clarity and convenience, such games will be referred to collectively as billiards or billiard.

Generally speaking, the most popular form of billiards involves a number of colored balls placed on a table. Over many years the sport has attracted significant public interest as a spectator event and also as a personal pastime or hobby, similar to golf and tennis. Because of its colorful nature there has been increased interest in applying modern video capture technology and computing and imaging technology to record or analyze game play. There are available sophisticated methods of video recording of game play, such as in tournaments or exhibits using booms or strategically placed cameras, where a director manually dictates which camera feed to use during play, and where a remote voice commentary is fed into the video record. There are also many examples of “do-it yourself” video recording from manually placed cameras, where the field of view covers the entire table and player activities from a distance and at a perspective view. There have additionally been attempts to place cameras directly over the table, using image analysis to locate the balls and perform ball tracking, but ignoring the view areas around the table and player's activities that are the prominent features of the simple recording of game play with video cameras.

Such attempts at video recording and image ball recognition systems, however, fail to be applicable to a wider general audience of billiard game players. A significant failure of prior attempts is the lack of an integrated approach to incorporate the lighting of the table, both forms of video recording and an audio record of ball sounds and player voices into one approach, at a low cost, and with the ease of use related to one single integrated apparatus.

For instance, to capture sufficiently high quality images of the balls to allow advanced game play by computerized image analysis, the billiard table surface should be substantially uniformly lighted. To achieve rapid, ball recognition by image analysis the scene segmentation portion of the image analysis procedure is most efficiently accomplished by having a flat uniformly lit background. Billiard tables, however, are typically lit using one or more light sources disposed above a center portion of the table such that lighting at the edges of the table is markedly worse than at the center. Although the World Pool-Billiard Association provides the following equipment specifications for lighting, such specifications do not suggest the level of uniform illumination typically needed for imaging projects: “15. Lights The bed and rails of the table must receive at least 520 lux (48 footcandles) of light at every point. A screen or reflector configuration is advised so that the center of the table does not receive noticeably more lighting than the rails and the corners of the table. If the light fixture above the table may be moved aside (referee), the minimum height of the fixture should be no lower than 40 inches (1.016 m) above the bed of the table. If the light fixture above the table is non-movable, the fixture should be no lower than 65 inches (1.65 m) above the bed of the table. The intensity of any directed light on the players at the table should not be blinding. Blinding light starts at 5000 lux (465 footcandles) direct view. The rest of the venue (bleachers, etc.) should receive at least 50 lux (5 footcandles) of light.” Under such specifications, uniform illumination sufficient for imaging analysis is not readily available. As a result, automatic image analysis of balls at table edges can result in inaccurate ball identification or insufficiently accurate ball location determinations. As a result, rapid automatic image analysis of ball movement and location cannot be accomplished efficiently and cost effectively with conventional billiard table lighting.

Additionally, there is a need to have one video recording of game play that can simultaneously allow for accurate ball movement and position, such as may be accomplished by image analysis in the plane of the table surface but at the same time, in the same video record provide the video views of player activity around the near periphery of the table.

SUMMARY

Generally speaking, pursuant to these various embodiments, a billiard table top lighting apparatus is described that provides substantially uniform lighting of a billiard table surface, but also optionally includes as an integrated component with multiple devices for recording and viewing game play embedded in the light structure itself. In one form, the lighting apparatus includes a frame supporting lights that are non-centrally placed such that no lights are supported above a middle portion of the billiard table surface. Other lighting configurations are possible.

So configured, the frame can have an attractive profile and further include additional items above the billiard table surface to enable a variety of other features. For example, the frame may support one or more cameras, one or more motion sensors, one or more microphones, and/or one or more computing devices to enable any of a variety of innovative features. Such features could include automatic game play recording from one or more perspectives by multiple cameras, automatically reconstructed, merged video track storage from the multiple camera views for replay, review, and analysis, automatic lighting and dimming control, control of the apparatus from any mobile device, and the like. These and other benefits may become clearer upon making a thorough review and study of the following detailed description.

DETAILED DESCRIPTION

Referring now to the drawings, and in particular toFIG. 1, an illustrative lighting apparatus100for lighting a billiard table surface110and that is compatible with many of these teachings will now be presented. The billiard table surface110is supported by a table115and is bounded by cushions117that define the edges of the billiard table surface. The lighting apparatus100includes a frame120configured to support one or more lights130at a spaced distance Z above the billiard table surface110. To enable certain of the game play image capture and analysis features, the one or more lights130include a light source or sources mounted in the frame120in a configuration to provide substantially uniform illumination of the billiard table surface110. Although the examples discussed in this disclosure relate to various peripheral lighting approaches, it is contemplated that any lighting arrangement for providing uniform illumination can be applied, such as using in any combination strategically placed lights, lensing, reflectors, shades, diffusers, and the like.

The concept of uniform illumination will be discussed further with respect toFIGS. 2-6. Different sets of illumination measurements were taken to confirm the uniform illumination of the disclosed lighting approach.FIG. 2illustrates how the table markers labeled1,2, and3at the foot end210and1through7at the table side220were used to establish measurement points230on the table surface where projections240from the table markers intersected. An Extech foot-candle/lux light meter was placed at each of the measurement points on a standard billiard table illuminated by a typical overhead, centered billiard table light manufactured by Diamond Billiards with no other ambient light sources on. The illumination measurements at the measurement points illustrated inFIG. 2are listed in both foot-candles and lux in Table 1 below. Table 1 further notes the average illumination (AVG), the coefficient of variation of the measurements (CV) (which is a normalized measure of dispersion of a probability distribution or frequency distribution and is defined as the ratio of the standard deviation to the mean or average), and the standard deviation of the measurements (STD).

Table 2 lists illumination measurements taken at the same measurement points of a billiard table using a light configured in accord with the approaches ofFIGS. 1 and 7-12with no other ambient light sources on.

FIG. 3illustrates a similar approach to establishing illumination measurement points X except to expand the points X to include the intersection of the marker projections with the cushions117to determine the lighting uniformity for more of the billiard table surface110. The illumination for the table having the standard Diamond Billiards light was re-measured using these measurement points with results shown in Table 3 below and inFIG. 4. Also, measurements were made on a number of other pool tables with other commonly used center table lights in two commercial pool establishments in the Chicago area with similar measurement results to those illustrated in Table 3 below.

The billiard table lighted with the prototype light for which first measurements are listed above also had its illumination re-measured but with a dimmer set to adjust the average lighting output to closer to the commercially available Diamond Billiard light using the measurement points ofFIG. 3. The results are listed in Table 4 below and are illustrated inFIG. 5.

To further illustrate this approach, a commercial lighting, ray tracing, simulation program (Photopia software from LTI Optics) was used to generate simulated illumination levels for a billiard table lit according to that of Table 4 andFIG. 5. The simulation results listed below in Table 5 and illustrated inFIG. 6are largely consistent with the physical illumination measurements separately taken.

Thus, the arrangements described herein demonstrate substantially uniform illumination of the billiard table surface of between about 50 and 115 foot-candles. In short, instead of a variance of 15% when using only mid-table readings or 28% when including illumination at the cushions, the disclosed lighting apparatus has a coefficient of variation of only 3% using the mid-table readings and only 7% when using illumination at the cushions. In short, the cushion to cushion overall illumination uniformity for the disclosed lighting apparatus was better than the mid-table illumination uniformity for a standard billiard table light. Accordingly, substantially uniform illumination for the table will include an illumination coefficient of variation of about 14% or less, more preferably 10% or less, from cushion to cushion measured at the locations described above as shown inFIG. 3.

Turning back to the example lighting approach measured above, and with reference toFIGS. 7-8, a frame120is mounted above the billiard table surface with one or more light sources130mounted in the frame120. The light source or sources may include light sources130mounted in the frame120in a configuration around a periphery of the billiard table surface110. The periphery will typically correspond to an area projected above the cushions or edges of the billiard table surface110. For example, the frame120can be configured to mount the one or more lights130within a given horizontal distance from edges of the billiard table110such as within ten inches of the vertical projection of the cushion's117edge, and more preferably within five inches. In one approach, the one or more lights are non-centrally placed such that no light sources are placed approximately directly above a middle portion of the billiard table surface110. The middle portion of the billiard table surface110will be generally understood to correspond to the area of the surface110between projections of the foot end210markers2and4ofFIG. 3from about marker2to about marker8of the side table220markers ofFIG. 3.

In one approach, the frame120is configured to mount pairs710of the one or more lights130in a generally perpendicular configuration above each corner of the billiard table surface110. In a further aspect, the frame120is configured to mount two of the one or more lights130generally equally spaced along each long side of billiard table surface110as illustrated inFIG. 1. The frame120may further include a middle frame portion720spanning across a middle portion160of the frame120corresponding to a middle portion of the billiard table surface110when mounted above the billiard table surface110.

The light sources130can comprise any suitable light source. In the illustrated examples, the light sources130each include a set of light emitting diode (LED) lights930mounted in a linear configuration such as illustrated inFIG. 9. In this example, the light source130is an off the shelf light bar manufactured by PHILIPS having a mounting surface940on which the LED lights930are mounted, here in a linear configuration only, although additional LED lights could be mounted in addition to those set in a linear configuration. Mounting holes950in the mounting surface940facilitate mounting of the light source130to the frame120. Electrical connectors960allow for wired connections to a power source or driver or to another light source130such that one power source or driver can power and drive more than one light source130.

FIG. 10illustrates an example driver or power circuit for powering the light sources130of the lighting apparatus, in this example, a PHILIPS XITANIUM 75W 0.7-2.0 A 0-10V dimming device. Here, the light sources130are divided into two groups with each group having its own otherwise identical circuit1010and1012. Each circuit in turn includes a power source1020, resistor1030, and rheostat1040connected in series with the light sources130. The rheostat1040controls the amount of current flowing through the LED light sources130thereby controlling the brightness of the lights as a group. Alternatively, and as is well known in the art, a pulse width modulation (PWM) circuit may be used to modulate the light intensity.

Generally speaking, the frame120further supports shades, reflectors, or the like to direct light from the light sources130to the billiard table surface and protect the players' eyes from direct exposure to the LEDs. Alternatively or in addition, one or more reflectors and/or a diffuser element can be added to diffuse the light from the LEDs and provide a more uniform aesthetic. In the example design of the frame's1120internal portion illustrated inFIG. 11, the frame1120is in the shape of a troffer, i.e., a narrow inverted trough, serving as a light source support, reflector, and holder of diffusers. The troffer frame1120is substantially hollow in its inner structure, which is configured to support one or more lights by a top portion or upper support construction1124of the inner structure opposing a portion of the troffer frame closest to the billiard table surface when installed. Opposing sides of the troffer frame1120extend down from the top portion encasing the lights and support reflective surfaces. The illustrated example is constructed from extruded aluminum and has continuous t-slots along the length of the top surface and inside surface to provide attachment points for supports to mount the frame to room ceilings, and for the attachment of video cameras750, or other devices. The t-slots accommodate a t-slot nut and screw/bolt1122which slides along the t-slot track to aid various attachments. The frame's upper support construction1124supports the lighting elements (here an LED circuit board1140supporting LEDs1130) to face in the direction of the table surface. In this example, the LEDs1130are supported to face essentially straight down, in other words, such that the plane in which the LEDs1130are supports is essentially horizontal with the table surface although other arrangements are possible. Light from the LED's1130, however, emits in a variety of directions and can be distractingly bright in one's field of view.

To spread the light and allow a player to play without distraction from the LEDs1130, the frame1120supports one or more diffusers. In the illustrated example ofFIG. 11, a particular diffuser arrangement is illustrated that redirects light rays from the LED1130that would otherwise be absorbed by the frame or be directed in a manner to not hit the table, so as to adequately light the billiard table surface and still diffuse light from the LEDs1130, and so as to not be distracting to a player. In this example, a first mirrored diffuser1150is disposed between the light source or sources (such as LED1130) and an outer wall1126of the frame1120that faces away from the center of the billiard table surface. The back surface1155is mirrored and the rest of the thicker portion of the diffuser1150is constructed from a substance that diffuses the reflected light, both before and after reflection by the mirrored surface1155. The width of the diffuser1150is oriented essentially perpendicular with the billiard table surface so that the mirrored surface1155of the diffuser1150reflects light toward the center of the billiard table to help provide adequate lighting of the table surface. In one approach, this diffuser1150is a commercially available diffuser (Evonik Platinum Ice OM001 X1) and is 0.34 inch thick. For the aluminum extrusion of the design illustrated inFIG. 11, the mirrored diffusor is 1.8 inches from top to bottom in the cross section ofFIG. 11and, for a standard size 9 foot pocket billiard table, is 95.8 inches along the sides of the frame and is 45.8 inches along the ends of the frame. Other lengths are possible.

A bottom diffuser1160is supported to be disposed between the light source or sources (such as LED1130) and the billiard table surface. In this example, the bottom diffuser1160is a commercially available diffuser (Evonik Satin Ice OD002 DF) that is 0.08 inch thick and 2.125 inches wide. This diffuser1160diffuses the LED's1130light so that game players will not be distracted by the strong light that can emanate from individual LEDs. Instead, the observed light is diffused to provide a more uniform appearing light. This diffusing also spreads the light is a more uniform manner across the table surface. In addition, the bottom diffuser1160protects the LEDs1130from being struck by a cue.

A second mirrored diffuser1170is disposed between the light source or sources (such as LED1130) and an inner wall1128of the frame1120that faces toward the center of the billiard table surface. The inner wall1128may define a t-slot channel1129which optionally supports additional elements such as one or more cameras, motion sensors, or the lighting apparatus's middle section160. In this example, the second mirrored diffuser1160is a commercially available diffuser (Evonik Platinum Ice OM001 X1) that is 0.118 inch thick, 1.3 inches wide, and, for a standard size 9 foot pocket billiard table, is 95.8 inches along the sides of the frame and is 45.8 inches along the ends of the frame. Other lengths are possible. The second mirrored diffuser1170is disposed at an angle so that its mirrored surface1175reflects light generally toward both the bottom diffuser1160and the first mirrored diffuser1150to effect direction of more light at the billiard table surface through the bottom diffuser1160and via additional reflection off of the first mirrored diffuser1140. The diffusers1150,1160, and1170extend at least the length of the frame1120corresponding to a length of the frame along which the LED's130are supported although the diffusers1150,1160, and1170can extend any length along the frame. Typically, for example, the bottom diffuser1160will extend around the entire frame1120to provide a more uniform aesthetic for the frame1120.

An example implementation of the frame1120as installed above a billiard table is illustrated inFIG. 12, where the combined internal frame mirror and diffuser configuration smooth's out the light from the LEDs creating an aesthetic, even illumination on the table surface.

Referring again toFIGS. 7 and 8, the middle frame portion720may be configured to support a variety of other elements to add a variety of features to the lighting apparatus. For example, all or some of the electrical and/or computing elements needed to provide a variety of function can be mounted on the middle frame portion720top side to be not visible to the players. In one application, an A/C power strip722is mounted to the middle frame portion720to provide outlet power to various elements. An A/C switch724provides a master power switch for the lighting apparatus100.

In one aspect, the middle portion720of the frame120supports a middle camera730directed to record images of the billiard table surface110. The camera730may be mounted so that the image sensor is in a plane essentially parallel to the table surface and high enough above the table such that the camera lens projects the entire table surface area onto its image sensor. If necessary, depending on the camera lens and image sensor size, and to keep the distance Z inFIG. 1within a preferred height, the light path from the table to the camera may be deflected with a 45 degree mirror or similar optical arrangement, so that the image sensor is perpendicular to the table surface. In effect the light path distance is made adjustable within the middle portion of the frame720, horizontally, so as not to alter the preferred height of the frame above the table, but to still obtain a mapping of the complete table surface area onto the image sensor. In either configuration, one frame of video from middle camera730maps the entire table surface onto the image sensor for rapid and efficient image processing purposes. The image processing of the table surface single frames is made even more efficient by the controlled uniform illumination provided by the lighting apparatus.

FIGS. 13A-Bshow example images captured by the middle camera730. The lighting apparatus as further described herein may include a processing device745in operative communication to receive the images recorded by the middle camera730. Those skilled in the art will recognize and appreciate that such a processor device can comprise a fixed-purpose hard-wired platform or can comprise a partially or wholly programmable platform. All of these architectural options are well known and understood in the art and require no further description here. So configured, the processing device745, and as further described herein, can also be configured to determine whether balls on the billiard table surface110are in motion or non-motion based on the images recorded by the middle camera730, such as by using image by image comparison techniques or by using object identification of billiard balls on a frame by frame comparison basis to track the ball motion between frames. Then, the processing device745can automatically control a setting for the lighting apparatus100based at least in part on the balls being either in motion or non-motion. For example, in response to determining non-motion of balls on the billiard table surface, the processing device745effects stopping recording or provision of images from the middle camera730. Thus, the processing device745and middle camera730can work together to operate efficiently because there is no reason to continue to transmit or record images of the billiard table surface110when image does not change, i.e., in between shots by the players. In one approach, the processing device745effects provision of images from the middle camera730and from the end cameras750to reconstruct a real time or recorded video record from the combination of cameras as described herein.

Similarly, in one example, the processing device745is configured to detect particular images in the field of view of the middle camera730, in response to which, the processing device745can effect starting, stopping, or pausing recording or provision of images. For example, a card having a particular image could be placed on the billiard table surface so as to be in the middle camera's730field of view or a particular hand gesture may be made over the table surface. Depending on which particular image is detected, the processing device745may react in particular corresponding ways. For example, in response to detecting one particular image (such as a large red dot on a card or other unique indicator), the processing device745can automatically stop execution of the program relating to the monitored game. In this way, players can readily “pause” the program in the middle of game play because the processing device740can automatically restart the program in response to detecting removal of the particular image. Similarly, the processing device745may automatically start recording of a “new” game in response to detecting a particular image associated with that action such as a large green dot on a card.

In an additional aspect, an end portion of the frame120corresponding to a head or foot end portion of the billiard table surface110when mounted above the billiard table surface110can support an end camera750directed to record images of at least a portion of the billiard table surface110and an area surrounding a head or foot end portion of the billiard table surface110opposite that over which the end camera750is mounted.FIG. 14shows an example of an image captured by an end camera750. As illustrated inFIGS. 7 and 8, the frame120can support end cameras750at both the head and foot ends to capture images of both ends of the table. The end cameras750provide video images of player movement around the table and the player's approach to a shot. Such images can be useful for real time viewing, recording or transmission. As further described herein the images can also be used to construct composite video frames together with the table view image frames obtained simultaneously by the middle camera730.

Motion sensors760can be used to facilitate operation of the lighting apparatus. In this respect, the processing device740can be in operative communication with the motion sensor760to automatically control a setting for the lighting apparatus in response to detection of motion. In one example, the processing device740may be configured to power off the plurality of lights130automatically in response to the motion sensor's760failing to detect motion for a threshold set time period by electronically communicating with an A/C switch circuit724. Similarly, the processing device740may be configured to increase the lighting level by communicating with a pulse width modulation circuit1040, going from dimmed to a brightness sufficient to enable image capture and recording as described herein. In another example, the processing device740may be configured to provide images from a first camera in response to detecting motion from a first motion sensor and to provide images from a second camera in response to detecting motion from a second motion sensor. For instance, video or images will be recorded or transmitted from a camera oriented to capture images from an area from which motion is detected to ensure that the player movement is automatically recorded or transmitted.

In still another aspect, the processing device745may be configured to monitor sound captured by a microphone770to detect a strike sound having characteristics of a cue striking a billiard ball, and in response to detecting the strike sound, to automatically control a setting for the lighting apparatus100. The microphone770may be mounted to the frame120or be a part of another device (such as one of the video cameras) that is in communication with the processing device745. Furthermore, the processing device can be configured to, in response to detecting the strike sound, start recording or provision of images from the middle camera730mounted on the middle frame portion720of the frame120to automatically capture images of the moving balls.

The above elements can be combined in a variety of ways to provide many combinations of automated and/or remotely controlled features. One such feature is the ability to completely control the lighting apparatus and record images from the cameras730and750from mobile devices wirelessly communicating with the lighting apparatus100. Generally speaking, a processing device is configured to communicate with a user communication device, here the mobile device1510although other devices could be used, to provide images from the one or more cameras730and750disposed to capture images of the billiard table surface110and/or areas surrounding the billiard table surface110. In other approaches, the processing device may communicate directly with the user communication device. The processing device may then communicate with at least two cameras of the one or more cameras730and750to coordinate storage of the images or provision of the images to the user communication device1510. One such example arrangement is illustrated inFIG. 15, utilizing first and second processing devices740and745(Computer1and Computer2respectively inFIG. 15) where the first processing device740is operating the lighting apparatus to turn on, adjust the brightness, and start game play, and where the second processing device745is dedicated to communications with the cameras730and750to facilitate selection of the cameras from which individual video streams will be stored and/or provided to a user communication device1510. The second processing device745is in operative communication with the first processing device740via the wired connection742(such as an Ethernet or similar method) to receive commands with respect to starting and stopping the viewing or recording of images. Here, a generally available router device1520can coordinate wireless communication such as through WiFi between the mobile device1510and the processing device740of the lighting apparatus100. In this example, the processing device745operates a server that hosts a web page operated from the mobile device1510. One example of the web page is shown inFIG. 16. The server is assigned a local IP address by the router1520, and the IP address is displayed in the LCD panel1560. The mobile device1510interacts through the web page to control the light and allows the user to turn the light on or off by clicking the button1720. The web page interface also allows the user to adjust the light level by clicking on one of a series of bars1730in the interface. Clicking higher on the screen provides a higher light level that is indicated by the bar1740moving up. The web page ofFIG. 16allows control of the lamp on/off and intensity through a background program running on the processing device1641interfaced via the web page to either switch power on or off through the A/C switch724from the processing device1641to the lamp circuit, or in the case of adjusting the intensity, by controlling a pulse width modulation circuit interface to the lamp power supplies1020.

Additionally, the background program monitors the motion detectors760and adjusts the light intensity according to whether there is motion in the field of view of the motion detectors. As long as motion is present the lamp stays at the level set by the user. If there is no motion for a preset period, then the lamp automatically dims to a lower level using the pulse width modulation control. If there continues to be no motion for an additional preset period, the lamp turns off through the power on/off switch.

In another example of the various uses of the components of theFIG. 15, the second processor745is operated by the display screen, keyboard, and mouse1685to run a software program to display in real time (e.g., 30 frames per second) an automatically generated composite video that combines the output from all three video cameras730and750. The composite video may optionally be recorded for later retrieval and review. The video is a recording, reproducing, or displaying of visual images made digitally of a scene captured sequentially in time, such that they can be viewed as moving visual images, even if there is no apparent motion for certain periods. By one approach, the method of creating a video of billiard game play recorded simultaneously from multiple video cameras includes operating at least three independent image capture threads individually associated with separate cameras asynchronously in a same time interval. The independent image capture threads use shared memory resources and event done flags to communicate with each other. The independent image capture threads asynchronously capture individual image frames from the separate cameras. mage analysis of the captured individual image frames from the separate cameras to compare the individual image frames from a given camera of the separate cameras to determine which of the individual image frames are recording motion in the respective sequence of recordings from the respective ones of the separate cameras. Certain frames are chosen, displayed, and saved in a single video memory based on which of the separate cameras is recording motion. The method includes recording at specific time intervals, such as 30 frames per second, the chosen frames into a video file of the billiard game play. The chosen frames recorded into the video file comprise whatever is present at that instant in the single video memory.

Referring again to the example ofFIG. 15, in one example implementation, the video streams from the three cameras730and750are connected through USB ports to the second processing device745(labeled Computer2inFIG. 15). The video streams from the three video cameras are connected through USB ports to the processing device745, also labeled Computer2. In this example, the processing device745has an Intel quad-core CPU with hyper-threading, i.e., eight separate logical CPUs. The operating system for Computer2is an Ubuntu system. The eight logical CPUs can run simultaneously and asynchronously in a multi-threaded, multi-processor environment, such that a separate software thread can be running in each of the eight logical CPUs simultaneously. In the example shown inFIG. 15and further detailed inFIGS. 18A, 18B and 18C, and Table 6 below, each video stream is input to a separate software module running on its own thread, sharing the memory allocations listed in Table 6.

TABLE 6Shared Memory AllocationsCFB0Current Frame Buffer-Camera 0PFB0Previous Frame Buffer-Camera 0DIF0Result of difference comparison calculation between CFB0 andPFB0CFLG0Frame Capture Flag 0PFLG0ProcessorThread 0 Done FlagCFB1Current Frame Buffer-Camera 1PFB1Previous Frame Buffer-Camera 1DIF1Difference Comparison Calculation 1CFLG1Frame Capture Flag 1PFLG1ProcessorThread 1 Done FlagCFB2Current Frame Buffer-Camera 2PFB2Previous Frame Buffer-Camera 2DIF2Difference Comparison Calculation 2CFLG2Frame Capture Flag 2PFLG2ProcessorThread 2 Done FlagTHRSComparison threshold, to determine motion in Camera 0 viewAVGDIFRunning average motion difference calculation for videostream 0DFBDisplay Frame Memory BufferPIPPIP Image Memory BufferDFLGDirectorThread Done FlagCTIMECurrent time (in milliseconds) from system clockPTIMEPrior time, i.e. the time that the last video frame was put in DFMRFLGRecord FlagDONEProcess Done Flag, initialized to NO and set to YES whenProcess is exitedHardware Device AllocationsCAM0Camera 0 hardware image buffer-generated image available tocomputerCAM1Camera 1 hardware image buffer-generated image available tocomputerCAM2Camera 2 hardware image buffer-generated image available tocomputerFile Storage AllocationsAVI.avi file storage on system disk or other storage mediaWAV.wav file storage on system disk or other storage media

Turning now toFIGS. 18A, 18B, and 18C, eight flow diagrams are shown1810,1820,1830,1840,1850,1860,1870, and1880wherein each flow diagram represents a separate software thread. A main parent software process runs from the processing device745, which launches all of these threads asynchronously. However, they may share images and data through the use of shared memory resources allocated in the parent process. These shared resources are named and listed in TABLE 6 to enable clarity of presentation and understanding of the action of the different threads.

There are three image frame capture threads1810,1830, and1850receiving frame by frame image input from the three camera video streams. For example, in1810CaptureThread0receives video input from the middle camera730inFIG. 7. Each successive frame of video is available in that camera's internal hardware image buffer for 33.3 msec (at 30 frames/sec). At step1812, the thread moves the image frame from CAM0internally into the processing device's745process allocated memory CFB0for that video stream. At step1814, the thread sets its Frame Capture Flag CFLG0to1, which then allows the companion ProcessThread0to compute a difference metric DIF0between the current frame and the previous frame PFB0at1822. ProcessThread0then continues to copy the current image in CFB0to replace the image in the previous frame buffer PFB0at step1824and then sets its PFLG0to1, thus allowing the asynchronously running DirectorThread to use the DIF0value inFIG. 18Cto update the AVGDIF. At the same time ImageCapture threads1and21830and1850are operating similarly to acquire image frames from their respective video cameras750inFIG. 7, and ProcessorThreads1and21840and1860are operating to compute the difference metrics DIF1and DIF2respectively between their current and previous image frames. Thus, in this example, six threads, operating on six separate logical CPUs are operating in this manner to achieve image frame input from the three video streams.

At the same time that the capture threads are running to acquire images, two other threads, the DirectorThread1870and the WriterThread1880, are running to construct a composite video stream, combining the three separate video streams into one final composite stream. This composite video is displayed by the DirectorThread, and the DirectorThread1870determines which frames of video will be used to construct the composite video from the three inputs. If the record flag RFLG is set the composite video is also stored at 30 frames/sec, and an audio file is also recorded that is combined with the composite video stream at the end of the process. In this example the SoX Sound Exchange program running on the Ubuntu operating system is used to merge the recorded .wav files with the recorded .avi files created by the WriterThread.

The primary input, collected on a frame by frame basis, for the reconstructed composite video comes from camera0, which is the middle camera730inFIG. 7, and which provides an overview of the table surface. One of the purposes of the composite video is to obtain an accurate record of ball positioning prior to each shot and during the movement of the balls afterwards. However, there are periods after the balls stop moving where other interesting and informational aspects of the play are occurring. The reconstructed composite video captures both of these activities through the utilization of the 3 cameras and reconstructing one composite video record. As long as motion is occurring, in the middle camera730view, the other two end table video streams from cameras750are not included in the reconstruction. The determination of motion in this example is determined by monitoring the difference DIF0between successive frames from camera0, and computed at1822in ProcessThread01820. The Director thread computes AVGDIF, a running average of the last30sequential DIF0values, and compares this value to a fixed threshold THRS to decide if motion is occurring.

Other methods of determining the start of motion, such as using the striking sound of the cue stick hitting the cue ball (as described in U.S. provisional patent application 62/032,187 and included herein by reference), or by specifically tracking object ball motion, as is well known is the art, could also be utilized to determine or define which frames from the multiple cameras to include in a composite video.

Returning again to the DirectorThread1870, if all of PFLG0, PFLG and PFLG2are equal to 1 at1871, signifying that new image frames are in the current frame buffers CFB0, CFB1and CFB2, then the thread proceeds to Copy CFB0to DFB at1872. DFB is the memory buffer that displays the current image frame, and it will either be the current CFB0video frame without modification, or will later in the thread be overlaid in the corner with a PIP image. At1873the AVGDIF value is updated by averaging the current DIF0into the running average, and then at1874the updated AVGDIF is compared to THRS to decide if motion is present in the middle camera table view.FIGS. 13A and 13Billustrate the type of frames that are included in the case of detected motion. These figures are from a sequence of a video frames showing the darker “4 ball” being hit by the white cue ball, propelling the “4” ball towards the side pocket. Because the ball motion from these and prior sequential frames results in an AVGDIF value above the threshold THRS, these frames would simply be added to the reconstructed composite video stream to the exclusion of the end camera views.

If motion is not present at1874then the thread compares DIF1and DIF2at1875to see which end view has the most motion going on. The video stream having the most motion as reflected by the higher difference value DIF1or DIF2will be chosen to provide the end view frame to be reduced in size by ¼ at step1876or1877and overlaid in a corner of DFB at step1879to create a picture-in-picture reconstructed image frame. If the difference value DIF1for video stream1is greater that the difference value DIF2for video stream2, the PIP frame is constructed1870from video frames from video streams0and1. If the difference value DIF2for video stream2is greater that the difference value DIF1for video stream1, the PIP frame is constructed1876from video frames from video streams0and2. The resulting video frames correspond to those illustrated inFIGS. 17A and 17Bwhere the end view frame chosen is the one with the most motion. Thus, the operation of the DirectorThread is to determine if there is motion and choose the appropriate next sequential image frame for the displayed video. If there is no motion on the table, then the reconstructed video should show the activity around the periphery of the table, which is usually the player getting ready for the next shot. The PIP frame shows this, but still in the context of the overview of the balls on the table surface prior to motion stoppage.

Turning now to step1880inFIG. 18C, the WriterThread controls the synchronization of the process of recording the composite video record. The WriterThread is activated optionally by the operator of the apparatus and in that case the WriterThread is created and started by the DirectorThread. It also starts the audio recording during its initialization and starts a msec clock to measure the recorded frame time interval precisely. Because the audio signal is being recorded in real time, a composite video stream must be generated that corresponds exactly to the audio record with regard to its visual content. Thus, the three separate video camera streams can be started and operated asynchronously in CaptureThreads0,1and2, and their captured images can be subjected asynchronously to image processing in ProcessorThreads0,1, and2to compute a motion analysis metric. And then, the DirectorThread may further select asynchronously and if necessary add PIP frames for inclusion. However, the WriterThread without time variance must record the contents of the display buffer DFM at exactly 1/30thof a second to acquire and construct the composite video. Whatever happens to be in the DFM display at that precise time is written to the recorded video output file.

The control described above regarding the provided images can be effected in a number of alternative ways. During a time with no ball motion, the determination of which end camera view to display can be made using information from motion sensors disposed on the frame to detect motion in areas around the table corresponding to the respective fields of view of the respective end cameras. In another approach, one or more the processing devices can perform image analysis on the video feeds provided by the respective cameras to determine which camera is capturing the most motion. The determination of when to remove the overlaid video feed from an end camera can be made by detecting the sound of the cue striking a ball, image analysis of the video feed from the center camera, or a combination of both. For instance, sometimes the center camera may capture motion other than that of the balls on the table such as movement of a cue which can be confused with motion of the balls. Combining image analysis of the balls with sound detection of the striking cue allows the removal of the overlaid video frame in response to sound detection of the striking cue and maintaining removal of the overlaid video frame while detecting ball motion. The processing required to effect these actions can done on any combination of processing devices.

Referring toFIG. 19, another example process executed by the lighting apparatus to automatically provide image storage and/or display from different cameras based on game play conditions will be described. In this example, the processing device is in communication with a middle camera730inFIG. 7, to capture images of the table surface into video memory VM2, and cameras750at both ends of the frame to respectively capture images from opposite ends of the billiard table and first and second areas surrounding the respective ends of the table into video memories VM1or VM3respectively, and motion sensors760inFIG. 7(labeled MS1and MS2inFIG. 19) disposed to detect motion in the areas surrounding the opposing ends of the table, and a microphone. The cameras and motion detectors could be mounted over and/or capturing images of or sensing motion around sides of the table in addition to or in lieu of the cameras over the ends of the table.

First, the processing device wakes or resets with an initialization process1910. Then, assuming that there is no motion of the balls on the table, the processing device monitors for motion from the first area and the second area surrounding the billiard table surface through the use of one or more motion detectors or real time image analysis techniques described above. Then, the processing device determines at step1920with respect to detected motion from which camera images should be stored or displayed in video memory VM4(the output or display/recorded video memory). More specifically, in response to detecting motion from the first area (or in the case of motion in both areas, where the motion signal is higher in the first area), the processing device effects stopping storing or providing images from the second side/end camera into VM4, and effects storing or providing images1933from the first side/end camera to store or provide images from the first area into VM4. For example, this includes storing or providing video or a video file that can be played on a display. In response to detecting motion from the second area (or in the case of motion in both areas, where the motion signal is higher in the second area), the processing device effects stopping storing or providing images from the first side/end camera into VM4, and effects storing or providing images1936from the second side/end camera to store or provide images from the second area. Under either case (same procedure for both steps labeled1940), the processing device next monitors for a ball hit; by detecting motion, for example, by real time image analysis, and/or by the striking of a ball, for example, by detecting a strike sound having characteristics of a cue striking a billiard ball. In response to detecting a ball hit, the processing device effects stopping storing or providing images from the respective side/end camera into VM4and effects storing or providing images VM2into VM4at step1950from the middle camera′ memory VM2so that images of the field of view capturing the table and the balls' motion are captured.

The processing device then monitors1960for stoppage of the balls' motion, such as through use of one or more motion detectors or real time image analysis techniques described above. In response to detecting stoppage of motion of balls on the billiard table surface, the processing device effects stopping storing or providing images from the middle camera and effects storing or providing images from the one of the side/end cameras as discussed above. The processing device may determine1970that the game is finished either by tracking which balls are on the table with respect to the game being played or by receiving a user initiated signal that indicates completion of play. In response to determining that the game is finished, the image capture and other game process is ended1980.

In an additional alternative embodiment, the functionality or logic described inFIGS. 18A, 18B, 18C, and 19may be embodied in the form of code that may be executed in a separate processor circuit. If embodied in software, each block may represent a module, segment, or portion of code that comprises program instructions to implement the specified logical function(s). The program instructions may be embodied in the form of source code that comprises human-readable statements written in a programming language or machine code that comprises numerical instructions recognizable by a suitable execution system such as a processor in a computer system or other system. The machine code may be converted from the source code, etc. If embodied in hardware, each block may represent a circuit or a number of interconnected circuits to implement the specified logical function(s). Accordingly, a computer readable medium (being non-transitory or tangible) may store such instructions that are configured to cause a processing device to perform operations as described herein.