Position information acquisition device, position information acquisition method, recording medium, and position information acquisition system

A position information acquisition device for acquiring position information of a position acquisition target arranged in a space includes a processor configured to detect light that is based on identification information included in-common in captured images that are images of the space captured from a plurality of shooting directions that are different from each other, acquire a three-dimensional position in the space of the position information acquisition target identified by the identification information, based on detection positions of the detected light in the captured images, and position information of image capturing devices during capturing performed by the image capturing devices, acquire reliability degree information of the acquired three-dimensional position of the position information acquisition target, based on information relating to an imaging state of each image capturing device during capturing of the captured images, and store the acquired the reliability degree information in a storage.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of Japanese Patent Application No. 2018-224457, filed on Nov. 30, 2018, and Japanese Patent Application No. 2019-105599, filed on Jun. 5, 2019, of which the entirety of the disclosures is incorporated by reference herein.

FIELD

The present disclosure relates to a position information acquisition device, a position information acquisition method, a recording medium, and a position information acquisition system.

BACKGROUND

As described in International Publication No. WO2005/124687, technology is known heretofore that identifies three-dimensional positions of multiple markers by using multiple cameras to image markers.

SUMMARY

According to the present disclosure, a position information acquisition device for acquiring position information of a position acquisition target arranged in a space includes a processor configured to

detect light that is based on identification information included in-common in captured images that are images of the space captured from a plurality of shooting directions that are different from each other,

acquire a three-dimensional position in the space of the position information acquisition target identified by the identification information, based on detection positions of the detected light in the captured images, and position information of image capturing devices during capturing performed by the image capturing devices by which the images are captured,

acquire reliability degree information of the acquired three-dimensional position of the position information acquisition target, based on information relating to an imaging state of each image capturing device of the image capturing devices during capturing of the captured images, and

store the acquired the reliability degree information in a storage.

In the present disclosure, a position information acquisition method for acquiring position information of a position acquisition target arranged in a space includes:

detecting light that is based on identification information included in-common in captured images that are images of the space captured from a plurality of shooting directions that are different from each other;

acquiring a three-dimensional position in the space of the position information acquisition target identified by the identification information, based on detection positions of the detected light in the captured images, and position information of image capturing devices during capturing performed by the image capturing devices by which the images are captured;

acquiring reliability degree information of the acquired three-dimensional position of the position information acquisition target, based on information relating to an imaging state of each image capturing device of the image capturing devices during capturing of the captured images; and

storing the acquired the reliability degree information in a storage.

In the present disclosure, a non-transitory computer-readable recording medium records a program that causes a computer included in a position information acquisition device for acquiring position information of a position acquisition target arranged in a space to function as:

detection means for detecting light that is based on identification information included in-common in captured images that are images of the space captured from a plurality of shooting directions that are different from each other;

means for acquiring a three-dimensional position in the space of the position information acquisition target identified by the identification information, based on detection positions of the detected light in the captured images, and position information of image capturing devices during capturing performed by the image capturing devices by which the images are captured,

means for acquiring reliability degree information of the acquired three-dimensional position of the position information acquisition target, based on information relating to an imaging state of each image capturing device of the image capturing devices during capturing of the captured images, and

means for storing the acquired the reliability degree information in a storage.

In the present disclosure, a position information acquisition system includes:

image capturing devices configured to capture images of a space from a plurality of shooting directions that are different from each other; and

a position information acquisition device configured to acquire position information of a position acquisition target arranged in the space.

The position information acquisition device includes a processor configured to

detect light that is based on identification information included in-common in captured images that are images captured by the image capturing devices,

acquire a three-dimensional position in the space of the position information acquisition target identified by the identification information, based on detection positions of the detected light in the captured images, and position information of the image capturing devices during capturing performed by image capturing devices by which the images are captured,

acquire reliability degree information of the acquired three-dimensional position of the position information acquisition target, based on information relating to an imaging state of each image capturing device of the image capturing devices during capturing of the captured images, and

store the acquired the reliability degree information in a storage.

DETAILED DESCRIPTION

A visible light communication system is described below as a position information acquisition system according to an embodiment of the present disclosure with reference to drawings.

FIG. 1illustrates configuration of the visible light communication system. As illustrated inFIG. 1, the visible light communication system1is configured to include: apparatuses100a,100b, and100cabbreviated below appropriately as the “apparatus100” when not limited to a given apparatus100a,100b, or100c) arranged within a space500; and a server200corresponding to a position information acquisition device.

A second marker102ais attached to the apparatus100a; a second marker102bis attached to the apparatus100b; a second marker102cis attached to the apparatus100c; and these markers are abbreviated below appropriately as the “second marker102” when not limited to a given second marker102a,102b, or102c. Cameras201a,201b,201c, and201dcorresponding to image capturing devices are attached to the server200; and are abbreviated below appropriately as the “camera201” when not limited to a given camera201a,201b,201c, or201d. Moreover, first markers300a,300b,300c,300d, and300eare arranged within the space500; and are abbreviated below appropriately as the “first marker300” when not limited to a given first marker300a,300b,300c,300d, or300e. The first marker300and the second marker102each include a non-illustrated light emitting diode (LED). The second marker102corresponds to a position information acquisition target.

In the present embodiment, the second marker102attached to the apparatus100transmits information by emitting light corresponding to information of various types of transmission targets such as a state of the apparatus100. The server200demodulates changes in emitted-light color in images of light obtained by time-series type capturing by the cameras201, and acquires information emitted by the second marker102.

In the present embodiment, positions and imaging directions of the cameras201ato201dare initially unknown. Thus prior to the acquiring by the server200of information such as the state of the apparatus100, firstly the positions (arrangement positions) and the imaging directions of the cameras201ato201din the space500that is a three-dimensional space are calculated by the server200on the basis of the positions, as two-dimensional coordinate information, of each representation of the first markers300a,300b,300c,300d, and300ein the images captured by the cameras201ato201d. The server200generates a conversion matrix for converting positions (two-dimensional coordinate information) of the images of the first markers300in the images obtained by capturing into positions (arrangement positions) within the space500.

FIG. 2illustrates an example of configuration of the server200. As illustrated inFIG. 2, the server200includes a controller202, an image input unit204, a storage205, an operation device206, a display207, and a communication device208. The server200is attached through lines to the cameras201ato201d.

The camera201aincludes a lens203a; the camera201bincludes a lens203b; the camera201cincludes a lens203c; the camera201dincludes a lens203d; and the lenses are abbreviated below appropriately as the “lens203” when not limited to a given lens203a,203b,203c, or203d. The lens203includes components such as a zoom lens. The lens203moves due to a zoom control operation from the operation device206and focal control by the controller202. The view angle and/or optical image captured by the camera201is controlled by movement of the lens203.

The cameras201ato201deach include multiple light-receiving elements orderly disposed in a two-dimensional array on a light-receiving surface. The light-receiving element is an image capturing device such as a charge coupled device (CCD), a complementary metal oxide semiconductor (CMOS), or the like. Each of the cameras201ato201dcaptures an optical image from light entering through the lens203(receives light) in the view angle of a prescribed range on the basis of a control signal from the controller202, and generates a frame by converting the image signal within the imaged view angle to digital data. Further, each of the cameras201ato201dcontinues over time with the imaging and the frame generation, and outputs the continuing frame to the image input unit204within the server200.

On the basis of the control signal from the controller202, the frame (digital data) output from the camera201is input to the image input unit204.

The controller202is a processor that includes components such as a central processing unit (CPU). The controller202controls various types of functions with which the server200is equipped by executing software processing in accordance with programs, that is, programs for achieving operations of the server200illustrated in a below-describedFIG. 3, stored in the storage205.

The storage205is random access memory (RAM) and/or read only memory (ROM), for example. The storage205stores various types of information, such as programs, used in operations such as control by the server200.

The operation device206includes components such as a numeric keypad and/or function keys, and is an interface that is used for input of operational content of a user. The display207includes a display such as a liquid crystal display (LCD), a plasma display panel (PDP), an electro-luminescence (EL) display, or the like. The display207displays an image in accordance with an image signal output from the controller202. The communication device208is a device such as a local area network (LAN) card. Under control by a communication controller242, the communication device208performs communication with an external communication device.

The controller202includes an image processor231, a camera position-imaging direction calculator232, a matrix generator234, a light-emission position acquirer236corresponding to calculation means, an imaging state acquirer238corresponding to information acquisition means, a reliability degree information acquirer240corresponding to reliability degree information acquisition means, and the communication controller242.

The image processor231adjusts image quality and/or image size by performing peripheral darkening correction and/or distortion correction to allow display of, as a through image on the display207, a frame, that is, digital data, output from each of the cameras201and input to the image input unit204. The image processor231has functions for, upon input of a control signal based on the command operation from the operation device206, encoding and file generation from the optical image within the view angle of the camera201or the optical image in the display range displayed on the display207at the time of the command, the encoding and file generation using a compression-encoding format such as the joint photographic experts group (JPEG) format. The camera position-imaging direction calculator232calculates the position (two-dimensional coordinate information) of representations of the first marker300in each of the images captured by the cameras201ato201d. Here, the arrangement positions (three-dimensional coordinate information) of the first markers300a,300b,300c,300d, and300ewithin the space500are assumed to be previously known. Each of the first markers300a,300b,300c,300d, and300eemits light that changes cyclically in a pattern of three colors red (R), green (G), and blue (B) modulated with an identification (ID) that enables unique identification of the marker.

The camera position-imaging direction calculator232, from among the cameras201ato201d, sets combinations of cameras as pairs of cameras201, that is, camera pairs. Six patterns (six sets) result from making combinations of pairs of the cameras201freely from among 4 cameras201.

The camera position-imaging direction calculator232detects light of the cyclical three-colored patterns included in various images captured by the cameras201ato201d. Further, the camera position-imaging direction calculator232attempts detecting the ID corresponding to the pattern of light emitted in these three colors, and attempts demodulating into the ID. The storage205stores the arrangement position in association with the ID for each of the first marker300a,300b,300c,300d, and300e.

Further, the camera position-imaging direction calculator232, for each camera pair, attempts detection of modulation light regions (pixel regions formed from an identified size and shape and having a luminosity value of at least a predetermined value) in which light is modulated with the same ID, from among both images captured by the pair of cameras201included in the camera pair. Thereafter, if the detection succeeds, the first marker300corresponding to the ID thereof is regarded to be detectable. For each camera pair, the camera position-imaging direction calculator232further recognizes a detection count of the first markers300.

Thereafter, for each of the camera pairs, the camera position-imaging direction calculator232sets an algorithm for calculation of the position (arrangement position) and imaging direction in the space500of the two cameras201included in the camera pair, in accordance with the count of the first markers300included in both of the images captured by the pair of cameras201included in the camera pair. The count of prepared algorithms depends on the count of first markers300included in the images, such as, for example, preparation of 5 algorithms in the case in which the count of the first markers300included in the images captured by the camera pair is 5, and preparation of 8 algorithms in the case in which the count is 8, and the prepared algorithms are stored in the storage205.

Thereafter, for each of the camera pairs, the camera position-imaging direction calculator232uses the set algorithm to calculate the arrangement positions and the imaging directions of the two cameras201included in the camera pair.

The algorithm is described below.FIG. 3illustrates an example of parallax obtained from images captured by the cameras201. Moreover,FIG. 4illustrates an example of calculation of the arrangement positions and the imaging directions of the camera201.

As illustrated inFIG. 3, in the case of capture of the same first marker300cby two cameras201(in this case, cameras201aand201b) included in the camera pair, the arrangement positions of the camera201aand the camera201bare different from each other, and thus parallax S occurs between a position (two-dimensional coordinate information) of a representation251aof the first marker300cin the image of an imaging plane250acaptured by the camera201aand a position (two-dimensional coordinate information) of a representation251bof the first marker300cin the image of an imaging plane250bcaptured by the camera201b.

Moreover, a distance calculation formula D=B×F/S is established in which, as illustrated inFIG. 4, F (same value) is taken to be a distance to the focal position from the imaging plane250aof one of the cameras201(camera201ain this case) among the two cameras201(cameras201aand201bin this case) included in the camera pair, F (same value) is taken to be a distance to the focal position from the imaging plane250bof the other camera201(camera201bin this case), B is taken to be a distance between the arrangement positions of the camera201aand the camera201b, D is taken to be a shortest distance between the first marker300cand a straight line interconnecting the focal position of the camera201aand the focal position of the camera201b, and S is taken to be the parallax between the position of the representation251aand the position of the representation251bof the first marker300cobtained by virtual overlapping of the imaging plane250aand the imaging plane250b. In this formula, F and S are taken to be previously known constants.

In the present embodiment, the camera position-imaging direction calculator232sets up the respective distance calculation formula for each of the detected first markers300, rather than just setting up the distance calculation formula for the first marker300c. Further, the camera position-imaging direction calculator232calculates the arrangement positions and the captures directions of the two cameras201included in the camera pair, on the basis of the set distance calculation formula and the arrangement positions of the first markers300measured beforehand.

Specifically, the camera position-imaging direction calculator232determines the relative arrangement positions and the imaging directions of the two cameras201from a combination of positions (Xga1, Yga1) of the representation251aof the first marker300included in the image captured by one of the cameras201of the two cameras201included in the camera pair and positions (Xgb1, Ygb1) of the representation251bof the first marker300included in the image captured by the other camera201.

Next, the camera position-imaging direction calculator232refers to the IDs of the first markers300ato300eto read the arrangement positions of the first markers300ato300estored in the storage205, and uses such read arrangement positions to calculate the arrangement positions and the imaging directions in the space500of the two cameras201included in the camera pair. Then the matrix generator234determines a conversion matrix capable of conversion to the arrangement position (position information defined in three-dimensional space coordinates) of the first marker300in the space500, from the combination of the position (two-dimensional coordinate information) of the representation of the first marker300included in the image captured by one of the cameras201and the position (two dimensional coordinate information) of the representation of the first marker300included in the image capture by the other of the cameras201, based on the calculated arrangement positions and the imaging directions of the one camera201and the other camera201. The conversion matrix is determined for each pair of cameras.

The second markers102a,102b, and102cemit light that changes cyclically in a three-color pattern of red (R), green (G), and blue (B) modulated with the ID that enables unique identification of the marker.

After the determination of the conversion matrixes for each of the camera pairs, the light-emission position acquirer236detects light of the cyclical pattern of three colors included in the various images captured by the cameras201ato201d. Further, the light-emission position acquirer236attempts detection of the IDs, and demodulation to the IDs, corresponding to the patterns of such three-colored light-emission. In the case in which detecting the same ID from both of the images captured by the two cameras201included in the camera pair is possible, the light-emission position acquirer236regards the second marker102corresponding to such an ID to be detectable.

Thereafter, for each of the camera pairs, the light-emission position acquirer236acquires a position (Xga2, Yga2) of a representation of the second marker102in the imaging plane of one of the cameras201among the two cameras201included in the camera pair and a position (Xgb2, Ygb2) of a representation the second marker102in the imaging plane of the other cameras201. Further, the light-emission position acquirer236acquires the combination of the positions (Xga2, Yga2) and (Xgb2, Ygb2) of both representations, and uses the conversion matrix to acquire the arrangement position (Xk2, Yk2, Zk2) of the second marker102in the space500.

The aforementioned processing is a case in which arrangement positions corresponding to each of multiple camera pairs are acquired by use of a single second marker102. In such a case, reliability degree information, that is, likelihood information, of the arrangement position within the space500is acquired for the second marker102for each of the camera pairs. The acquiring of the reliability degree information is described below.

For each camera pair, the imaging state acquirer238sets the reliability degree relating to the position of the representation of the second marker102in the acquired image from the imaging plane of one camera201among the two cameras201included in this camera pair such that the reliability degree is set higher as the position of the representation approaches the center of the image, and is set lower with increased distance of the position of the representation from the center. In the same manner, the imaging state acquirer238set the reliability degree relating to the position of the representation of the second marker102in the acquired image from the imaging plane of the other camera201such that the reliability degree is set higher as the position of the representation approaches the center of the image, and is set lower with increased distance of the position of the representation from the center. Such setting processing is related to distortion correction processing in the image processor231; this distortion correction processing lessens correction intensity of distortion correction with increased closeness to the center in the imaging plane; and this distortion correction processing increases the correction intensity of distortion correction with increased closeness to the periphery. Therefore, as the position of the representation of the second marker102approaches the periphery, positional displacement due to distortion correction occurs that decreases the reliability degree.

Due to the aforementioned processing, reliability degree information (image position reliability degree information) B1is acquired relating to the image position with respect to one second marker102under consideration for one of the cameras201among the two cameras201included in the camera pair, and reliability degree information (image position reliability degree information) B2is acquired relating to the image position with respect the one second marker102under consideration for the other camera102among the two cameras201included in the camera pair.

Moreover, the imaging state acquirer238refers to multiple frames (images) acquired by consecutive capture by one of the cameras201among the two cameras201included in the camera pair, and calculates movement velocity of the second marker102on the basis of change of the position of the second marker102.

FIG. 5illustrates an example of the velocity calculation. For example, the position of the representation of the second marker102in a frame F1among multiple frames F1, F2, . . . Fn captured consecutively along a time direction t is “710a”, and the position of the representation of the second marker102in the frame Fn is “710n”. A case is considered below in which L is a distance between the positions710aand710nof the representation in a frame Fx that superimposes the frames F1and Fn on each other. In such a case, as a result of a comparison between sizes of the representation of the second marker102in both frames versus a size of the second marker102that is previously known and stored beforehand in the storage205, the imaging state acquirer238can calculate the movement velocity of the second marker102on the basis of the distance L and a frame rate. Further, the imaging state acquirer238sets the reliability degree of such movement velocity higher with increased slowness of the movement velocity, that is, with decreased distance L.

Similarly, the imaging state acquirer238calculates the movement velocity of this second marker102from the distance L between the positions of the representations of the second marker102in the images captured consecutively by the camera201that is the other camera201among the two cameras201included in the camera pair. Further, with respect to the reliability degree relating to such movement velocity, the imaging state acquirer238sets the velocity reliability degree higher with increased slowness of the movement velocity, that is, with decrease in the distance L.

The aforementioned processing is executed due to the possibility that the arrangement position changes, even when the second marker102is fixed at the same arrangement position for a long period, since the view angle of the camera201may change slightly and/or the second marker102may move slightly. Specifically, reliability degree information (velocity reliability degree information) C1relating to the movement velocity with respect to the second marker102is acquired concerning one of the cameras201among the two cameras201included in the camera pair, and reliability degree information (velocity reliability degree information) C2relating to the movement velocity with respect to the same second marker102is acquired concerning other camera201.

Moreover, the imaging state acquirer238partitions the space500into multiple regions.FIG. 6is an example of the spatial partitioning. InFIG. 6, 9regions, that is, partitioning regions,501a,501b,501c,501d,501e,501f,501g,501h, and501i(abbreviated below appropriately as the “partitioning region501” when not limited to a given partitioning region501ato501i) are formed by dividing the image longitudinally into three equally dimensioned portions and dividing the image laterally into three equally dimensioned portions.

Further, on the basis of the relative positional relationship between the arrangement position of one of the second markers102under consideration and the arrangement positions of the two cameras201included in the camera pair, the imaging state acquirer238acquires reliability degree information D, that is, arrangement position reliability degree information, relating to the arrangement position of this second marker102.

Specifically, due to the ability to capture a larger representation of the second marker102, the imaging state acquirer238sets the arrangement position reliability degree higher with increased closeness to the arrangement position of the two cameras201included in the camera pair. For example, in the case in which the camera201aand the camera201dare arranged as illustrated inFIG. 6, the arrangement position degree is set a high value when the second marker102is present in the partitioning region501a,501b, or501c; the arrangement position reliability degree is set to an intermediate value when the second marker102is present in the partitioning region501d,501e, or501f; and the arrangement position reliability degree is set low when the second marker102is present in the partitioning region501g,501h, or501i.

Moreover, for each pair of cameras, the reliability degree information acquirer240calculates an arrangement position, that is, position information defined by three-dimensional space coordinates, of the first marker300within the space500, for the representations of the first marker300captured by both of the cameras201included the camera pair, by using the conversion matrix corresponding to the camera pair in combination with the position (two-dimensional coordinate information) captured by one of the cameras201and the position (two-dimensional coordinate information) captured by the other camera201. Further, the reliability degree information acquirer240calculates an error between the calculated arrangement position of the first marker300within the space500and the arrangement position (previously known information) of the first marker300stored in the storage205. Further, the reliability degree information acquirer240sets an error reliability degree information A so as to increase in reliability degree (error reliability degree) concerning the error with decrease in the error.

Thereafter, for each camera pair, the reliability degree information acquirer240calculates reliability degree information (reliability degree information of the second marker102) N concerning the arrangement position calculation for one second marker102under consideration by use of the image position reliability degree information B1and B2, the velocity reliability degree information C1and C2, the arrangement position reliability degree information D, and the error reliability degree information A acquired in the aforementioned processing. For example, the calculation is by a formula N=A×(B1+B2+C1+C2+D).

Thereafter, the light-emission position acquirer236calculates the arrangement position of the second marker102within the space500. During this calculation, sometimes the arrangement positions from multiple pairs of cameras are calculable for one second marker102under consideration. In such a case, for the one second marker102under consideration, the light-emission position acquirer236compares the reliability degree information N acquired for each camera pair. Then the light-emission position acquirer236selects the camera pair corresponding to the reliability degree information N that is highest.

Thereafter, the light-emission position acquirer236acquires the position (two-dimensional coordinate information) of the representation of the second marker102in the image captured by one camera201among the selected camera pair and the position (two-dimensional coordinate information) of the representation of the second marker102in the image captured by the other camera201. Further, the light-emission position acquirer236uses the conversion matrix and the combination of the positions of these two representations to calculate the arrangement position (position information defined by three-dimensional space coordinates) of the second marker102within the space500.

Operation of the server200is described below with reference to flowcharts.FIG. 7is a flowchart illustrating an example of reliability degree information acquisition by the server200. The operation illustrated inFIG. 7is performed for each camera pair, and for each first marker300captured by both of two cameras included in the camera pair.

The two cameras201included in the one camera pair capture the same first marker300, and the first marker300is identified on the basis of the acquired ID (step S101).

Next, the imaging state acquirer238acquires the image position degree reliability information B1such that the image position reliability degree increases, for the position of the representation of the second marker102in the image captured by one of the cameras201among the two cameras201included in the camera pair, with increased closeness to the center of the captured image, and acquires the image position degree reliability information B2such that the image position reliability degree increases, for the position of the representation of the second marker102in the image captured by the other camera201, with increased closeness to the center of the captured image (step S102).

Thereafter, on the basis of each of the images captured consecutively by one of the cameras201among the two cameras201included in the camera pair, the imaging state acquirer238calculates the movement velocity of the second marker102, and acquires the velocity reliability degree information C1such that the velocity reliability degree increases with increased slowness of the movement velocity. In a similar manner, on the basis of each of the images captured consecutively by the other camera201, the imaging state acquirer238calculates the movement velocity of the second marker102, and acquires the velocity reliability degree information C1such that the velocity reliability degree increases with increased slowness of the movement velocity (step S103).

Thereafter, on the basis of the positional relationship between the arrangement position of the two cameras201included in the camera pair and the arrangement position of the second marker102, the imaging state acquirer238acquires the arrangement position reliability degree information D such that the arrangement position reliability degree increases with increased closeness of the arrangement position of the second marker102to the arrangement position of the two cameras201included in the camera pair (step S104).

Thereafter, concerning the representation of the first marker300captured by both of the two cameras201included in the camera pair, the reliability degree information acquirer240uses the conversion matrix corresponding to the camera pair and the combination of the position obtained by capture by one of the cameras201and the position obtained by capture by the other camera201to calculate the arrangement position of the first marker300within the space500. The reliability degree information acquirer240calculates the error between the calculated arrangement position of the first marker300within the space500and the arrangement position (previously known information) of the first marker300stored in the storage205, and acquires the error reliability degree information A such that the error reliability degree increases with increased smallness of the error (step S105).

Further, the reliability degree information acquirer240acquires the reliability degree information N of the second marker102by using the acquired image position reliability degree information B1and B2, the velocity reliability degree information C1and C2, the arrangement position reliability degree information D, and the error reliability degree information A (step S106).

FIG. 8is a flowchart illustrating an example of acquisition processing of the arrangement position of the second marker102by the server200. Multiple cameras201capture the second marker102within the space500(step S201).

Thereafter, for a single second marker102, in the case in which multiple camera pairs exist in which both of the two cameras201capture the second marker102, the light-emission position acquirer236selects the reliability degree information that is highest among the reliability degree information of the second marker102acquired by each of these multiple camera pairs. Further, the light-emission position acquirer236selects the camera pair that corresponds to such selected reliability degree information (step S202).

Thereafter, the light-emission position acquirer236acquires the position of the representation of the second marker102in the image captured by one camera201among the selected camera pair and the position of the representation of the second marker102in the image acquired captured by the other camera201. Further, the light-emission position acquirer236uses the conversion matrix and the combination of the two acquired positions to calculate the arrangement position of the second marker102(step S203).

Thereafter, the light-emission position acquirer236determines whether the arrangement position is calculated for all the second markers102captured in step S201(step S204). If the arrangement positions are calculated for all of the second markers102(YES in step S204), the sequence of processing ends. Moreover, if a second marker102exists for which the arrangement position is not calculated (NO in step S204), the operations of step S202and beyond are repeated.

In the present embodiment in this manner, for each of the camera pairs, the server200acquires the reliability degree information concerning the calculation of the arrangement position the second marker from the positions of the second marker102in the images captured by both of the two cameras201included in the camera pair. Further, the server200calculates the arrangement position of the second marker102on the basis of the images captured by the camera pair. At this time, in the case in which the arrangement position of the second marker102is calculable for each of the camera pairs by capturing the second marker102by multiple camera pairs, the server200selects the camera pair having the highest reliability degree of the second marker102, and calculates the arrangement position of the second marker102on the basis of the images captured by such a pair of cameras. Due to such operation, the arrangement position of the second marker102is calculable on the basis of the camera pair having a high reliability degree of the second marker102, and accuracy of the calculation can be improved.

Specifically, the server200acquires the image position reliability degree information such that the image position reliability degree of the image position of the second marker102in the captured image increases as the image position of the second marker102approaches the center of the image. Due to such operation, the reliability degree of the second marker102can decrease with increased distance of the image position of the second marker102from the center of the image, and reliability degree information can be acquired that is suitable in accordance with the characteristic of the image that is increased distortion with increased separation from the center.

Moreover, the server200calculates the velocity of the second marker102on the basis of the captured images, and acquires the velocity reliability degree information such that the velocity reliability degree increases with increased slowness of the velocity. Due to such operation, reliability degree information can be acquired that is suitable in accordance with the lowering of calculation accuracy of the arrangement position of the second marker102with increase in the movement velocity.

Moreover, the server200acquires the arrangement position reliability degree information such that the arrangement position reliability degree is higher with increased closeness of the arrangement position of the second marker102to the arrangement positions of the two cameras201included in the camera pair. Due to such operation, reliability degree information can be acquired that is suitable in accordance with the decrease in calculation accuracy of the arrangement position with increased distance of separation from the cameras201in general triangulation.

Moreover, the server200calculates the error between the calculated arrangement position of the first marker300within the space500and the previously known information of the arrangement position of the first marker300, and acquires the error reliability degree information such that the error reliability degree increases with increased smallness of this error. Due to such operation, the camera pair having a small error, that is, having high calculation accuracy, can be prioritized for use in the calculation of the arrangement position of the second marker102.

Another embodiment is explained below. In the present embodiment, the visible light communication system1is similar to that ofFIG. 1, and the server200is similar to thatFIG. 2. In the present embodiment, arrangement positions of the first marker300and the second marker102are multiply calculated with respect to one marker, and reliability degree information is set with respect the respective arrangement positions.

FIG. 9is a flowchart illustrating an example of generating and retaining the position-reliability degree information table by the server200according to the other embodiment. The operations illustrated inFIG. 9are performed for each of the first markers300.

For each of the camera pairs, upon capture of the same first marker300by the two cameras201included in the camera pair, this captured image is acquired via the image input unit204, and the controller202attempts to identify the first marker300by controller ID acquisition (step S301).

Thereafter, the camera position-imaging direction calculator232in step S301captures the first marker, and selects a camera pair capable of acquiring the ID (step S302).

Thereafter, for each camera pair selected in step S302, the light-emission position acquirer236calculates the arrangement position of the first marker300on the basis of the captured image captured by the two cameras201included in the camera pair (step S303). Specifically, similarly to step S203inFIG. 8, the light-emission position acquirer236acquires the position of the representation of the first marker300in the image captured by one of the cameras201among the camera pair and the position of representation of the first marker300in the image captured by the other camera201. Further, the light-emission position acquirer236calculates the arrangement position of the first marker300by using the conversion matrix corresponding to the camera pair and the combination of such two acquired positions.

Next, the reliability degree information acquirer240generates, and retains in the storage205, a position-reliability degree information table2051concerning the first marker300for which the arrangement position is calculated in step S303(step S304).

FIG. 10illustrates an example of the generated position-reliability degree information table2051retained in a certain storage region of the storage205in step S304. The position-reliability degree information table2051illustrated inFIG. 10includes, for each first marker300that is a marker: the ID of the first marker300, the arrangement positions captured by the camera pairs capturing the first marker300, the information of the camera pair capturing the image used in the calculation of the arrangement position, the reliability degree information of the arrangement position, the update time and date indicating the time and date of calculation of the arrangement position, and the error.

The reliability degree information is set in three stages as A, B, and C in order of decreasing reliability degree. For the first marker300, the reliability degree information acquirer240sets the reliability degree information by suitably selecting the image position reliability degree information acquired in a manner similar to that of step S102inFIG. 7, the velocity reliability degree information acquired in a manner similar to that of step S103in the same drawing, the arrangement position reliability degree information acquired in a manner similar to that of step S104in the same drawing, and the error reliability degree information acquired in a manner similar to that of step S105in the same drawing, or the like.

The error is set in three stages as R1, R2, and R3in order of increasing size of the error. The reliability degree information acquirer240, for example, sets the error so as to be smaller with increased approach of the update time and date to the present time and date.

FIG. 11is a flowchart illustrating another example of generating and retaining the position-reliability degree information table by the server200according to another embodiment. The operation illustrated inFIG. 11is performed for each second marker102.

For each camera pair, the two cameras201included in the camera pair capture the same second marker102, and by acquiring the ID, attempt to identify the second marker102by ID acquisition (step S401).

The camera position-imaging direction calculator232selects the camera pairs capable of capturing the second marker102and acquiring the ID in step S401(step S402).

Next, for each of the camera pairs selected in step S402, the light-emission position acquirer236calculates the arrangement position of the second marker102on the basis of the captured image captured by the pair of cameras201included in the camera pairs (step S403). Specifically, in a manner similar to that of step S203inFIG. 8, the light-emission position acquirer236acquires the representation of the second marker102in the image captured by one of the cameras201among the camera pair, and acquires the position of the representation of the second marker102in the image captured by the other camera201. Further, the light-emission position acquirer236calculates the arrangement position of the second marker102by using the conversion matrix corresponding to the camera pair and the combination of these two image positions.

Thereafter, the reliability degree information acquirer240generates position-reliability degree information concerning the second marker102for which the arrangement position is calculated in step403, and appends and retains such position-reliability degree information in the position-reliability degree information table2051(step S404).

FIG. 12illustrates an example of a position-reliability degree information table2052in step S404.FIG. 12illustrates the position-reliability degree information table2052obtained by appending the position-reliability degree information generated for each of the second markers102in step S404to the position-reliability degree information table2051generated for each of the first markers300as illustrated inFIG. 10.

The position-reliability degree information table2052generated for each of the second markers102, in a manner similar to that of the position-reliability degree information table2051generated for each of the first markers300, includes the ID of the second marker102, the arrangement position obtained by capture by the camera pair capturing this second marker102, the information of the camera pair capturing the image used in the calculation of the arrangement position, the reliability degree information of the arrangement position, the update time and date indicating the time and date at which the arrangement position is calculated, and the error.

The reliability degree information is set in three stages, that is, A, B, and C in order of decreasing reliability degree. The reliability degree information acquirer240, for the second marker102, selects and sets reliability degree information such as the image position reliability degree information acquired similarly to step S102ofFIG. 7, the velocity reliability degree information acquired similarly to step S103of the same drawing, the arrangement position reliability degree information acquired similarly to step S104of the same drawing, the error reliability degree information acquired similarly to step S105of the same drawing, or the like.

Further, for the second marker102, the reliability degree information acquirer240may be configured to acquire the arrangement position, the reliability degree information, and the error of the second marker102by acquiring the arrangement position of the first marker300.

The error is set in three stages as R1, R2, and R3in order of increasing size of the error. The reliability degree information acquirer240, for example, sets the error smaller with increasing nearness of the update time and date to the present time and date.

Next, the light-emission position acquirer236determines whether the arrangement position is calculated for all of the second markers102captured in step S401(step S405). If the arrangement position is calculated for all of the second markers102(YES in step S405), the series of operations ends. If a second marker102exists for which the arrangement position is not calculated (NO in step S405), the operations of step S402and beyond are repeated.

By generating and retaining the position-reliability degree information table concerning the first marker300and the second marker102in this manner, the reliability degree information is acquired for the calculated arrangement positions of the first marker300and the second marker102. Therefore, the arrangement position of highest reliability degree can be calculated for the first marker300and the second marker102, and the most appropriate arrangement position can be selected in consideration of both the reliability degree information and the error.

Further, the identification of arrangement positions can be performed appropriately in cases such as when identification of the arrangement position is not possible for the first marker300and the second marker102for which only the arrangement position of low reliability degree is calculated, for the first marker300and the second marker102for which the suitable arrangement position is not calculated in consideration of both the reliability degree information and the error, or the like.

Moreover, in the case in which only the position-reliability degree information table2051is generated, the arrangement position, the reliability degree information, and the error of the second marker102can be determined thereafter. That is to say, a case is described below in which, after generation of only the position-reliability degree information table2051, and then after removal of the first marker300from the space500, the interior of the same space500is captured by the cameras201ato201d. As illustrated inFIG. 13, after generation and retention of the position-reliability degree information table2051for the space500, the first marker300is removed, and the second marker102cis taken to be newly arranged (normal operation state). Then upon capturing images of the space500in this state by the cameras201ato201dand then inputting of the captured images to the image input unit204, the image processor231detects the second marker102cfrom these images. Further, the arrangement position of the second marker102cis calculated from the matrix and the representations of the second marker102cin these captured images. Then the position-reliability degree information table2051is referred to, and information is read for the first marker (first markers300dand300einFIG. 13)300captured by the camera pair having a high degree of reliability and positioned close to the calculated arrangement position of the second marker102c. The error E of the second marker102cis determined by the below-described method. The error E of the position of the second marker102cis determined by substitution into the formula E=(Ef×Xg+Eg+Xf)/(Xf+Xg), in which, as illustrated inFIG. 13, an X direction distance between the first marker300dand the second marker102cin the space is represented by Xf, an X direction distance between the first marker300eand the second marker102cis represented by Xg, an error between the calculated position and the position based on previously known information for the first marker300dis represented by Ef, and error between the calculated position and the position based on previously known information for the first marker300eis represented by Eg.

Further, the present invention is not limited to the description and drawings of the aforementioned embodiments, and suitable modifications of the aforementioned embodiments and drawings are possible.

For example, in the aforementioned embodiments, the reliability degree information acquirer240calculates the reliability degree information N concerning a single second marker102under consideration based on the formula N=A×(B1+B2+C3+C2+D), using for each camera pair, the image position reliability degree information B1and B2, the velocity reliability degree information C1and C2, the arrangement position reliability degree information D, and the error reliability degree information A that are acquired from the aforementioned processing.

However, the calculation formula is not limited to the above formula, and for example, the image position reliability degree information B1and B2, the velocity reliability degree information C1and C2, the arrangement position reliability degree information D, and the error reliability degree information A may be all multiplied together. Alternatively, the reliability degree information acquirer240may calculate the reliability degree information N for the second marker102by appropriate selection of the image position reliability degree information B1and B2, the velocity reliability degree information C1and C2, the arrangement position reliability degree information D, and the error reliability degree information A. For example, a configuration may be used that does not multiply by the error reliability degree information A when the highest reliability degree information of the second marker102is less than a threshold. Further, for example, the arrangement position reliability degree information D may be calculated for each of the two cameras201included in the camera pair. Further, a configuration may be adopted that reads and uses the information of the camera pair having the most recent update time and date.

Moreover, in the aforementioned embodiment, the server200selects the camera pair having the highest reliability degree for the second marker102, and calculates the arrangement position of the second marker102on the basis of the images captured by such a camera pair. However, the calculation procedure for the arrangement position is not limited to this configuration.

For example, the server200may calculate the arrangement position of the second marker102on the basis of captured images captured by camera pairs that are all the camera pairs capturing images of the second marker102using both of the two cameras201, and may, with increasing reliability degree of the second marker102for the camera pair, increase a weighting of the arrangement position of the second marker102calculated on the basis of capture by these camera pairs. Moreover, the server200may be configured to calculate an average value of the arrangement position of the second marker102calculated on the basis of the images of camera pairs that are highly ranked in reliability degree for the second marker102.

Moreover, in the aforementioned embodiments as illustrated inFIG. 6, although the space500is partitioned into 9 partitioning regions501ato501i, the arrangement position reliability degree information is taken to be high if the second marker102is present in the partitioning regions501a,501b, or501c, the arrangement position reliability degree information is taken to be intermediate if present in the partitioning regions501d,501e, or501f, and the arrangement position reliability degree information is taken to be low if present in the partitioning regions501g,501h, or501i, the setting of the partitioning regions501and/or the arrangement reliability degree information corresponding to each of the partitioning regions is not limited to this configuration. The arrangement reliability degree information may differ between each of the partitioning regions501.

Moreover, although the first marker300is stored in association with the IDs in the storage205and in association with the arrangement position within the space500, by visible light communication, light may be emitted that is modulated in accordance with the arrangement position within the space500.

Moreover, in the aforementioned embodiments, although the reliability degree information within the position-reliability degree information table is set in three stages as A, B, and C in order of decreasing reliability degree, this configuration is not limiting, and the reliability degree information may be set to a greater number of stages, or may be set to a numerical value. Moreover, although the reliability degree information is set by appropriate selection of the image position reliability degree information, the velocity reliability degree information, the arrangement position reliability degree information, the error reliability degree information, or the like, this configuration is not limiting.

Further, although the error within the position-reliability degree information table is set in three stages as R1, R2, and R3in order of increasing size of the error, this configuration is not limiting, and the error may be set using a greater number of stages, or may be set to a numerical value. Moreover, although the error is set so as to decrease with increased closeness of the update time and date to the present time and date, this configuration is not limiting.

For example, the first marker300and the second marker102are not limited to LEDs. For example, the marker may include part of an LCD, PDP, EL display, or the like included in a display.

Moreover, the server200may be any device to which the cameras are attachable.

Further, a program for execution of the aforementioned embodiments may be stored and distributed on a flexible disc, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), magneto-optical (MO) disc, or the like computer-readable recording medium, and by installation of the program, a system may be configured that executes the aforementioned processing.

Further, the program may be stored on a disc device or the like of a certain server on a communication network such as the Internet and, for example, may be superimposed on carrier waves and downloaded, or the like.

Moreover, in the case of realization of the aforementioned functions by execution allocated to an operating system (OS) or by execution in cooperation between the OS and an application, performance of the containing and distributing of the aforementioned recording medium, or downloading or the like, for the non-OS portion alone is permissible.