ELECTRONIC APPARATUS, INFORMATION PROCESSING SYSTEM, AND CONTROL METHOD

An electronic apparatus includes a request unit configured to transmit search request information including information of a search object, an acquisition unit configured to acquire location information indicating a location of a second apparatus at a time when the second apparatus has acquired a captured image through image capturing, and a specification unit configured to specify, in case where the search object is included in the captured image, a location of the search object based on the location information.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to an electronic apparatus, an information processing system, and a control method.

Background Art

As a technique for causing a user to perceive a sense different from reality by performing image processing on an image (captured image) acquired by imaging an object that exists in a real space, there is a cross reality (XR) technique. In addition, as a typical device for displaying an image to which the XR technique is applied, there is a head mount display (HMD), which is worn on a head of a person.

PTL 1 discloses a technique in which a captured image acquired by a camera mounted on an HMD is analyzed to display information of a product similar to a product presented in the captured image. PTL 2 discloses a technique for controlling a display in accordance with a result (recognition result) of character recognition performed on a captured image.

The location of an object (search object) for which a user is searching can be specified by analyzing a captured image acquired by a camera (electronic apparatus). However, even with the techniques of PTL 1 and PTL 2, since the captured image acquired by a camera worn by the user is used for analysis or character recognition, if the search object is not included in the captured image acquired by the camera, the location of the search object cannot be specified.

CITATION LIST

Patent Literature

SUMMARY OF THE INVENTION

Thus, it is an object of the present invention to provide a technique for easily specifying the location of a search object even when the search object is not included in a captured image acquired by a specific electronic apparatus.

According to one aspect of the present invention, an electronic apparatus serving as a first apparatus includes: a request unit configured to transmit search request information including information of a search object; an acquisition unit configured to acquire location information indicating a location of a second apparatus at a time when the second apparatus has acquired a captured image through image capturing; and a specification unit configured to specify, in case where the search object is included in the captured image, a location of the search object based on the location information.

According to one aspect of the present invention, an electronic apparatus includes: an acquisition unit configured to acquire search request information including information of a search object from a first apparatus; a determination unit configured to determine, based on the information of the search object, whether or not the search object is included in a captured image acquired by a second apparatus through image capturing; and a transmission unit configured to transmit, in case where the determination unit determines that the search object is included in the captured image, location information indicating a location of the second apparatus at a time when the second apparatus has acquired the captured image to the first apparatus.

According to one aspect of the present invention, an information processing system that includes a first apparatus and a second apparatus includes: a request unit configured to transmit search request information including information of a search object from the first apparatus; a determination unit configured to determine, based on the information of the search object, whether or not the search object is included in a captured image acquired by the second apparatus through image capturing; an acquisition unit configured to acquire location information indicating a location of the second apparatus at a time when the second apparatus has acquired the captured image through image capturing; and a specification unit configured to specify, in case where the search object is included in the captured image, a location of the search object based on the location information.

According to one aspect of the present invention, a control method of an electronic apparatus serving as a first apparatus includes the steps of: requesting by transmitting search request information including information of a search object; acquiring location information indicating a location of a second apparatus at a time when the second apparatus has acquired a captured image through image capturing; and specifying, in case where the search object is included in the captured image, a location of the search object based on the location information.

According to one aspect of the present invention, a control method of an electronic apparatus includes the steps of: acquiring search request information including information of a search object from a first apparatus; determining, based on the information of the search object, whether or not the search object is included in a captured image acquired by a second apparatus through image capturing; and transmitting, in case where the determining determines that the search object is included in the captured image, location information indicating a location of the second apparatus at a time when the second apparatus has acquired the captured image to the first apparatus.

According to one aspect of the present invention, a control method of an information processing system that includes a first apparatus and a second apparatus includes the steps of: requesting by transmitting search request information including information of a search object from the first apparatus; determining, based on the information of the search object, whether or not the search object is included in a captured image acquired by the second apparatus through image capturing; acquiring location information indicating a location of the second apparatus at a time when the second apparatus has acquired the captured image through image capturing; and specifying, in case where the search object is included in the captured image, a location of the search object based on the location information.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the present invention according to the scope of claims, and all combinations of features described in the embodiments are not necessarily essential to solving units of the present invention. In addition, the same components are denoted by the same reference characters.

An example of a hardware configuration of an image processing apparatus101according to Embodiment 1 will be described with reference toFIG.1A. The image processing apparatus101is an electronic apparatus (camera device) such as a smartphone, a tablet device, or a head-mounted display (HMD). In Embodiment 1, the image processing apparatus101will be described as an HMD. The image processing apparatus101includes a central processing unit (CPU)102, a read-only memory (ROM)103, a random access memory (RAM)104, an imaging unit105, a display unit106, an operation unit107, and a communication unit108as components. The components are connected with each other via a bus109.

The CPU102is an arithmetic processing device that comprehensively controls the image processing apparatus101. The CPU102executes various programs stored in the ROM103or the like to perform various kinds of processing.

The ROM103is a read-only non-volatile memory device that stores programs (such as image processing programs and initial data) and parameters that do not need to be changed.

The RAM104temporarily stores input information, calculation results in image processing, etc. The RAM104is a memory device that provides a working area to the CPU102.

The imaging unit105is a camera that acquires an image (captured image) by imaging a real space. In Embodiment 1, the imaging unit105images a field of view of a user wearing an HMD serving as the image processing apparatus101. This field of view is assumed to be a view that the user has when the user is not wearing the HMD.

The operation unit107is an operation unit including an operation member such as a power button or a dial.

The communication unit108performs data transmission and reception with an external device by wired communication or wireless communication (a wireless LAN, a local 5G, or the like). The communication unit108is a device compliant with communication standards such as Ethernet or IEEE 902.11.

The image processing apparatus101may be configured without the imaging unit105and the display unit106. For example, there is an HMD including the imaging unit105and the display unit106, which is provided separately from the image processing apparatus101, and the image processing apparatus101may be a personal computer that controls the HMD (the imaging unit105and the display unit106).

Hereinafter, the user of the image processing apparatus101who requests search cooperation (who makes a search request) is referred to as a “requester”. The user of the image processing apparatus101to whom the search cooperation is requested is referred to as a “cooperator”. As illustrated inFIG.1B, the image processing apparatus101(image processing apparatus101A) of the requester and the image processing apparatus101(image processing apparatus101B) of at least one cooperator are connected to each other via a network to be able to communicate with each other. The image processing apparatus101A of the requester and the image processing apparatus101B of at least one cooperator may be collectively regarded as an information processing system.

(Functional Configuration of Image Processing Apparatus of Requester)

FIG.2is a block diagram illustrating a functional configuration of an image processing apparatus101A of a “requester” according to the present embodiment. The image processing apparatus101A includes a target determination unit201, a search request unit203, an image acquisition unit204, an image analysis unit206, a result acquisition unit207, a location acquisition unit208, and a specification unit212. The processes performed by the functional configuration can be realized by the CPU102executing a program. Therefore, it can be said that the CPU102has this functional configuration. However, the CPU102does not need to perform all the processes.

For example, the image processing apparatus101may include a processing circuit dedicated to realize one or more processes. In the present embodiment, the individual components of the functional configuration may exchange information with the other components by any given method. For example, each component may temporarily store acquired or generated information in the RAM104so that the other components can acquire this information. That is, in the present embodiment, information may be exchanged between the components via the RAM104. Further, each component may output the acquired or generated information to the other component without storing the information in the RAM104.

The search request unit203transmits search request information211including the target information202to the image processing apparatus101B of the cooperator via the communication unit108. In this way, the search request unit203makes a request to search (a request to cooperate in searching) for the search object to the image processing apparatus101B. The search request unit203may request a search for the search object to a single image processing apparatus101B or may request a search for the search object to a plurality of image processing apparatuses101B.

The image acquisition unit204acquires a field-of-view image205. The field-of-view image205is a captured image that the imaging unit105has acquired by imaging a field of view of the requester, which is assumed to be a view that the requester has when the requester is not wearing the HMD serving as the image processing apparatus101A.

The image analysis unit206stores a result of an analysis performed on subjects included in the field-of-view image205in the RAM104as an analysis result210(seeFIG.3). The analysis result210is information for identifying each subject included in the field-of-view image205. The image analysis unit206determines whether or not the search object is included in the field-of-view image205based on the analysis result210and the target information202. The image analysis unit206then stores, in the RAM104, a search result213indicating whether or not the search object is included in the field-of-view image205. Therefore, the image analysis unit206is an analysis analyzes an image as well as a determination unit that determines whether a specific subject is included in the image.

The result acquisition unit207acquires a request result214from the image processing apparatus101B and stores the acquired request result214in the RAM104. The request result214includes information indicating whether or not the image processing apparatus101B has accepted the search request, information indicating whether or not the search object is included in a field-of-view image205(a search result213acquired by the image processing apparatus101B), and location information209indicating the location of the image processing apparatus101B. The request result214may include attitude information indicating the attitude of the image processing apparatus101B. In the following description, the “location” may be information on latitude and longitude, may be a combination of information on latitude and longitude and information on altitude (height), or may be a concept that is abstract to some extent, such as “inside a facility”. Therefore, the “location information” may be any information as long as an approximate location can be specified.

The location acquisition unit208acquires the location information209indicating the current location of the image processing apparatus101A (requester) by using a GPS or an altimeter (by communicating with a GPS or an altimeter). The location acquisition unit208may be capable of acquiring attitude information that is information indicating the attitude of the image processing apparatus101A by using a gyro sensor, an acceleration sensor, or the like.

The specification unit212specifies the location of the search object based on the search result213and the location information209(or based on the request result214).

(Functional Configuration of Image Processing Apparatus of Cooperator)

FIG.3is a block diagram illustrating a functional configuration of the image processing apparatus101B of the “cooperator” according to the present embodiment. The image processing apparatus101B includes, as a functional configuration, a request acquisition unit301, a request determination unit302, a user notification unit303, a request response unit304, an image acquisition unit204, an image analysis unit206, and a location acquisition unit208. The processes performed by this functional configuration can be realized by the CPU102executing a program. Therefore, it can be said that the CPU102has this functional configuration. However, the CPU102does not need to perform all the processes. For example, the image processing apparatus101may include a processing circuit dedicated to realize one or more processes. In the present embodiment, the individual components of this functional configuration may exchange information with the other components by any given method. For example, each component may temporarily store acquired or generated information in the RAM104so that the other components can acquire this information. That is, in the present embodiment, information may be exchanged between the components via the RAM104. Further, each component may output the acquired or generated information to the other component without storing the information in the RAM104.

The request acquisition unit301acquires the search request information211including the target information202from the image processing apparatus101A via the communication unit108. Thus, the request acquisition unit301receives the request to search (the request to cooperate in searching) for the search object from the image processing apparatus101A.

The request determination unit302determines whether or not to accept (reject) the search request. The request determination unit302stores information indicating whether or not the search request has been accepted in the RAM104as acceptance/rejection information305.

The user notification unit303notifies the cooperator of information of the search object (for example, displays the target information202on the display unit106).

The request response unit304is a transmission unit that transmits the request result214to the requester. The request result214includes the location information209of the image processing apparatus101B, the search result213(information indicating whether or not the search object is included in the field-of-view image205acquired by the image processing apparatus101B), and the acceptance/rejection information305. The request result214may include the analysis result210instead of the search result213.

In the following description, the letter “A” is added to the end of the reference numeral assigned to the component or information of the image processing apparatus101A, and the letter “B” is added to the end of the reference numeral assigned to the component or information of the image processing apparatus101B. For example, the image acquisition unit204of the image processing apparatus101A is referred to as the “image acquisition unit204A”. Note that, as for the target information202, the search request information211, and the request result214, since the same information is used in the image processing apparatus101A and the image processing apparatus101B, neither the letter “A” nor “B” is added to the end of the reference numeral.

FIGS.4A and4Bare diagrams for describing setting information that can be set by the requester or the cooperator operating the operation unit107.FIG.4Aillustrates disclosure setting information indicating whether or not to disclose information of the search object to the cooperators.FIG.4Billustrates display setting information indicating whether or not to display information of the search object, and cooperation setting information indicating whether or not to accept the search request. The disclosure setting information is stored in the RAM104A, and the display setting information and the cooperation setting information are stored in the RAM104B. The information of the search object may be the target information202or may be information indicating the location of the search object in a field-of-view image205B.

The requester can set whether or not to disclose the information of the search object to the cooperator by setting the disclosure setting information to either “to be disclosed” or “not to be disclosed”. The individual cooperator can set whether or not to display the information of the search object on the display unit106B by setting the display setting information to either “to be displayed” or “not to be displayed”. Further, the individual cooperator can set whether or not to accept the search request by setting the cooperation setting information to either “accept” or “reject”. That is, the requester and the individual cooperators can change the setting of information handling in view of security and privacy. Further, for example, an incentive may be given if the cooperation setting information is set such that the cooperator accepts the search request so as to encourage the cooperator to actively cooperate in the search.

A process (a specification process; a specification method; a control method of the image processing apparatus101A) in which the image processing apparatus101A specifies the location of a search object will be described with reference to the flowchart inFIG.5. The process of the present flowchart is realized by the CPU102A executing a control program according to the present flowchart.

In step S501, the target determination unit201A determines target information202indicating a search object. The target information202is, for example, information of a character string (keyword) or an image specified by the user. The target determination unit201A may determine information of an image of a specific subject captured in advance by the imaging unit105as the target information202. The target determination unit201A may determine input information provided by an external device such as a smartphone as the target information202. The target determination unit201A may determine information (information of a character string or an image) estimated on the basis of a past activity history of the requester as the target information202. For example, if the requester performs an activity such as searching for a key of a warehouse at a specific time every day, the target determination unit201A estimates (determines) an image of the key of the warehouse as the target information202based on the activity history when the specific time is reached.

In addition, instead of the information (the character string or the image) specified by the user, the target determination unit201A may determine information associated with (based on) the information specified by the user as the target information202. For example, when the user specifies an image of a key of a warehouse, if the key of the warehouse is normally kept in a specific shelf, an image of the specific shelf may be determined as the target information202. In this case, the search object is not the key of the warehouse but the specific shelf. In this case, for example, the image of the key of the warehouse and the image of the specific shelf are associated with each other and stored in the RAM104A.

In step S502, the image acquisition unit204A acquires a field-of-view image205A, which is an image of the field of view of the requester captured by the imaging unit105A. The image analysis unit206A analyzes the field-of-view image205A to determine subjects included in the field-of-view image205A. An existing object recognition technique or character recognition technique can be used for performing this analysis. The image analysis unit206A stores information of each subject determined to be included in the field-of-view image205A in the RAM104A as an analysis result210A.

In step S503, the image analysis unit206A determines whether or not the search object is included in the field-of-view image205A based on the target information202and the analysis result210A. For example, the image analysis unit206A determines whether or not the search object indicated by the target information202is included in the plurality of subjects indicated by the analysis result210A, thereby determining whether or not the search object is included in the field-of-view image205A. The image analysis unit206A then stores, in the RAM104A, a search result213A indicating whether or not the search object is included in the field-of-view image205A. If it is determined that the search object is included in the field-of-view image205A, there is no need to make a search request to the image processing apparatus101B, and the processing proceeds to step S507. If it is determined that the search object is not included in the field-of-view image205A, the processing proceeds to step S504.

Alternatively, in step S502, the image analysis unit206A may determine whether or not the search object is included in the field-of-view image205A by analyzing the field-of-view image205A based on the target information202. In this case, the processing in step S503is not performed, and if it is determined that the search object is included in the field-of-view image205A, the processing proceeds to step S507, and if it is determined that the search object is not included in the field-of-view image205, the processing proceeds to step S504.

In step S504, the search request unit203A makes a search request to the image processing apparatus101B, which is connected to the same network as the image processing apparatus101A of the requester. Specifically, the search request unit203A transmits search request information211including the target information202to the image processing apparatus101B. In Embodiment 1, the search request information211includes the disclosure setting information (seeFIG.4A).

In step S505, the result acquisition unit207A receives a request result214from the image processing apparatus101B. The result acquisition unit207A stores the request result214in the RAM104.

In step S506, the specification unit212A determines whether or not the search object is included in the field-of-view image205B of the cooperator based on the request result214(a search result213B). If it is determined that the search object is included in the field-of-view image205B, the processing proceeds to step S507. If it is determined that the search object is not included in the field-of-view image205B, the process of the present flowchart ends.

In step S507, the specification unit212A specifies the location (locational coordinates) of the search object. For example, if it is determined in step S503that the search object is included in the field-of-view image205A, the specification unit212A specifies the location of the search object based on location information209A and the coordinates of the search object in the entire field-of-view image205A. Since the location acquisition unit208A stores the location information209A in the RAM104A, the specification unit212A can acquire the location information209A from the RAM104A. On the other hand, for example, when the request result214(search result213B) indicates that the search object is included in the field-of-view image205B (YES in step S506), the specification unit212A specifies the location of the image processing apparatus101B as the location of the search object. In addition, for example, when the request result214indicates that the search object is included in the field-of-view image205B, the specification unit212A may specify the location (coordinates) of the search object based on the location (location information) and the attitude (attitude information) of the image processing apparatus101B. Specifically, by determining the optical axis direction of the imaging unit105B based on the attitude of the image processing apparatus101B, the specification unit212A can specify that the search object is located at a location in the optical axis direction from the image processing apparatus101B.

The image processing apparatuses101A and100B may include a distance sensor. In this case, for example, when the search object is included in the field-of-view image205B, the specification unit212A may specify the location of the search object based on the location (or the location and the attitude) of the image processing apparatus101B and the distances between the image processing apparatus101B and the search object. In this way, the location of the search object can be specified more accurately than specifying the location of the search object only based on the location (or the location and the attitude) of the image processing apparatus101B.

Alternatively, in step S507, the specification unit212A may specify (calculate) the relative location of the search object viewed from the requester by comparing the location information209A indicating the location of the image processing apparatus101A with location information209B indicating the location of the image processing apparatus101B. In this case, for example, as illustrated inFIG.6, the CPU102A generates an XR item601used for guidance indicating the direction in which the search object exists on the basis of the relative location of the search object. The CPU102A then displays the XR item601on the display unit106A. Thus, the image processing apparatus101A can guide the requester to the location of the search object.

InFIG.6, the XR item601is an item indicating the location of the search object with respect to the location of the image processing apparatus101A (requester). However, the XR item601may be an item representing a map including the location of the image processing apparatus101A (requester) and the location of the search object (a map from the location of the image processing apparatus101A to the location of the search object). In this case, for example, a map generation unit included in the image processing apparatus101A generates the XR item, which is a map, based on the specified location of the search object. For example, the map generation unit can generate a map representing the locations of a plurality of products in a certain store or warehouse by specifying the locations of the plurality of products in accordance with the flowchart inFIG.5.

A process (a control method of the image processing apparatus101B) in which the image processing apparatus101B transmits a search result to the image processing apparatus101A will be described with reference toFIG.7. The process of the present flowchart is realized by the CPU102B executing a control program according to the present flowchart.

The process of the flowchart inFIG.7starts when the request acquisition unit301B acquires search request information211from the image processing apparatus101A via the communication unit108B. The request acquisition unit301B stores the search request information211in the RAM104B. The search request information211includes the target information202and the disclosure setting information (seeFIG.4A).

In step S701, the request determination unit302B reads the cooperation setting information (seeFIG.4B) from the RAM104B and determines whether or not the cooperation setting information indicates that the search request is accepted. If it is determined that the cooperation setting information indicates that the search request is accepted, the processing proceeds to step S702. If it is determined that the cooperation setting information indicates that the search request is rejected, the processing proceeds to step S706.

In step S702, the image analysis unit206B analyzes a field-of-view image205B captured by the imaging unit105B to acquire an analysis result210B. The processing in step S702is the same as that in step S502, and the description thereof is omitted.

In step S703, the image analysis unit206B determines whether or not the search object is included in the field-of-view image205B based on the target information202and the analysis results210B. The image analysis unit206B then stores, in the RAM104B, a search result213B indicating whether or not the search object is included in the field-of-view image205B. The processing in step S703is the same as that in step S503. If it is determined that the search object is included in the field-of-view image205B, the processing proceeds to step S704. If it is determined that the search object is not included in the field-of-view image205B, the processing proceeds to step S706. Alternatively, in step S702, the image analysis unit206B may determine whether or not the search object is included in the field-of-view image205B by analyzing the field-of-view image205B based on the target information202.

In step S704, the user notification unit303B reads the disclosure setting information and the display setting information (seeFIG.4AandFIG.4B) from the RAM104B. When the disclosure setting information indicates “to be disclosed” and the display setting information indicates “to be displayed”, the user notification unit303B determines that the information of the search object is to be displayed. Otherwise, the user notification unit303B determines that the information of the search object is not to be displayed. If it is determined that the information of the search object is to be displayed, the processing proceeds to step S705. If it is determined that the information of the search object is not to be displayed, the processing proceeds to step S706.

In step S705, the user notification unit303B displays the information of the search object on the display unit106B. The information of the search object may be any information as long as the user (cooperator) can recognize the search object, such as information indicating the location of the search object, the target information202, or information indicating the name of the search object.

For example, as illustrated inFIG.8, the user notification unit303B superimposes an XR item802on a search object801in the field-of-view image205B. That is, the user notification unit303B notifies the user (cooperator) of information of the search object801by displaying the XR item802indicating the location of the search object801on the display unit106B. The XR item802is an item (display item) for highlighting the search object801. The XR item802may be displayed as a frame shape as illustrated inFIG.8or may be displayed as a different type of emphasized expression. Further, the user notification unit303B may display the target information202indicating information of the search object801(for example, an image or characters indicating the search object801) on the display unit106B. When the target information202is character information, the user notification unit303B may notify the user of the information of the search object801by emitting a voice reading out the characters.

Note that, if it is determined in step S704that the information of the search object is not to be displayed, the XR item802is not displayed on the display unit106B as illustrated inFIG.9even if a search object901is included in the field-of-view image205B.

In step S706, the request response unit304B transmits a request result214to the image processing apparatus101A. The request result214includes acceptance/rejection information305B, the search result213B, and the location information209B of the image processing apparatus101B. The location information209B indicates a location of the image processing apparatus101B at a time when the imaging unit105B has acquired the field-of-view image205B through image capturing. Since the location acquisition unit208B stores the location information209B in the RAM104B, the request response unit304B can acquire the location information209B from the RAM104B. The request result214may include attitude information of the image processing apparatus101B. The attitude information indicates an attitude of the image processing apparatus101B at a time when the imaging unit105B has acquired the field-of-view image205B through image capturing. When the search request is not accepted or when the search object is not included in the field-of-view image205B, the request result214does not need to include the location information209B. This is because, in such a case, the location of the search object is not specified on the basis of the location information209B.

When the cooperation setting information indicates that the search request is rejected, the request response unit304B transmits the acceptance/rejection information305B indicating that the search request has been rejected to the image processing apparatus101A. When the search object is not included in the field-of-view image205B, the request response unit304B transmits the search result213B indicating that the search object is not included in the field-of-view image205B (the search object has not been found) to the image processing apparatus101A.

According to Embodiment 1, even when the search object is not included in the field-of-view image205A acquired by the image processing apparatus101A, the location of the search object can be specified if the search object is included in the field-of-view image205B acquired by the image processing apparatus101B. Therefore, even when the search object is present at a location distant from the location of the image processing apparatus101A, the image processing apparatus101A can specify the location of the search object.

In Embodiment 1, the example in which the field-of-view image205A is displayed on the display unit106A has been described. However, the present invention is not limited this example. For example, there is a case where the display unit106A has a high degree of transparency and the requester can directly see the real space through the display unit106A. In this case, the display unit106A does not need to display the field-of-view image205A. Unless the XR item or the like is displayed on the display unit106A, the range viewed by the requester through such a display unit106A is approximately the same as the range to be captured by the imaging unit105to acquire the field-of-view image205A.

Further, the image processing apparatus101B may include a specification unit212B. In this case, when the search object is included in the field-of-view image205B, the specification unit212B may specify the location of the search object. Then, the request return unit304B may transmit the information of the location of the search object to the image processing apparatus101A.

In addition, the image processing apparatus101B may not determine whether or not the search object is included in the field-of-view image205B. In this case, the image processing apparatus101B may transmit the field-of-view image205B to the image processing apparatus101A, instead of transmitting the search result213B. In this case, the image analysis unit206A of the image processing apparatus101A determines whether or not the search object is included in the field-of-view image205B. That is, part of the above-described processing performed by the image processing apparatus101B may be performed by the image processing apparatus101A, or part of the above-described processing performed by the image processing apparatus101A may be performed by the image processing apparatus101B.

Further, in addition to the information (character string or image) specified by the user, the target determination unit201A may determine information associated with the information specified by the user as the target information202. For example, when the information specified by the user is an image of a key of a warehouse and the information associated with the information specified by the user is an image of a specific shelf, the image processing apparatus101A determines whether at least one of the key of the warehouse and the specific shelf is included in the field of view image205A or205B. If the key of the warehouse or the specific shelf is included in the field-of-view image205A or205B, the image processing apparatus101A specifies the location of the key of the warehouse or the specific shelf. In this case, the key of the warehouse and the specific shelf are the search objects.

In Embodiment 1, the image processing apparatus101A of the requester makes a search request to the image processing apparatus101B of the cooperator. According to Embodiment 1, the search for the search object can be performed (the location of the search object can be specified) by using not only the field-of-view image205A acquired by the image processing apparatus101A through image capturing but also the field-of-view image205B acquired by the image processing apparatus101B through image capturing. However, in Embodiment 1, the field-of-view image205B is the image acquired at the time when the image processing apparatus101B has received the search request, and therefore, the field-of-view image205B that has previously been acquired cannot be used for the search.

In view of the foregoing, in Embodiment 2, analysis results210B of field-of-view images205B that have previously been acquired are accumulated in a server, and an image processing apparatus101A of a requester makes a search request to the server. The server is, for example, a local 5G network server or a mobile edge computing (MEC) server. Since the server accumulates the analysis results210B, the image processing apparatus101A can search for a search object (specify the location of a search object) based on the previously-acquired field-of-view images205B.

FIG.10is a block diagram illustrating a functional configuration of a CPU102B of the image processing apparatus101B and a functional configuration of a server1002according to Embodiment 2. The CPU102B of the image processing apparatus101B according to Embodiment 2 includes, as the functional configuration, an image acquisition unit204B, an image analysis unit206B, a location acquisition unit208B, a request determination unit302B, and an information transmission unit1001B. In the following description, description of the functional configuration that has been described in Embodiment 1 will be omitted.

When an imaging unit105B has acquired a field-of-view image205B through image capturing, the information transmission unit1001B transmits location information209B corresponding to the field-of-view image205B and an analysis result210B corresponding to the field-of-view image205B to the server1002.

The server1002is an electronic apparatus including a search information DB1003. The server1002is, for example, a local 5G network server or an MEC server.

The search information DB1003is a database in which the location information209B and the analysis result210B transmitted from the image processing apparatus101B are accumulated.

FIG.11is a flowchart illustrating a process in which the image processing apparatus101B transmits the analysis result210B and the location information209B to the server1002. The process of the present flowchart is realized by the CPU102B executing a control program according to the present flowchart. The process of the present flowchart starts when the image processing apparatus101B is connected to a network (local 5G network or the like).

In step S1101, the request determination unit302B determines whether or not the cooperation setting information indicates that the search request is accepted. If it is determined that the cooperation setting information indicates that the search request is accepted, the processing proceeds to step S1102. If it is determined that the cooperation setting information indicates that the search request is rejected, the process of the present flowchart ends.

In step S1102, the image acquisition unit204B acquires the field-of-view image205B from the imaging unit105B. The image analysis unit206B analyzes the field-of-view image205B to determine subjects included in the field-of-view image205B. The image analysis unit206B stores information of the subjects determined to be included in the field-of-view image205B in the RAM104B as an analysis result210B.

In step S1103, the information transmission unit1001B transmits the location information209B and the analysis result210B to the server1002. Thus, the location information209B and the analysis result210B are stored in the search information DB1003in the server1002. The information transmission unit1001may transmit the location information209B and the analysis result210B to the server1002at timing when the transmission is requested by the server or in accordance with a time interval set by the cooperator.

The process performed by the image processing apparatus101A is the same as the process illustrated inFIG.5, and therefore, the description thereof is omitted. However, the peer to which the search request unit203A makes the search request in step S504is not the image processing apparatus101B but the server1002. Thus, when the server1002receives the search request information211, the server1002performs the same processing as that in step S703inFIG.7based on the analysis result210B and transmits a request result214to the image processing apparatus101A as in step S706. Therefore, the server1002may have a functional configuration similar to all or at least part of the functional configuration of the image processing apparatus101B illustrated inFIG.3. The request result214may always include acceptance/rejection information305indicating that the search request has been accepted or may not include the acceptance/rejection information305.

According to Embodiment 2, a search can be performed on the field-of-view image205B previously acquired by the imaging unit105B of the image processing apparatus101B. Thus, for example, even when the search object is not present in any of the field-of-view image205A and the field-of-view image205B at the current time point, the location where the search object has previously been present can be specified.

In addition, the information transmission unit1001may transmit the field-of-view image205B to the server1002, instead of transmitting the analysis result210B. In this case, the field-of-view image205B may be analyzed in the server1002.

Further, the information transmission unit1001may transmit time information indicating the time (imaging-capturing time) at which the imaging unit105B has acquired the field-of-view image205B through image capturing, together with the analysis result210B and the location information209B. The server1002accumulates the time information together with the analysis result210B and the location information209B. In this case, when the server1002receives the search request information211, the server1002transmits the request result214including the time information to the image processing apparatus101A. Next, for example, after specifying the location of the search object, the CPU102A of the image processing apparatus101A displays an item representing the time indicated by the time information (that is, the time at which the presence of the search object has been confirmed) on the display unit106A together with the XR item indicating the information of the search object. In this way, for example, even when the search object is not present at the specified location of the search object, since the requester knows the time indicated by the time information, the requester can estimate, based on the indicated time, a location to which the search object is likely to have moved between the indicated time and the current time.

The embodiments described above are all illustrative embodiments of the present invention and are not intended to limit the present invention, and the present invention can be implemented by using other various modified modes and altered modes.

Each functional unit of each embodiment (each modification) described above may be realized by separate hardware, but not limited thereto. The functions of two or more functional units may be implemented by common hardware. Each of the plurality of functions of one functional unit may be implemented by separate hardware. Two or more functions of one functional unit may be implemented by common hardware. Each functional unit may be realized by hardware such as an ASIC, an FPGA, or a DSP, but not limited thereto. For example, the apparatus may include a processor and a memory (storage medium) in which a control program is stored. The function of at least a part of the functional units included in the apparatus may be realized by the processor reading and executing the control program in the memory.

According to the present invention, even when a search object is not included in a captured image acquired by a specific electronic apparatus, the location of the search object can easily be specified.

Other Embodiments

The present invention is not limited by the embodiments described above, and various changes and modifications can be made without departing from the spirit and scope of the present invention. Accordingly, the following claims are appended to disclose the scope of the present invention.