Information collection system and information center

Each of vehicles acquires its own position, and transmits an image of a nearby outside taken by an own camera and a piece of information of the own position. An information center communicatable wirelessly with the vehicles receives the images and the pieces of information of the vehicles' positions transmitted by the vehicles, and recognizes pieces of first character information appearing in the received images, respectively. Based on the received pieces of information of the vehicles' positions as well as at least either points of interest or areas stored as collection target information, the information center determines at least either points of interest or areas for the recognized pieces of first character information, respectively, and stores association information where the pieces of first character information are associated with the determined at least either points of interest or areas.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims the benefit of priority of Japanese Priority Application No. 2016-088307, filed on Apr. 26, 2016, the entire contents of which are hereby incorporated herein by reference.

BACKGROUND

1. Technical Field

The present disclosure relates to an information collection system and an information center.

2. Description of the Related Art

An information collection system for storing images taken by cameras of vehicles has been known (for example, see Japanese Laid-Open Patent Application No. 2013-239087).

According to Japanese Laid-Open Patent Application No. 2013-239087, images taken by cameras of vehicles are stored, and useful images selected from the stored images are provided to vehicles.

SUMMARY

According to one aspect of the present disclosure, an information collection system includes a plurality of vehicles. Each of the vehicles includes one or more processors configured to acquire a position of the vehicle, and transmit an image of a nearby outside taken by a camera included in the vehicle and a piece of information of the position of the vehicle acquired when the image is taken. The information collection system further includes an information center wirelessly communicatable with the vehicles, respectively. The information center includes one or more processors configured to receive the images and the pieces of information of the positions of the vehicles from the vehicles, respectively, and recognize pieces of first character information appearing in the received images, respectively. The one or more processors of the information center are further configured to, based on the received pieces of information of the positions of the vehicles as well as at least either points of interest or areas stored as collection target information, determine at least either points of interest or areas for the recognized pieces of first character information, respectively, and store pieces of association information where the pieces of first character information are associated with the determined at least either points of interest or areas, respectively.

Other objects, features and advantages will become more apparent from the following detailed description when read in conjunction with the accompanying drawings.

DETAILED DESCRIPTION OF EMBODIMENTS

According to Japanese Laid-Open Patent Application No. 2013-239087 mentioned above, it is possible to prevent useless images from being provided to users. However, stored images are provided to users as they are, and therefore, the users may be not able to use information shown in the images effectively. For example, an image provided to a user may show a POI (Point Of Interest) of a shop that a user is interested in, a signboard or a banner concerning an event for a certain area, and so forth. However, it may be difficult for the user to immediately understand in which type the shop is, in which area the event will be held, and so forth. Especially, when an image is provided to a vehicle's driver or an occupant during traveling, it may be desired that the information indicated by a signboard, banner, or the like, shown in the image, can be immediately understood by the driver or the occupant. Otherwise, the vehicle may actually be far away from the corresponding shop, area, or the like, when the driver or the occupants have understood the information indicated by the signboard, banner, or the like. Thus, according to the technology disclosed by Japanese Laid-Open Patent Application No. 2013-239087, it may be actually difficult for users to effectively use information appearing in provided images.

In consideration of the point, an object of the present disclosure is to provide an information collection system with which it is possible to store pieces of information appearing in photographed images taken by cameras of vehicles in such a manner that users can use the information more effectively.

Below, embodiments will be described using the accompanying drawings.

FIG. 1generally illustrates one example of a configuration of an information collection system1according to an embodiment. The information collection system1includes a plurality of vehicles2and an information center3. The information center3and each of the vehicles2are bidirectionally connected via, for example, a communications network4such as a cellular phone network. Below, description will be made assuming that the vehicles2have the same or similar configurations.

Each vehicle2includes a camera21, a sensor22, a microphone23, an ECU (Electronic Control Unit)24, and a wireless communications device25.

The camera21photographs an outside near the vehicle2in a certain range in response to an operation instruction from the ECU24(actually, a photographing control unit242, described later). An image (photographed image data) taken by the camera21is transmitted to the ECU24via an on-vehicle network such as a 1-to-1 communications line or a CAN (Controller Area Network), and is temporarily stored in a storage unit245described later.

The sensor22detects various states of the vehicle2(vehicle states). The sensor22includes a GPS sensor that detects the position of the vehicle2, an angular acceleration sensor (i.e., a gyro sensor) that detects angular accelerations along the three axes of the vehicle2(i.e., the forward and backward axis, the left and right axis, and the vertical axis), and so forth. Data (vehicle state data) detected by the sensor22is transmitted to the ECU24via an on-vehicle network such as a 1-to-1 communications line or a CAN (Controller Area Network).

The microphone23is installed in the vehicle cabin, and acquires sounds of occupants (including the driver) of the vehicle2in response to an operation instruction from the ECU24(actually, a sound recognition unit244described later). The sounds acquired by the microphone23are transmitted to the ECU24via an on-vehicle network such as a 1-to-1 communications line or a CAN (Controller Area Network).

The ECU24carries out control operations concerning acquisition of various sorts of data to be transmitted to the information center3, and transmission of the acquired information to the information center3. The ECU24, for example, includes a microcomputer, and implements various control processes by executing various programs stored in a ROM (Read-Only Memory). The ECU24includes, as functional units implemented as a result of a CPU (Central Processing Unit) executing the various programs stored in the ROM, a transmission and reception process unit241, a photographing control unit242, a QR code reading unit243, and a sound recognition unit244. Also, the ECU24includes, as a storage area prepared in an internal nonvolatile memory, a storage unit245.

The transmission and reception process unit241transmits signals to the information center3and receives signals transmitted from the information center3, through the wireless communications device25.

The transmission and reception process unit241transmits the position of the vehicle2(for example, information such as latitude and longitude information) to the information center3successively based on data that is input from the GPS sensor included in the sensor22.

Also, in response to receiving a signal to request to photograph a nearby outside of the vehicle2using the camera3from the information center3, the transmission and reception process unit241sends the request to the photographing control unit242.

Also, in response to a transmission request from the photographing control unit242, the transmission and reception process unit241transmits image data photographed by the camera21and corresponding additional information (i.e., image additional information) to the information center3. The image additional information includes the time when the photographed image is taken, the position of the vehicle2, the angular acceleration information of the vehicle2, and so forth, acquired when the photographed image is taken, camera information (i.e., the position in the vehicle where the camera21is installed, the installation angle, the angle of field, and so forth, of the camera21).

Also, in response to a transmission request from the QR code reading unit243, the transmission and reception process unit241transmits QR code read data and additional information corresponding to the QR code read data (i.e., QR code additional information) to the information center3. The QR code additional information includes the time when the photographed image corresponding to the QR code read data is taken, the position of the vehicle2, the angular acceleration information of the vehicle2, and so forth, acquired when the photographed image is taken.

Also, in response to a transmission request from the sound recognition unit244, the transmission and reception process unit241transmits sound recognized data and additional information corresponding to the sound recognized data (i.e., sound additional information) to the information center3. The sound additional information includes the time when the sound data corresponding to the sound recognized data is acquired, the position of the vehicle2acquired when the sound data is acquired, and so forth.

The photographing control unit242controls the camera21. Actually, when the transmission and reception process unit241has received a photographing request from the information center3, the photographing control unit242responds to the photographing request to transmit an operation instruction to the camera21. In more detail, the photographing control unit242transmits an operation instruction corresponding to the contents of the instruction included in the photographing request (for example, timing to start and end photographing, the position to photograph, and so forth). Below, with reference toFIG. 2, a process flow of the photographing control unit242to control photographing will be described.

FIG. 2is a flowchart generally illustrating one example of a control process carried out by the photographing control unit242. The process of the flowchart is repetitiously carried out every certain time interval, during, for example, a period of time starting from the completion of an initial process carried out after ignition ON (IG-ON) of the vehicle2and ending at ignition OFF (IG-OFF).

In step S102, the photographing control unit242determines whether the transmission and reception process unit241has received a photographing request from the information center3. When the transmission and reception process unit241has received a photographing request from the information center3, the photographing control unit242proceeds to the step S104. When the transmission and reception process unit241has not received a photographing request from the information center3, the photographing control unit242ends the current process.

In step S104, the photographing control unit242transmits an operation instruction corresponding to the instruction contents of the photographing request to the camera21. Thereby, the camera21photographs a situation near the vehicle2during the period of time for the traveling zone according to the operation instruction.

In step S106, the photographing control unit242transmits a transmission request, to transmit the photographed image data of the camera21to the information center3, to the transmission and reception process unit241, and ends the current process. Thus, the transmission and reception process unit241transmits the photographed image data and the image additional information to the information center3.

Returning toFIG. 1, the QR code reading unit243recognizes a QR code appearing in the photographed image of the camera21. Also, the QR code reading unit243analyzes the recognized QR code, to acquire QR code read data (second character information). Below, with reference toFIGS. 3A, 3B, and 4, an actual example of a QR code reading process of the QR code reading unit243, and a process flow in the QR code reading unit243will be described.

First,FIG. 3Aillustrates one example of a QR code appearing in a photographed image, andFIG. 3Billustrates one example of corresponding QR code read data (second character information).

In the example ofFIG. 3A, the front of a B department store51, facing an intersection, appears in a photographed image of the camera21. Then, approximately at the center of the photographed image, an advertisement that includes a relatively large QR code52of the B department store51appears. The QR code reading unit243recognizes the QR code52appearing in the photographed image illustrated inFIG. 3A, analyzes the QR code52, and acquires the QR code read data illustrated inFIG. 3B.

In the example ofFIG. 3B, the QR code read data is text data concerning an event of the B department store51. Actually, the QR code read data has the contents “<B DEPARTMENT STORE EVENT> YOU CAN FIND DELICIOUS FOODS AND GOOD THINGS OF KYUSHU. SUMMER LARGE EXHIBITION OF KYUSHU FROM AUGUST XX (WED) THRU YY (MON)”.

Next,FIG. 4is a flowchart generally illustrating one example of a process to acquire QR code read data (second character information) from a QR code appearing in a photographed image (a QR code reading process). The process of the flowchart is carried out, for a photographed image, for example, each time immediately after the photographed image of the camera21is stored in the storage unit245.

In step S202, the QR code reading unit243tries to recognize a QR code of a photographed image acquired from the storage unit245.

In step S204, the QR code reading unit243determines whether any QR code appears in the photographed image. If a QR code appears in the photographed image, the QR code reading unit243proceeds to step S206. If no QR code appears in the photographed image, the QR code reading unit243ends the current process.

In step S206, the QR code reading unit243analyzes the recognized QR code to acquire QR code read data.

In step S208, the QR code reading unit243transmits a transmission request, to transmit the QR code read data to the information center3, to the transmission and reception process unit241, and ends the current process. Thus, the transmission and reception process unit241transmits the QR code read data and the corresponding QR code additional information to the information center3.

Returning toFIG. 1, the sound recognition unit244responds to a predetermined operation of a user to activate the microphone23, carry out a sound recognition process on sound data acquired by the microphone23, and generate sound recognized data. Then, after acquiring the sound recognized data, the sound recognition unit244transmits a transmission request, to transmit the sound recognized data to the information center3, to the transmission and reception process unit241. For example, when the user of the vehicle2has found useful information near the vehicle2, the user of the vehicle2performs the predetermined operation and murmurs the contents of the useful information. As a result, the sound recognition unit244uploads the sound recognized data (text data) corresponding to the murmur to the information center3, and the information can be thus shared with other users.

The storage unit245temporarily stores a photographed image of the camera21in response to an instruction from the photographing control unit242.

The wireless communications device25carries out wireless communications with the information center3via the communications network4, and includes, for example, a DCM (Data Communication Module).

The information center3is installed remotely from the vehicles2, and is configured to be able to bidirectionally carry out wireless communications with the vehicles2, respectively. The information center3includes a wireless communications device31and an information collection apparatus32.

The wireless communications device31can be any device for carrying out wireless communications with each of the vehicles2via the communications network4.

The information collection apparatus32collects information concerning predetermined POIs (hereinafter, referred to as “target POIs”) and predetermined areas (hereinafter, referred to as “target areas”). Generally speaking, the information collection apparatus32generates information (i.e., “delivery information”, or “association information” described later) concerning the target POIs and the target areas from photographed image data, QR code read data, and so forth, transmitted from vehicles2(actually, the corresponding transmission and reception process units241), and stores the generated delivery information.

Also, the information collection apparatus32acquires photographed image data, QR code data, and so forth, for monitoring, from the relatively small number of vehicles2from among vehicles2traveling near each target POI and each target area, for example, generates the delivery information for monitoring, and stores the delivery information for monitoring for each target POI and each target area.

Then, if it is determined that the stored delivery information for monitoring for a target POI or a target area has a change more than a predetermined amount, an updating determination unit323(described later) determines to update the delivery information for the target POI or the target area. In response, the information collection apparatus32acquires photographed image data, QR code data, and so forth, from the relatively large number of vehicles2traveling near the target POI or the target area, for example, generates the latest delivery information for the target POI or the target area, and stores the generated latest delivery information in a manner of adding the generated information or updating the exiting information with the generated information. Thus, the delivery information for being actually delivered by an information delivery unit331(described later) for the target POI or the target area is updated.

Thus, the information collection apparatus32generates and stores the delivery information for monitoring, and also, generates and stores the delivery information for being actually delivered to vehicles2or the like by the information delivery unit331through a transmission and reception process unit321(described later).

The information collection apparatus32includes, for example, one or more computers. The information collection apparatus32includes the transmission and reception process unit321, a photographing instruction unit322, the updating determination unit323, an image acquisition unit324, an additional information acquisition unit325, a character information recognition unit326, a POI-and-area determination unit327, an association information generation unit328, a character information correction unit329, and a storage process unit330.

Also, the information collection apparatus32includes the information delivery unit331, an image storage unit332, an additional information storage unit333, a POI-and-area information DB334, a map information DB335, a web crawling DB336, a sound recognition DB337, an unfixed information DB338, and a delivery information DB339.

Note that, the information concerning the target POIs and the target areas is previously stored in the POI-and-area information DB334. When the target POIs and the target areas have been updated (some have been added, some have been deleted, and so forth), the information concerning the target POIs and the target areas is updated.

The transmission and reception process unit321transmits signals to the vehicles2and receive signals from the vehicles through the wireless communications device31.

When the transmission and reception process unit321has received photographed image data and image additional information, the transmission and reception process unit321sends information indicating this fact to the image acquisition unit324, and stores the received photographed image data and image additional information in the image storage unit332and the additional information storage unit333, respectively.

Also, when the transmission and reception process unit321has received QR code read data and QR code additional information from a vehicle2, the transmission and reception process unit321sends information indicating this fact to the storage process unit330.

Also, when the transmission and reception process unit321has received sound recognized data and sound additional information from a vehicle2, the transmission and reception process unit321stores the sound recognized data in the sound recognition DB337in a predetermined format, and stores the sound additional information in the additional information storage unit333.

Also, when the transmission and reception process unit321has received a transmission request from the photographing instruction unit322, the transmission and reception process unit321transmits the photographing requests to vehicles2designated by the transmission request.

When the transmission and reception process unit321has received a delivery request from the information delivery unit331, the transmission and reception process unit321transmits the delivery information to vehicles2designated by the delivery request, registered smartphones of the users of the designated vehicles, or the like.

The photographing instruction unit322transmits a transmission request to the transmission and reception process unit321according to predetermined conditions, to transmit photographing requests to vehicles2designated by the transmission request through the transmission and reception process unit321.

For example, for each target area and each target POI, if the updating determination unit323does not determine to update the delivery information, the photographing instruction unit322transmits the photographing requests to the relatively small number of vehicles2selected according to a predetermined rule from among vehicles2traveling near the target area or the target POI, periodically, for example. As a result, the photographing instruction unit322finally acquires the corresponding photographed image data, QR code read data, and so forth for monitoring, with which the above-mentioned delivery information for monitoring is generated and stored.

On the other hand, if the updating determination unit323determines, based on, for example, the corresponding delivery information for monitoring stored, to update the delivery information for a target POI or a target area, the photographing instruction unit322transmits the photographing requests to the relatively large number of vehicles2selected according to a predetermined rule from among vehicles2traveling near the target area or the target POI. As a result, the photographing instruction unit322finally acquires the corresponding photographed image data, QR code read data, and so forth for keeping the delivery information in the latest state.

That is, the latest delivery information is generated using the thus acquired photographed image data, QR code read data, and so forth for keeping the delivery information in the latest state, and is stored in such a manner that the generated information is added or the exiting information is updated with the generated information.

Thus, the delivery information for being actually delivered by the information delivery unit331is updated for each target POI and each target area.

The photographing requests include instruction contents such as timings to start and end photographing, the places to photograph, and so forth.

The updating determination unit323determines whether to update the delivery information stored in the delivery information DB339. For example, the updating determination unit323periodically compares, in a time-series manner, the above-mentioned delivery information for monitoring stored in the delivery information DB339, to acquire the difference in the delivery information for monitoring between different time points, for each target POI and each target area. Then, the updating determination unit323determines, based on the acquired difference, whether to update the delivery information for the target POI or the target area.

The image acquisition unit324acquires the photographed image data from the image storage unit332, and sends the photographed image data to the character information recognition unit326.

The additional information acquisition unit325acquires the image additional information corresponding to the photographed image data acquired by the image acquisition unit324from the additional information storage unit333, and sends the acquired image additional information to the character information recognition unit326.

The character information recognition unit326recognizes character information (first character information) appearing in the photographed image acquired by the image acquisition unit324through a character recognition process.

The POI-and-area determination unit327determines a target POI or a target area concerning the character information recognized by the recognition unit326, based on the position of the vehicle2acquired when the photographed image is taken included in the image additional information, and so forth.

The association information generation unit328generates association information where the character information (first character information) recognized by the character information recognition unit326is associated with the target POI and the target area determined by the POI-and-area determination unit327.

Below, with reference toFIGS. 5A, 5B, and 6, an actual example of a POI-and-area determination process of the POI-and-area determination unit327, and a flow of a process starting from recognition of character information and ending at generation of association information, carried out by the character information recognition unit326, the POI-and-area determination unit327, and the association information generation unit328, will be described.

FIGS. 5A and 5Billustrate an actual example of a process of the POI-and-area determination unit327to determine a target POI or a target area concerning character information (first character information).FIG. 5Aillustrates an actual example of a photographed image acquired by the image acquisition unit324.FIG. 5Bis a plan view illustrating a situation where the photographed image ofFIG. 5Ais photographed. In more detail,FIG. 5Billustrates, in a superposing manner, lines and polygons corresponding to the respective planimetric features such as the buildings and the road that are GIS (Geographic Information System) data stored in the map information DB335, as well as the vehicle2and the angle-of-field information (the angle range of photographing in the left and right directions).

As illustrated inFIG. 5A, in the photographed image, a nearby situation centering on the forward direction of the vehicle2appears including the road53on which the vehicle2is traveling, the A restaurant54present along the road53, and so forth. The banner55installed by the A restaurant54appears at a right end part of the paragraphed image, and character information “STEAK FAIR IS NOW BEING HELD” is printed on the banner55. Therefore, the character information recognition unit326recognizes the character information appearing in the photographed image. Then, the POI-and-area determination unit327determines a target POI or a target area concerning the character information thus recognized by the character information recognition unit326.

The POI-and-area determination unit327can recognize the position of the vehicle2acquired when the photographed image is taken, and the corresponding camera information (i.e., the angle of field of the camera21), from the image additional information acquired by the additional information acquisition unit325. Thus, as illustrated inFIG. 5B, the POI-and-area determination unit327can recognize the nearby situation of the vehicle2occurring when the photographed image is photographed, and identifies the appearing planimetric features such as the buildings, from the GIS data of the map information DB335, the image additional information, and so forth.

Also, the POI-and-area determination unit327can determine whether the planimetric feature such as the shop or the building appearing in the photographed image corresponds to a target POI or is present in a target area, by comparing the name, or the like, of the shop or the building appearing in the photographed image, with the target POIs, the target areas, and so forth, stored in the POI-and-area information DB334.

Thus, in the example ofFIGS. 5A and 5B, the POI-and-area determination unit327can determine that the character information printed on the banner55relates to the A restaurant54adjacent to the banner55.

Also, in the example ofFIGS. 5A and 5B, the corresponding target POI can be determined to be a restaurant according to the contents of the character information. Therefore, it is also possible to use the contents of the character information to determine the corresponding target POI.

Next,FIG. 6generally illustrates one example of a flow of a process starting from recognition of character information and ending at generation of association information carried out by the character information recognition unit326, the POI-and-area determination unit327, and the association information generation unit328. The process of the flowchart is carried out, for example, each time when the transmission and reception process unit241receives photographed image data and image additional information data from a vehicle2. Also, the process of the flowchart is carried out, for each photographed image.

In step S302, the character information recognition unit326recognizes character information (first character information) appearing in the photographed image.

In step S304, the POI-and-area determination unit327tries to determine a target POI and a target area for the character information recognized by the character information recognition unit326based on the position of the vehicle2where the photographed image is taken. It is also possible that, as described above, the POI-and-area determination unit327tries to determine a target POI and a target area for the character information, based on the corresponding image additional information (the position of the vehicle2, and the angle of field of the camera21), the information concerning the target POIs and the target areas stored in the POI-and-area information DB334, and the map information (GIS data) stored in the map information DB335.

In a simpler way, it is also possible that the POI-and-area determination unit327tries to determine a target POI and a target area for the character information based on the position of the vehicle2where the photographed image corresponding to the character information is photographed, and the information concerning the target POIs and the target areas stored in the POI-and-area information DB334.

It is also possible that, as described above, the POI-and-area determination unit327tries to determine a target POI and a target area, using the contents of the character information.

In step S306, the association information generation unit328determines whether the POI-and-area information DB334includes a target POI that has high relevancy with the character information recognized by the character information recognition unit326. If the POI-and-area information DB334includes a target POI that has high relevancy with the character information recognized by the character information recognition unit326, the association information generation unit328proceeds to step S308. If the POI-and-area information DB334includes no target POI that has high relevancy with the character information recognized by the character information recognition unit326, the association information generation unit328proceeds to step S314.

In step S308, the association information generation unit328determines whether the POI-and-area information DB334includes a target area having high relevancy with the character information recognized by the character information recognition unit326. If the POI-and-area information DB334includes a target area having high relevancy with the character information recognized by the character information recognition unit326, the association information generation unit328proceeds to step S310. If the POI-and-area information DB334includes no target area having high relevancy with the character information recognized by the character information recognition unit326, the association information generation unit328proceeds to step S312.

In step S310, the association information generation unit328generates information (i.e., association information) where the character information recognized by the character information recognition unit326is associated with the target POI and the target area determined by the POI-and-area determination unit327, stores the generated association information in the unfixed information DB338, and ends the current process.

In step S312, the association information generation unit328generates information (i.e., association information) where the character information recognized by the character information recognition unit326is associated with the target POI determined by the POI-and-area determination unit327, stores the generated association information in the unfixed information DB338, and ends the current process.

If it is determined in step S306that the POI-and-area information DB334includes no target POI for the character information, the association information generation unit328determines whether the POI-and-area information DB334includes a target area having high relevancy with the character information recognized by the character information recognition unit326, in step S314.

If the POI-and-area information DB334includes a target area having high relevancy with the character information recognized by the character information recognition unit326, the association information generation unit328proceeds to step S316. If the POI-and-area information DB334includes no target area having high relevancy with the character information recognized by the character information recognition unit326, the association information generation unit328proceeds to step S318.

In step S316, the association information generation unit328generates information (i.e., association information) where the character information recognized by the character information recognition unit326is associated with the target area determined by the POI-and-area determination unit327, stores the generated association information in the unfixed information DB338, and ends the current process.

In step S318, the association information generation unit328stores the character information recognized by the character information recognition unit326in the unfixed information DB338, and ends the current process.

Returning toFIG. 1, the character information correction unit329carries out error correction on the character information included in the association information stored in the unfixed information DB338. This is because the character information recognized based on the photographed images through the character recognition process may include errors.

The character information correction unit329carries out, for example, probabilistic error correction in a general-purpose method. For example, it is possible to detect and correct errors that may occur in the character information included in the association information, using an N-Gram model, an editing distance (Levenshtein distance), or the like, which is known elemental technology of statistical natural language processing. It is also possible, for example, to use a database (i.e., an OCR (Optical Character Recognition) similar character DB) concerning similar characters for which errors may likely to occur in OCR.

Also, the character information correction unit329detects and corrects errors that may occur in the character information included in the association information, using web crawling acquired data concerning at least either the target POIs or the target areas included in the association information.

Actually, the character information correction unit329acquires web crawling acquired data concerning at least either the target POIs or the target areas included in the association information, from the web crawling DB336. Then, the character information correction unit329can detect and correct errors that may occur in the character information included in the association information, by extracting texts similar to the character information from the acquired web crawling acquired data concerning at least either the target POIs or the target areas included in the association information, and carrying out comparison (matching) between the character information and the extracted similar texts.

Note that, the web crawling acquired data stored in the web crawling DB336is web information acquired through web crawling operations on each POI and each area, and is updated periodically to correspond to the latest web information.

Also, the character information correction unit329detects and corrects errors that may occur in the character information included in the association information, using sound recognized data concerning at least either the target POIs or the target areas included in the association information.

Actually, the character information correction unit329searches the sound recognition DB337for the sound recognized data acquired at the same time when the photographed images corresponding to the character information included in the association information are taken near the positions where the paragraphed image are taken. Then, the character information correction unit329can detect and correct errors that may occur in the character information included in the association information, by extracting texts similar to the character information from the thus acquired sound recognized data, and carrying out comparison (matching) between the character information and the extracted similar texts.

The storage process unit330stores the association information for which the character information correction unit329carried out error correction on the character information, in the delivery information DB339. Also, after receiving the information indicating the fact that the QR code read data and the QR code additional information have been received from the transmission and reception process unit321, the storage process unit330stores the QR code read data in the delivery information DB339in a predetermined format, and stores the QR code additional information in the additional information storage unit333.

Below, with reference toFIGS. 7 and 8, an actual example of the character information correction process of the character information correction unit329, and a flow of a process carried out by the storage process unit330starting from the character information error correction and ending at storing association information in the delivery information DB339will be described.

FIG. 7illustrates an actual example of the error correction process of the character information correction unit329.

As illustrated inFIG. 7, in this example, character information “STEAK FAIR IS NOW BEING HELD” printed on the banner55installed by the A restaurant54illustrated inFIG. 5Ais extracted from photographed images photographed by three different vehicles2. The corresponding three pieces of association information where the A restaurant54that is a target POI is associated with the corresponding three pieces of character information, respectively, are stored in the unfixed information DB338.

Actually, the respective pieces of character information in the respective pieces of association information are: “STEAT FAIR IS NOW BEING HELD”; “STEAK FAI IS NOW BEING HELD”; and “STEAK FAIR IS NOW BEING HELD”. As can be seen, the first and second pieces of character information include errors, respectively.

As described above, the character information correction unit329can carry out: (1) probabilistic error correction according to a general-purpose method; (2) error correction using web information acquired for the target POIs or the target areas; and (3) error correction using sound recognized data.

In particular, for example, concerning the error correction (2), the character information correction unit329can extract a text “steak fair is now being held” similar to the corresponding character information, based on web crawling acquired data concerning (i.e., web information acquired for) the A restaurant54that is the corresponding target POI.

In another example, concerning the error correction (3), if sound recognized data having the contents “A RESTAURANT, IT SAYS THAT A STEAK FAIR IS NOW BEING HELD! I WANT TO HAVE MEAT” is included in the sound recognition DB337, as illustrated inFIG. 7, the character information correction unit329can extract the text “steak fair is now being held” from the sound recognized data.

Therefore, the character information correction unit329carries out error correction to correct the first and second pieces of character information included in the corresponding pieces of association information stored in the unfixed information DB338into “STEAK FAIR IS NOW BEING HELD”. Then, the storage process unit330stores the association information for which the errors have been thus corrected in the delivery information DB339as the corresponding delivery information.

Next,FIG. 8is a flowchart generally illustrating a flow of a process of the character information correction unit329and the storage process unit330starting from the error correction of character information and ending at storing association information in the delivery information DB339. The process of the flowchart is carried out each time when, for example, the association information generation unit328stores association information in the unfixed information DB338.

In step S402, the character information correction unit329acquires association information from the unfixed information DB338.

In step S404, the character information correction unit329tries to detect an error from character information included in the acquired association information, in a general-purpose method (probabilistic error correction). If an error is detected, the character information correction unit329corrects the error.

In step S406, the character information correction unit329acquires the web crawling acquired data concerning the target POIs and the target areas included in the association information from the web crawling DB336. Then, the character information correction unit329tries to detect an error from the character information included in the acquired association information, based on the web crawling acquired data. If an error is detected, the character information correction unit329corrects the error.

In step S408, the character information correction unit329determines whether sound recognized data acquired near the place where the photographed image corresponding to the character information included in the acquired association information is taken at the same time is included in the sound recognition DB337. If the sound recognized data that satisfies the conditions is included in the sound recognition DB337, the character information correction unit329proceeds to step S410. If no sound recognized data that satisfies the conditions is included in the sound recognition DB337, the character information correction unit329proceeds to step S412.

In the step S410, the character information correction unit329tries to detect an error from the character information included in the acquired association information based on the acquired sound recognized data. If an error is detected, the character information correction unit329corrects the error.

In step S412, the character information correction unit329determines whether the accuracy of the corrected character information included in the acquired association information is high. A criterion to determine whether the accuracy is high can be determined from various viewpoints. For example, it is possible to increase the possibility to determine that the accuracy is high, if step S410has been passed through.

If the accuracy of the corrected character information included in the acquired association information is high, the character information correction unit329proceeds to step S414. If the accuracy of the corrected character information included in the acquired association information is not high, the character information correction unit329ends the current process.

In step S414, the storage process unit330stores the acquired association information on which the character information correction unit329has carried out the error correction, in the delivery information DB339, and ends the current process.

Returning toFIG. 1, the information delivery unit331delivers, to vehicles2near a target POI or a target area or the smartphones of the registered users of the vehicles2, the delivery information concerning the target POI or the target area stored in the delivery information DB339through the transmission and reception process unit321. Thus, the information delivery unit331delivers the association information where the character information is associated with the target POI or the target area, as well as the QR code read data concerning the target POI or the target area, to the corresponding vehicles2or the like. Below, with reference toFIG. 9, an actual example of giving the delivery information to a user, which is delivered by the information delivery unit331, will be described.

FIG. 9illustrates one example of giving the delivery information to a user, which is delivered by the information delivery unit331.FIG. 9illustrates an image shown on the screen of a navigation apparatus of a vehicle2when the QR code recognized data (second character information) acquired based on the QR code recognized from the photographed image illustrated inFIG. 3Aand the association information acquired based on the character information (first character information) recognized from the photographed image illustrated inFIG. 5Ahave been delivered to the vehicle2.

As illustrated inFIG. 9, the message that “STEAK FAIR IS NOW BEING HELD” is displayed for the A restaurant54that is the target POI, and the fact that “SUMMER LARGE EXHIBITION OF KYUSHU” is held is indicated for the B department store51. That is, instead of providing the character information and the QR code appearing in the photographed image as they are, the first character information and the second character information can be provided as the information associated with the POIs or the areas. Thus, the user can immediately understand the relevancy of the provided information with the POIs or the areas, and thus, the user can use the information more effectively.

Note that, normally, QR code read data includes information concerning the corresponding target POI or target area.

Thus, according to the embodiments, the transmission and reception process unit321of the information center3receives, from a plurality of vehicles2, images photographed by their cameras21and image additional information such as the positions acquired when the photographed images are taken, and so forth.

The character information recognition unit326of the information center3recognizes the character information (the first character information) appearing in the photographed images. Also, the POI-and-area determination unit327determines target POIs or target areas for the character information recognized by the character information recognition unit326based on the positions acquired when the photographed images are taken, and the information concerning the target POIs and the target areas stored in the POI-and-area information DB334.

The association information generation unit328of the information center3generates the association information where the character information recognized by the character information recognition unit326is associated with at least either the target POIs or the target areas determined by the POI-and-area determination unit327, and the storage process unit330stores the generated association information in the delivery information DB339.

Thus, it is possible to store the association information where the character information appearing in the photographed images is associated with the target POIs or the target areas determined for the character information, and it is possible to provide the association information to users.

Therefore, the users can immediately understand that the character information is associated with the particular POIs or areas because the provided association information is displayed on the displays of the vehicles or the like. As a result, the users can immediately go toward the POIs or the areas nearby. Thus, the users can effectively use the provided information.

Also, according to the embodiments, the character information correction unit329of the information center3carries out error correction on the character information included in the association information. Therefore, even if errors occur in character recognition carried out by the character information recognition unit326, it is possible to store the more accurate association information in the delivery information DB339because error correction is thus carried out. That is, it is possible to provide information that is more accurate to users.

Also, according to the embodiments, the character information correction unit329of the information center3carries out error correction on the character information included in the association information based on the web crawling acquired data concerning the target POIs and the target areas included in the association information. Also, the character information correction unit329of the information center3carries out error correction on the character information included in the association information based on the sound recognized data concerning the target POIs and the target areas. Thus, it is possible to carry out error correction on the character information included in the association information, using the web information acquired for the target POIs and the target areas included in the association information or the sound recognized data (for example, text data corresponding to murmurs for providing information of the users of vehicles2). Therefore, it is possible to precisely correct errors that may occur in character recognition carried out by the character information recognition unit326, which errors may be difficult to deal with through general-purpose error correction technology (for example, for a case where error correction is carried out on character information that includes an intentional typographical error, character information that includes a coined word, or the like).

Also, according to the embodiments, the QR code reading unit243of each vehicle2recognizes a QR code appearing in a photographed image, and analyzes the QR code to acquire QR code read data (second character information). Then, the storage process unit330of the information center3stores the QR code read data in the delivery information DB339. Thus, the QR code read data acquired through analysis of the contents of the QR code appearing in the photographed image is stored in the delivery information DB339, and it is possible to provide the QR code read data to users. Also, in many cases, QR code read data includes, as illustrated inFIG. 3B, information concerning a POI or an area, and therefore, it is possible to provide the QR code read data as information concerning the POI or the area. Also, for example, if a photographed image is provided as it is to a user, the user himself or herself reads a QR code appearing in the photographed image using an application of a smartphone. In contrast thereto, according to the embodiments, the user himself or herself need not read a QR code using an application of a smartphone, for example, and therefore, it is possible to improve the convenience of the user.

Thus, according to the embodiments, a position of each vehicle is acquired, and an image of a nearby outside taken by a camera of the vehicle and a piece of information of the position of the vehicle acquired when the image is taken are transmitted from the vehicle to an information center that is communicatable wirelessly with the vehicle. The information center receives the images and the pieces of information that indicate positions of the vehicles, respectively, and recognizes pieces of first character information appearing in the photographed images, respectively. Based on the received pieces of information of the positions of the vehicles as well as at least either points of interest or areas stored as collection target information, the information center determines at least either points of interest or areas for the recognized pieces of first character information, respectively, and stores, in a storage unit, pieces of association information where the pieces of first character information are associated with the determined at least either points of interest or areas, respectively.

Thus, the pieces of association information where the pieces of first character information appearing in the photographed images are associated with the points of interest or the areas determined for the pieces of first character information are stored, and are provided to users. Therefore, when the provided pieces of association information are displayed on displays of the respective vehicles, or the like, the users can immediately understand that the pieces of first character information relate to the particular points of interest or area. Therefore, the users can actually go to the points of interest or areas nearby immediately, and thus, it is possible to effectively use the provided information. Thus, it is possible to store the information appearing in the photographed images taken by the cameras of the vehicles in such a manner that it is possible to more effectively use the information.

The information center corrects errors in the pieces of first character information included in the pieces of association information.

Therefore, even if errors occur when the pieces of first character information are recognized from the images, the errors are corrected, and thus, the more accurate pieces of association information can be stored. Thus, it is possible to provide the more accurate information to the users.

When the information center corrects errors occurring when the pieces of first character information are recognized, the web information acquired for the at least either points of interest or areas is used. Therefore, it is possible to correct errors in the pieces of association information using the web information acquired for the points of interest or areas included in the pieces of association information. Therefore, even if errors occurring when the pieces of first character information are recognized are those which are difficult to correct through general-purpose error correction technology, i.e., in cases where the character information includes intentional typographical errors, the character information includes coined words, or the like, it is possible to precisely correct the errors.

Each vehicle or the information center recognizes QR codes appearing in the images, and analyzes the recognized QR codes to acquire pieces of second character information, respectively. Then, the vehicle or the information center stores the pieces of second character information.

Thus, the pieces of second character information acquired from analyzing the contents of the QR codes appearing in the images are stored and provided to the users. In many cases, the pieces of second character information include information concerning the corresponding points of interest or areas. Therefore, it is possible to provide the information as information concerning the particular points of interest of areas to the users. For example, if a photographed image is provided to a user as it is, the user himself or herself reads the QR code appearing in the photographed image using a corresponding application, or the like. In contrast thereto, according to the embodiments, the user is free from such work of operating a corresponding application to read a QR code by himself or herself. Thus, it is possible to improve the convenience of the users.

Thus, the information collection systems and the information centers have been described in the embodiments. However, the present disclosure is not limited to these embodiments. Various modifications and improvements can be made within the scope of the present disclosure.

For example, in the above-described embodiments, the POI-and-area determination unit327tries to determine both target POIs and target areas for the character information. However, it is also possible that the POI-and-area determination unit327tries to determine either target POIs or target areas for the character information. In this case, it is sufficient that the POI-and-area information DB334stores information concerning either targets POI or target areas. Also, the association information generation unit328associates the character information recognized by the character information recognition unit326with either the target POIs or the target areas determined by the POI-and-area determination unit327.

Also, in the above-described embodiments, the QR code reading unit243is included in each of the vehicles2. However, it is also possible that the QR code reading unit243is included in the information center3, as shown inFIG. 1indicated by a block243of a broken line.

Also, in the above-described embodiments, the character information correction unit329is used. However, the character information correction unit329can be omitted. In this case, the storage process unit330simply stores the association information generated by the association information generation unit328in the delivery information DB339as delivery information as it is.

According to the embodiments, it is possible to provide information collection systems with which it is possible to store information appearing in photographed images taken by cameras of vehicles in such a manner that users can use the information more effectively.