Source: http://www.google.com/patents/US20020081020?dq=6,563,928
Timestamp: 2017-07-28 17:32:31
Document Index: 602437400

Matched Legal Cases: ['art 101', 'art 105', 'art 101', 'art 102', 'art 104', 'art 105', 'art 101', 'art 101', 'art 101', 'art 102', 'art 101', 'art 101', 'art 101', 'art 101', 'art 102', 'art 101', 'art 105', 'art 102', 'art 104', 'art 105', 'art 102', 'art 102', 'art 102', 'art 101', 'art 101', 'art 102', 'art 102', 'art 102', 'art 104', 'art 104', 'art 102', 'art 102', 'art 105', 'art 101', 'art 105', 'art 105', 'art 105', 'art 105', 'art 101', 'art 102', 'art 104', 'art 105', 'art 106', 'art 107', 'art 109', 'art 101', 'art 109', 'art 109', 'art 109', 'art 104', 'art 105', 'art 109', 'art 109', 'art 109', 'art 109', 'art 109', 'art 202', 'art 109', 'art 202', 'art 109', 'art 202', 'art 109', 'art 201', 'art 202', 'art 203', 'art 203', 'art 202', 'art 203', 'art 202', 'art 109', 'art 109', 'art 202', 'art 203', 'art 202', 'art 109', 'art 109', 'art 107', 'art 105', 'art 203', 'art 101', 'art 102', 'art 104', 'art 105', 'art 121', 'art 122', 'art 123', 'art 124', 'art 125', 'art 127', 'art 128', 'art 121', 'art 122', 'art 122', 'art 123', 'art 123', 'art 127', 'art 124', 'art 121', 'art 124', 'art 122', 'art 125', 'art 124', 'art 123', 'art 125', 'art 128', 'art 127', 'art 102', 'art 101', 'art 102', 'art 101', 'art 102', 'art 105', 'art 107']

Patent US20020081020 - Infromation providing server, client, information providing system ... - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsThe present invention has a three-dimensional image data conversion part 101 for converting aerial photographs A and B of the same area photographed from two different locations to three-dimensional image data, an advertisement placing information storage part 105 for storing advertisement placing information...http://www.google.com/patents/US20020081020?utm_source=gb-gplus-sharePatent US20020081020 - Infromation providing server, client, information providing system processing method, recording medium recording a program, and advertisement providing methodAdvanced Patent SearchTry the new Google Patents, with machine-classified Google Scholar results, and Japanese and South Korean patents.Publication numberUS20020081020 A1Publication typeApplicationApplication numberUS 10/023,886Publication dateJun 27, 2002Filing dateDec 21, 2001Priority dateDec 25, 2000Publication number023886, 10023886, US 2002/0081020 A1, US 2002/081020 A1, US 20020081020 A1, US 20020081020A1, US 2002081020 A1, US 2002081020A1, US-A1-20020081020, US-A1-2002081020, US2002/0081020A1, US2002/081020A1, US20020081020 A1, US20020081020A1, US2002081020 A1, US2002081020A1InventorsHideo ShimazuOriginal AssigneeNec CorporationExport CitationBiBTeX, EndNote, RefManPatent Citations (5), Referenced by (49), Classifications (9), Legal Events (1) External Links: USPTO, USPTO Assignment, EspacenetInfromation providing server, client, information providing system processing method, recording medium recording a program, and advertisement providing method
[0039] [0039]FIG. 2 shows the configuration of an information providing server according to a first embodiment of the present invention. An information providing server 1 according to the first embodiment of the present invention has a three-dimensional image data conversion part 101, a three-dimensional image synthesizing part 102, a three-dimensional image browser 103, a clock part 104, and an advertisement placing information storage part 105. [0040] The three-dimensional image conversion 0part 101 converts two-dimensional image information to three-dimensional image data. At least two aerial photographs (aerial photographs A and B) of the same area photographed from different locations are inputted to the three-dimensional image data conversion part 101. The physical position of the photographed area (the physical position of a reference point included in the photographs) is specified in the two aerial photographs. The three-dimensional image data conversion part 101 uses the inputted two aerial photographs to perform a stereo process, a DEM process and a polygon process, and generates three-dimensional image data which is then outputted to the three-dimensional image synthesizing part 102. [0041] In the following explanation, photographs of the same area (the earth's surface) photographed periodically like a satellite image of a space satellite or an aerial photograph of an airplane are referred to as an aerial photograph. The aerial photographs A and B must not be always photographed in the right and left directions. The aerial photographs A and B of the same area photographed at different angles may be photographed in the up and down directions. [0042] [0042]FIG. 3 is a diagram of assistance in explaining the process of the three-dimensional image data conversion part 101. The three-dimensional image data conversion part 101 uses a parallax between at least two aerial photographs of the same location photographed (the aerial photographs A and B) to calculate a value in the height direction of a building or a mountain (the stereo process). The three-dimensional image data conversion part 101 then allows the calculated height direction value to be elevation meshed (DEM imaging: Digital Elevation Model) to reproduce an intermediate representation 31 (DEM image).Further, the three-dimensional image data conversion part 101 performs a polygon process to the reproduced intermediate representation 31 (DEM image) to automatically generate three-dimensional image data 32. The information providing server 1 gradually changes the three-dimensional image data 32 generated based on the viewpoint position from the user so as to provide the user with a three-dimensional aerial sightseeing service as if the user is sightseeing in the air. [0043] The above-mentioned photograph is generally sold by an aerial photograph provider. For example, an aerial photograph for one kilometer around photographed by the artificial satellite IKONOS operated by Space Imaging in the United States is sold for several thousand yen (in the case of a monochrome photograph) by the subsidiary thereof in Japan. Since the IKONOS takes a photograph of the entire earth for a fixed period, the system manager can purchase a latest aerial photograph of the surface of the entire earth every week. Such an aerial photograph is provided with exact latitude and longitude information. The purchaser can exactly grasp the physical position information to show what area is photographed. [0044] In the stereo process using an aerial photograph, information on which is a building or which is a mountain is not extracted. In the stereo process using an aerial photograph, height information for each unit location is only calculated. For example, in the stereo process using an aerial photograph taken by an airplane which can fly low, a height of the earth's surface for 25 cm around can be calculated. The artificial satellite IKONOS takes a photograph high up in the sky. In the stereo process using an aerial photograph of the artificial satellite IKONOS, a height of the earth's surface for 1 m around can be calculated. As shown in FIG. 3, the result of the stereo process (height information for each unit location) becomes the intermediate representation 31 (DEM image) like a needle point holder used for flower arrangement. [0045] The stereo process is an already-established technique. The present invention may use the existing suitable stereo process. In the present invention, the stereo process is not limited to a specific stereo process. [0046] [0046]FIG. 4 is a diagram showing a specific example of the intermediate representation 31 (DEM image). The DEM image extracts height information for each element position. As shown in FIG. 4, the height information for each element position is shown by a rectangular parallelepiped shape. The case of calculating height information for 1 m around will be described. When a building having a height of 30 m is located in the range for 20 m around, height information of a height of 30 m is extracted on 400 (20×20) points in the range in which the building is located. When the periphery of the building is a level ground, height information of a height of 0 m is extracted on a point other than the 400 points. When a rule that “in the case that points of the same height are continuous, this means one building” is applied here, it can be assumed that there is a building in the range of the 400 points. In the case that elements each have the same height information in this way, a continuous plane is assumed, thereby supposing a stereoscopic construction. The polygon process is performed to the DEM image. As a result, polygon data can be obtained. [0047] When the stereo process and the polygon process are performed, the so-called landmark such as a building or mountain is extracted as polygon data. In the present invention, the data >includes one or more landmarks of the extracted polygon data is referred to as three-dimensional image data. An aerial photograph is provided with exact positional information of the reference point in the photograph. The positional information of the landmark included in the three-dimensional image data (the positional information on the earth's surface such as a latitude or longitude) can be easily calculated by a distance from the reference point. [0048] [0048]FIG. 5 shows a specific example of the three-dimensional image stereoscopic representation. The information providing server 1 uses the three-dimensional image data 32 generated by the above-mentioned stereo process, DEM process and polygon process, and creates a three-dimensional image stereoscopic representation 33 shown in FIG. 5 which is then delivered to the client terminal 2. To generate the three-dimensional image stereoscopic representation 33, the information providing server 1 needs a viewpoint position to indicate from what viewpoint it is viewed. The information providing server 1 uses the viewpoint position to generate the three-dimensional image stereoscopic representation 33 viewed from different positions. [0049] The information providing server 1 uses the viewpoint position which is gradually, continuously moved so as to generate the three-dimensional image stereoscopic representation which is then provided to the user. The information providing server 1 can thus provide the user with a service as if the user is performing a sightseeing flight in the air. [0050] The three-dimensional image synthesizing part 102 uses the three-dimensional image data converted by the three-dimensional image data conversion part 101 and advertisement placing information stored in the advertisement placing information storage part 105 to generate synthesis three-dimensional image data. The three-dimensional image synthesizing part 102 refers to the clock part 104 to acquire the current date data, fetches the advertisement placing information (including information of an advertisement ID, a location, a placing period, and the advertisement contents) stored in the advertisement placing information storage part 105, and judges whether the “placing period” item included in each of the advertisement placing information includes the acquired current date data or not. The three-dimensional image synthesizing part 102 judges whether the “location” item included in each of the advertisement placing information is a location included in the three-dimensional image data or not. The three-dimensional image synthesizing part 102 extracts the advertisement placing information in which the “placing period” includes the current date data and the “location” is a location included in the three-dimensional image data, and fetches data registered in “the advertisement contents” item included in the extracted advertisement placing information. The three-dimensional image synthesizing part 102 uses the fetched advertisement contents data and the three-dimensional image data outputted from the three-dimensional image data conversion part 101 to generate synthesis three-dimensional image data and outputs the generated three-dimensional image data to the three-dimensional image browser 103. The positional information of the three-dimensional image data can be calculated from the position of the aerial photograph. The three-dimensional image data outputted from the three-dimensional image data conversion part 101 is represented by a set of polygon data such as a cylinder or parallelepiped. In order to represent an advertising balloon shaped advertisement item as a new advertisement medium, the three-dimensional image synthesizing part 102 may only add advertising balloon-shaped polygon data to the three-dimensional image data. [0051] The three-dimensional imagebrowserlo3 uses the synthesis three-dimensional image data outputted from the three-dimensional image synthesizing part 102 to generate a three-dimensional image stereoscopic representation. When the three-dimensional image browser 103 receives the synthesis three-dimensional image data outputted from the three-dimensional image synthesizing part 102, it stores once the synthesis three-dimensional image data into the storage part. The three-dimensional image browser 103 waits for a command from the client terminal 102. When a specific viewpoint position is inputted from the client terminal, the three-dimensional image browser 103 creates a three-dimensional image stereoscopic representation viewed from the inputted viewpoint position which is then provided to the user. The three-dimensional image browser 103 waits again for a command input of the viewpoint position from the client terminal 2. In the drawing, the information providing server 101 has the three-dimensional image browser 103, but the client terminal 2 may have the three-dimensional image browser 103. [0052] The clock part 104 manages and holds the current date data. The clock part 104 outputs the current date data to the three-dimensional image synthesizing part 102 based on an acquisition request of the current date data from the three-dimensional image synthesizing part 102. [0053] The advertisement placing information storage part 105 stores advertisement placing information including the advertisement contents an advertisement provider desires to place in order to place a specific advertisement in the three-dimensional image data converted by the three-dimensional image data conversion part 101. The advertisement placing information stored in the advertisement placing information storage part 105 includes an advertisement ID, a placed location to place an advertisement, a placing period, and the advertisement contents to be placed. Here, the advertisement ID is an identifier for specifying the advertisement placing information. As one form of registering the advertisement placing information, the advertisement ID, placed location, placing period, and advertisement contents are represented in an internal representation bit string and can be registered in a record unit including those. The advertisement placing information storage part 105 registers all the advertisement placing information to be placed in the three-dimensional image data. [0054] The advertisement contents included in the advertisement placing information includes shape information showing the shape of an advertisement medium and text information such as a message specified to be placed in the advertisement medium. Here, the shape information showing the shape of an advertisement medium includes various pattern shapes. For example, a shape like a drum, a shape of an advertising balloon, an airship and a swallow as shown in FIG. 6, a shape of animals, and a shape of an airplane or a thunderhead are included. A shape of a sandwich man holding a placard describing text information such as a message may be included. The text information includes not only a text but also an image, figure, or sign. [0055] For example, a certain advertisement provider desires to place certain advertisement contents (to place an advertisement describing a message “Invitation to the Internet: NEC” in an advertising balloon-shaped advertisement medium) (IDID).The advertisement provider desires a placing period of one month from September, 2000 and a placed location in which an advertising balloon floats over the NEC head office. In this case, in the advertisement placing information storage part 105, ID1 is inputted to the [advertisement ID] item, the same latitude and longitude values as the latitude and longitude of the NEC head office are inputted to the [location] item, a value of “Y2000M9D1T0: 00-Y2000M9D30T23: 59” is inputted to the [placing period] item, and information of an advertising balloon-shaped three-dimensional polygon figure and the massage “Invitation to the Invent: NEC” of the text information displayed on the side of the advertising balloon are inputted to the [advertisement contents] item. [0056] [0056]FIG. 6 shows a display example of the three-dimensional image stereoscopic representation actually provided from the information providing server 1 of the present invention to the client terminal. As shown in FIG. 6, advertisements displaying text information of “A automobile” for a balloon-shaped advertisement medium, “C beer of B food” for an airship-shaped advertisement medium, and “E cosmetic of D medicine” for a swallow-shaped advertisement medium are displayed in the three-dimensional image stereoscopic representation in a specific city. The user looks at these advertisements while the user is sightseeing in the three-dimensional image stereoscopic representation generated by the information providing server 1. [0057] The information providing server system 1 can paste link information to a home page related to the advertisement (including a home page managed by the advertisement provider who requests to place the advertisement) onto the advertisement displayed in the three-dimensional image stereoscopic representation provided to the client terminal 2. When the user finds an interesting advertisement of the advertisements displayed during the three-dimensional aerial sightseeing, the user moves the mouse onto the advertisement and clicks on it so as to access the home page linked. The user can naturally use online shopping provided in the accessed home page. In this case, the client terminal can place home page information jumped while the display screen of the three-dimensional image stereoscopic representation is displayed. [0058] The client terminal 2 of the present invention displays the three-dimensional image stereoscopic representation. The user can thus browse or search information as if the user is sightseeing in a three-dimensional space. When the user is sightseeing in the three-dimensional image stereoscopic representation of a specific location in order to browse information, the advertisement of an advertisement medium of a three-dimensional shape (such as advertising balloon) is displayed in the three-dimensional image stereoscopic representation. The advertisement medium of a three-dimension shape (such as an advertising balloon) is polygon data like the three-dimension shaped object (polygon data) automatically generated from an aerial photograph of another building or a park in the three-dimensional image stereoscopic representation. [0059] A difference between the advertisement medium of a three-dimension shape and the three-dimensional shaped object such as another building or a park will be explained here. The three-dimension shaped object is created from an aerial photograph of the earth's surface really photographed and actually exists in photographing. On the contrary, the three-dimension shaped advertisement medium is not photographed when an aerial photograph is taken, that is, is of a non-existing virtual shape. [0060] The information providing server 1 according to the first embodiment of the present invention allows the user to browse or search for information as if the user is sightseeing in a three-dimensional space. The information providing server 1 can provide the user with a service in which the user can enjoy a search operation. The advertisement contents included in the advertisement placing information stored in the advertisement placing information storage part 105 are placed in the three-dimensional image stereoscopic representation provided by the information providing server 1, so that the user can look at the advertisement irrespective of his/her own intension. It is possible to give the advertisement provider a large merit in which his/her own advertisement can be displayed to the user. [0061] <The Configuration of an Information Providing Server of a Second Embodiment of the Present Invention>
[0080] [0080]FIG. 10 shows the configuration of an information providing server 1 according to a third embodiment of the present invention. The information providing server 1 according to the third embodiment of the present invention has a three-dimensional image data conversion part 101, a three-dimension image synthesizing part 102, a clock part 104, an advertisement placing information storage part 105, an advertisement contract part 106, an advertisement contract storage part 107, a three-dimensional image database 108, and a position input type three-dimensional image synthesizing part 109. The same components as those of the first embodiment of the present invention are indicated by similar numerals. The description of the same components as those of the first embodiment of the present invention is omitted. [0081] The three-dimensional image database 108 stores the three-dimensional image data generated by the three-dimensional image data conversion part 101. The three-dimensional image database 108 may be either a relational database or a typical file format. The three-dimensional image data includes an image ID for uniquely identifying three-dimensional image data, “position” information of three-dimensional image data (e.g., the latitude and longitude of a reference position), and image data. The image data includes one or more polygon data showing a landmark. The image ID is a number provided for uniquely identifying generated three-dimensional image data. As one form for registering the three-dimensional image data into the three-dimensional image database 108, the image ID, position and image data are represented in an internal representation bit string and can be registered in a record unit including those. [0082] The “position” information included in three-dimensional image data will be explained. When the size of the range of the earth's surface included in three-dimensional image data generated (vertical and horizontal lengths) is fixed, it is sufficientwhenthepositionofthereferencepointoftheearth's surface included in three-dimensional image data generated based on two aerial photographs A and B (the latitude and longitude) is stored as the “position” information into the three-dimensional image data 108. Since the aerial photographs are provided with exact information of the reference point in the photographs, a position of the reference point of the earth's surface included in the three-dimensional image data can be decided using the information of the reference point in the photographs. When the size of the range of the earth's surface included in the three-dimensional image data (vertical and horizontal lengths) is different in the individual three-dimensional image data, the “position” information including the data related to the size of the range of the earth's surface (vertical and horizontal lengths) must be stored into the three-dimensional image database 108. As an example of storing the “position” information in this case, when the size of the range of the earth's surface is rectangular, it can be realized by the structure for storing two points diagonally positioned in the rectangle. A method for storing the “position” information is not limited to this, and any method may be used which can find the size of the range of the earth's surface included in the three-dimensional image data (vertical and horizontal lengths) and the position on the earth's surface. [0083] The position input type three-dimensional image synthesizing part 109 synthesizes the three-dimensional image data based on the positional information specified from a position output type three-dimensional image browser 201 with the advertisement information. When the position output type three-dimensional image browser 201 specifies positional information, the position input type three-dimensional image synthesizing part 109 searches the three-dimensional image database 108, refers to the specified positional information and the “position” information included in the three-dimensional image data, and searches for the three-dimensional image data including the viewpoint position to fetch the corresponding three-dimensional image data. The position input type three-dimensional image synthesizing part 109 refers to the clock part 104 to acquire the current date data and refers to the acquired current date data, and the “placing period” included in the advertisement placing information stored in the advertisement placing information storage part 105 to search for the advertisement placing information in which the “placing period” includes the current date data. The position input type three-dimensional image synthesizing part 109 judges whether the “placed location” information included in the advertisement placing information is included in the size of the range of the earth's surface included in the three-dimensional image data (vertical and horizontal lengths) or not. In this manner, the position input type three-dimensional image synthesizing part 109 can search for the advertisement placing information in which the “placing period” includes the current date data and the “placed location” is included in the size of the range of the earth's surface included in the three-dimensional image data (vertical and horizontal lengths). The position input type three-dimensional image synthesizing part 109 fetches the “advertisement contents” information included in the searched advertisement placing information. The position input type three-dimensional image synthesizing part 109 synthesizes the fetched “advertisement contents” information with the three-dimensional image data fetched from the three-dimensional image database 108 to generate synthesis three-dimensional image data and delivers the generated synthesis three-dimensional image data to the client terminal 2. [0084] The position output type three-dimensional image browser 201 included in the client terminal 2 generates a three-dimensional image stereoscopic representation based on the synthesis three-dimensional image data delivered from the position input type three-dimensional image synthesizing part 109 and the viewpoint position outputted from an input part 202. [0085] The position output type three-dimensional image browser 201 performs an image request to the position input type three-dimensional image synthesizing part 109 based on the positional information outputted from the input part 202 and receives the synthesis three-dimensional image data from the position input type three-dimensional image synthesizing part 109. The position output type three-dimensional image browser 201 converts the received synthesis three-dimensional image data to a three-dimensional image stereoscopic representation. [0086] When the user inputs from the input part 202 information of a specific position where the user desires to sightsee in the air, the position output type three-dimensional image browser 201 sends the positional information to the position input type three-dimensional image synthesizing part 109, and as a result, receives the synthesis three-dimensional image data. The position output type three-dimensional image browser 201 stores the received synthesis three-dimensional image data into the position output type three-dimensional image browser part 201. When the user inputs a specific viewpoint position from the input part 202, the position output type three-dimensional image browser 201 generates the received three-dimensional image data into a three-dimensional image stereoscopic representation viewed from the viewpoint position and outputs the generated three-dimensional image stereoscopic representation to a display part 203. When the user inputs a new viewpoint position, the position output type three-dimensional image browser 201 regenerates the three-dimensional image stereoscopic representation viewed from the new viewpoint position. In this embodiment, the position output type three-dimensional image browser 201 is provided on the client terminal 2, but may be provided on the information providing server 1. [0087] [0087]FIG. 11 shows a first display example of a screen displayed by the display part 203 of the client terminal 2. When the user requests the “menu” screen to use the three-dimensional aerial sightseeing service via the input part 202, the screen as shown in FIG. 9 is displayed on the display part 203. [0088] The display contents of a menu 91 shown in FIG. 11 is an example for displaying the position of the entire surface of the earth by a tree structure of a hierarchy structure. It is displayed in this manner so that the user can easily input positional information. The menu 91 shown in FIG. 11 positions the entire earth in the uppermost position of a hierarchy, and the continents of Asia, Africa, America and Europe are positioned in the layer below it. In the lower layer of each of the continents, the subdivided country name of “North America” or “Japan” is positioned. In the lower layer of the country name, a city name is positioned. The display format is the same as the Explorer format (R) used in the Windows (R). [0089] As shown in FIG. 12, the menu screen can be created by an image format. As shown in FIGS. 12(a) to 12(c), images of photographs of a globe, a Japanese map and a specific area are correspondingly pasted. The image pasted here may be either of a two-dimensional bit map format or of a snapshot in a three-dimensional space. [0090] When the user clicks on a flag set in a given position in the earth's map shown in FIG. 12(a), the map of the countries or continents managed in the lower layer is displayed on the display part. A flag is set in a given position of each of the countries or continents, and when the user clicks on the flag, the map of each region or area is displayed on the display part. For example, when the user clicks on the flag set in Japan, the display part displays the Japanese map as shown in FIG. 12(b) in which a flag is set in each region or area. When the user clicks on the flag set on Tokyo of the flags set in the regions, the Tokyo map as shown in FIG. 12(c) in which a flag is set in each city is displayed on the display part. When the user clicks on a given flag of the flags set in the cities, a three-dimensional image stereoscopic representation of each city (Shinjuku, Harajuku, and so on) managed in the lower layer of the clicked flag is displayed on the display part, as shown in FIG. 12(d). The flag set on the map is not necessarily limited to a flag shape, and may be of other shapes. [0091] As described above, until the user selects the menu item of the lowest level in which the user can actually use the sightseeing service in a three-dimensional space, the display part will not display a three-dimensional image stereoscopic representation. For example, when the user selects Shinjuku as a desired area, the display part of the client terminal displays the three-dimensional image stereoscopic representation of “Shinjuku”. The user can use the three-dimensional aerial sightseeing service as if the user is sightseeing over “Shinjuku”. In this case, the manager of the information providing server can paste a snapshot in a still state of the three-dimensional space image onto the lowest level menu item. The manager of the information providing server 1 can freely set the position of the lowest level that the user can actually use the aerial sightseeing service. [0092] When the user is using the three-dimensional aerial sightseeing service and desires to enjoy an aerial sightseeing in another area or region, the following two methods can be considered. [0093] A first method is that the three-dimensional image representations are continuously switched and continue to be displayed along with movement of the user and the user sequentially travels over a region or area. This enables the user to continuously enjoy sightseeing from over Tokyo to another region or area. For example, the user sequentially travels via the three-dimensional image displays of the regions or areas included between Tokyo and Osaka. [0094] A second method is that the display part displays the menu screen again and the user selects the position (the area or region) where the user desires to sightsee from the menu screen. This enables the user to move instantly. For example, when the user desires to switch the image of over Tokyo to the image of over Osaka, the menu screen shown in FIG. 11 is displayed on the display part and the user moves to the hierarchy of “Japan” on the menu screen to select “Osaka” there. In this manner, the user displays the three-dimensional image representation of Osaka on the display part. Such a state is not limited to the same country. In other words, on the menu screen shown in FIG. 11, the user can instantly move to over all the countries of the world. The user can enjoy the aerial sightseeing as if the user moves instantly from over Tokyo to over Amazon or Champs Élysées in Paris. For example, when the user desires to move from Tokyo to over Champs Élysées in Paris, on the menu screen shown in FIG. 11, the user sequentially moves from the “Japan” hierarchy to the “Asia” hierarchy, the uppermost hierarchy of “the earth”, the “Europe” hierarchy, and the “France” hierarchy to select “Paris” there. [0095] As a method in which the user specifies a specific position (location), the user can specify it based on information of the latitude and longitude. For example, when the user desires to sightsee in the three-dimensional image stereoscopic representation near Champs Élysées in Paris, the user looks up information of the latitude and longitude including the Champs Élysées area using the existing map. The user uses the input part 202 to input the information of the latitude and longitude to the position output type three-dimensional image browser 201. [0096] As a method in which the user inputs a specific position (location), there is also the following method. For example, the information providing server 1 allows the display part of the client terminal 2 to display map information, and the user specifies a given location (a location where the user desires to sightsee in the air) with the mouse. The client terminal 2 instantly calculates information of the latitude and longitude of the specified point to send the information of the calculated latitude and longitude to the position input type three-dimensional image synthesizing part 109 of the information providing server 1. The position input type three-dimensional image synthesizing part 109 generates synthesis three-dimensional image data based on the information of the latitude and longitude which is then outputted to the client terminal 2. When the client terminal 2 receives the synthesis three-dimensional image data by the position output type three-dimensional image browser 201, it generates a three-dimensional image stereoscopic representation based on the viewpoint position from the input part 202 which is then displayed on the display part 203. [0097] When the user inputs a desired location from the input part 202 of the client terminal 2 using the above methods, the position input type three-dimensional image synthesizing part 109 searches the three-dimensional image database 108 for the three-dimensional image data including a given location where the user desires to go. The position input type three-dimensional image synthesizing part 109 synthesizes the searched three-dimensional image data with the advertisement contents to generate synthesis three-dimensional image data. The position output type three-dimensional image browser 201 receives the generated synthesis three-dimensional image data and uses the received three-dimensional image data and the viewpoint position specified from the user to generate and display a three-dimensional image stereoscopic representation. [0098] [0098]FIG. 13 shows a second processing example in the advertisement providing server 1 according to an embodiment of the present invention. The sponsor terminal 3 sends to the information providing server 1 an advertisement placing request to place an advertisement (step S201). When the information providing server 1 receives the advertisement placing request, it inquires of advertisement information including the advertisement placing information and the contract information (step S202). [0099] To the inquiry from the information providing server 1 in step S202, the sponsor terminal 3 sends desired advertisement information (step S203). The information providing server 1 sends a confirmation of the advertisement contract including advertisement contents, a placed location, a placing period, and contract money amount and a request of the approval based on the sent advertisement information and a request of the approval (step S204). [0100] In step S204, the sponsor terminal 3 which has approved the advertisement contract contents sent from the information providing server 1 requests an advertisement placing process (step S205) and pays the presented contracted money to the information providing server 1 by a predetermined payment method (step S206). [0101] The information providing server 1 which has been requested the advertisement placing process registers or updates the contract information into the advertisement contract storage part 107 in accordance with the advertisement information, and further registers or updates the advertisement placing information into the advertisement placing information storage part 105 (step S207). [0102] The client terminal 2 requests an access to the information providing server 1 based on a command from the user (step S208). When the information providing server 1 approves the access, it sends the menu screen for the three-dimensional image aerial sightseeing service to the client terminal 2 (step S209). [0103] When the user selects a location where the user desires to sightsee in the air from the menu screen displayed on the display part 203 of the client terminal 2, the client terminal sends positional information based on the selected location to the information providing server 1 (step S210). The information providing server 1 creates synthesis three-dimensional image data based on the sent positional information (step S211) which is then sent to the client terminal (step S212). [0104] When the user inputs a desired viewpoint position, the client terminal 2 uses the inputted viewpoint position and the sent synthesis three-dimensional image data to generate and display a three-dimensional image stereoscopic representation (step S213). [0105] The information providing server 1 according to the third embodiment of the present invention can automatically store the three-dimensional; image data created from the photographed aerial photographs one after another into the three-dimensional image database 108. When aerial photographs of the earth's surface all over the world are taken and three-dimensional image data generated therefrom is stored into the three-dimensional image database 108, the user selects a given location to which he/she desires to go (Champs Élysées in Paris or the great plain in Siberia) from all over the world and can sightsee in the three-dimensional image stereoscopic representation of the selected location. [0106] <The Configuration of an Information Providing Server According to a Fourth Embodiment of the Present Invention>
[0130] [0130]FIG. 18 shows the schematic configuration of an information providing server according to a sixth embodiment of the present invention. An information providing server 1 according to the sixth embodiment of the present invention has a three-dimensional image data conversion part 101, a three-dimension image synthesizing part 102, a three-dimensional image browser 103, a clock part 104, an advertisement placing information storage part 105, a landmark position database 110, a search engine 120, an all movement history storing part 121, a range inside and outside judgment distance storage part 122, a stay time threshold value storage part 123, a stay time calculation part 124, a user stay landmark judgment part 125, a user stay landmark storage database 126, an all information search command history storing part 127, and a questionnaire summing result presentation part 128. The same components as those of the above-mentioned embodiments of the present invention are indicated by similar numerals, and the description thereof is omitted. [0131] The all movement history storing part 121 stores movement history information including the viewpoint position of the user and the time. [0132] The range inside and outside judgment distance storage part 122 defines a range around a landmark which is a judgment reference for judging that the user is interested in a landmark. For example, when the user is within a radius of 500 m from a certain landmark and the user is assumed to be interested in the landmark, the range inside and outside judgment distance storage part 122 stores a value of “within a radius of 500 m”. [0133] The stay time threshold value storage part 123 stores a stay time threshold value showing a stay time to stay around a landmark which is a judgment reference for judging that the user is interested in a landmark. For example, when the user stays around a certain landmark for over one minute and the user is assumed to be interested in the landmark, the stay time threshold value storage part 123 stores a time of the “one minute”. [0134] The user stay landmark storage database 126 stores a landmark stay history including a landmark where the user stays and the stay time. [0135] The all information search command history storing part 127 records an information provision command history including the name of a landmark in which the user commands related information provision and time in which the user commands related information provision. [0136] The stay time calculation part 124 refers to the landmark position database 110 and extracts the landmark position through which the user passes and the time from the movement history information stored by the all movement history storing part 121. The stay time calculation part 124 calculates the first time and the last time when the viewpoint position of the user is within the definition of the periphery stored in the range inside and outside judgment distance storage part 122 from a certain landmark. [0137] The user stay landmark judgment part 125 compares a difference between the last time and the first time calculated by the stay time calculation part 124 with the threshold value stored in the stay time threshold value storage part 123. When the difference between the times is above the threshold value, the user stay landmark judgment part 125 judges that the user stays at the landmark and stores the landmark name and the difference value into the user stay landmark storage database 126. [0138] When the manager commands to provide the summing result, the questionnaire summing result provision part 128 outputs the landmark stay history of the user stay landmark storage database 126 or the information provision command history of the all information search command history storing part 127. [0139] The information providing server according to the sixth embodiment of the present invention can record all information showing the types of users and their visiting time and location, and sum up the social trends showing the types of people and their gathering location. For example, the information providing server 1 grasps all the information that how many people goes to see cherry blossoms of Yoshino, Nara Prefecture on April 30 when spring cherry blossoms are in full blossom or where the people visit after seeing the cherry blossoms of Yoshino. The information providing server 1 provides the information so that the user who receives information provision can make very strong marketing information based on the information. [0140] The information providing server according to the sixth embodiment of the present invention allows the manager of the information providing server 1 to propose an effective advertisement placing position by presenting the marketing result to the sponsor side and differentiate an advertisement placing fee in an area where users gather from that of other areas. [0141] When the user uses the information providing server 1, the user is admitted to use it anonymously. The reason is that the manager of the information providing server 1 knows about where the user performs searching in a state that existence of the user is apparent is an invasion of the user's privacy and the user is difficult to admit it. This enables the user side to use the information providing server 1 with a sense of security. [0142] The information providing server 1 can report to a sponsor as an advertisement placing person that how many users look at an advertisement. [0143] In the above-mentioned fourth to sixth embodiments of the present invention, the position input type three-dimensional image synthesizing part may be used in place of the three-dimensional image synthesizing part 102, or the position output type three-dimensional image browser may be used in place of the three-dimensional image browser 103. [0144] The system configurations of the above-mentioned embodiments is a client/server type distribution process system, but the present invention is not limited to these configurations. In other words, various modifications can be executed within the range without deviating from the purpose of the present invention. [0145] The information providing server and client can be realized by the configuration shown in FIG. 19. Referring to FIG. 19, the information providing server 1 in this embodiment has a data processor 301 configured by a CPU, a storage unit 302 configured by a memory whose read and write time is short, a recording medium 303 such as a hard disk having a large storage capacity, and an input unit 304 and an output unit 305 which control sending/receiving of data to/from an external device via a line. In this configuration, for example, with respect to the information providing server 1 shown in FIG. 2, the data processor 301 generates, executes and operates the three-dimensional image data conversion part 101, the three-dimensional image synthesizing part 102 and the three-dimensional image browser 103. A program for generating, executing and operating these parts is stored onto the recording medium 302. [0146] When the data processor 301 generates, executes and operates the three-dimensional image data conversion part 101, the three-dimensional image synthesizing part 102 and the three-dimensional image browser 103, the data processor 301 reads and executes each of the programs from the recording medium 302. The advertisement placing information storage part 105, the advertisement contract storage part 107, and the three-dimensional image database 108 are stored into the storage unit 302 or the recording medium 303. [0147] Referring to FIG. 19, the client terminal in this embodiment has a data processor 301 configured by a CPU, a storage unit 302 configured by a memory whose read and write time is short, a recording medium 303 such as a hard disk having a large storage capacity, and an input unit 304 and an output unit 305 which control sending/receiving of data to/from an external device via a line. In this configuration, for example, with respect to the client terminal 2 shown in FIG. 10, the data processor 301 generates, executes and operates the position output type three-dimensional image browser 201. A program for generating, executing and operating the position output type three-dimensional image browser 201 is stored onto the recording medium 302. [0148] When the data processor 301 generates, executes and operates the position output type three-dimensional image browser 201, the data processor 301 reads and executes the program from the recording medium 302. As described above, the system of the present invention can replace a manual operation for making a map using an artificial satellite image or aerial photographs with a mechanical operation and automatically synthesize and output advertisement information. The present invention can easily update map information or advertisement information. For example, using IKONOS, the entire surface of the earth can be updated every week. Since map information is automatically updated using an artificial image satellite, the information providing server can provide the user with latest map information on an area where rebuilding of a building in the center of a city is frequent together advertisement information. [0149] Much information related to information on weather or volcanoes is provided on WWW by an information provider. Each information provider updates the related information to latest information. Without updating the related information by the manager of the new information providing server 1, the present invention allows the information providing server 1 to provide latest information within the related information in which the user is interested. In the information providing server of the prior art method, the server manager must paste related information onto a map. In order that the information providing server of the prior art method can provide latest information, the manager himself/herself must always update information manually. In consideration of an actual operation, such an operation is not realistic. The present invention can clear the problem and provide latest information to the user. [0150] Since the information providing server 1 of the present invention generates and provides a three-dimensional stereoscopic image corresponding to change of the viewpoint position from the user, the user can search for related information while the user is sightseeing in the three-dimensional stereoscopic image. The information providing server 1 of the present invention can give the user a visual effect which is much more enjoyable than the information search from the prior art map. [0151] Further, the present invention allows the user to use the information providing server 1 for free and can establish the operation of the information providing server 1 in business. This can provide the user with a three-dimensional aerial sightseeing service on the Internet, but to establish it as business, the flow of money must be created. Of course, collecting a fee from the user is most direct. However, on the Internet, there are few successful examples in business in which a fee is collected directly from the user. The present invention employs a form in which an advertisement placing fee is collected from an advertisement placing person like a commercial TV broadcasting station and can make a three-dimensional aerial sightseeing service by the advertisement placing fee into business. In other words, provision of an expensive three-dimensional aerial sightseeing service can be provided to the user for free. The information providing server 1 allows the three-dimensional aerial sightseeing user who searches for information in the three-dimensional aerial sightseeing service to look at the advertisement irrespective of his/her own intension. The information providing server 1 of the present invention gives a large merit to the advertisement provider. [0152] The manager of the information providing server 1 changes an advertisement placing fee corresponding to a location desired to place an advertisement, so that the business range can be increased. For example, the sponsor side compares a placing fee decided by a placed location and a placing period with his/her own budget, and selects the placed location corresponding to the budget and requests the information providing server 1 to place an advertisement. [0153] When the user of the present invention looks at an advertisement unconsciously placed while the user is walking in the air by the three-dimensional aerial sightseeing service and thinks of purchasing of the product, the user can use the Internet shopping there. In other words, although the user confirms a placed advertisement handbill and then goes shopping for the product as in the prior art, the user can take a procedure for purchasing the product on the client terminal. [0154] The above embodiments are preferable embodiments of the present invention, and various modifications can be executed within the range without deviating from the purpose of the present invention. Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS5848373 *Jul 18, 1997Dec 8, 1998Delorme Publishing CompanyComputer aided map location systemUS6734873 *Jul 21, 2000May 11, 2004Viewpoint CorporationMethod and system for displaying a composited imageUS6882348 *Jan 16, 2002Apr 19, 2005Sony CorporationInformation displaying system, information providing apparatus, and information providing methodUS6948131 *Mar 8, 2000Sep 20, 2005Vidiator Enterprises Inc.Communication system and method including rich media toolsUS20020073196 *Dec 7, 2000Jun 13, 2002Westervelt John P.Integrated communication and geographic positioning system and method of using same* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS7958115Jul 29, 2004Jun 7, 2011Yahoo! Inc.Search systems and methods using in-line contextual queriesUS8051080Jun 3, 2008Nov 1, 2011Yahoo! Inc.Contextual ranking of keywords using click dataUS8069099 *Jun 26, 2008Nov 29, 2011Yahoo! Inc.Systems and methods for presenting advertising content based on publisher-selected labelsUS8108385Sep 3, 2009Jan 31, 2012Yahoo! Inc.User interfaces for search systems using in-line contextual queriesUS8150868Jun 11, 2007Apr 3, 2012Microsoft CorporationUsing joint communication and search dataUS8155431 *Oct 24, 2006Apr 10, 2012Olympus CorporationImage file processing apparatus which generates an image file to include stereo image data, collateral data related to the stereo image data, information of a date and time at which the collateral data is updated, and information of a date and time at which the image file is generated or updated, and corresponding image file processing methodUS8301614Jan 7, 2012Oct 30, 2012Yahoo! Inc.User interfaces for search systems using in-line contextual queriesUS8478792Jul 2, 2008Jul 2, 2013Yahoo! Inc.Systems and methods for presenting information based on publisher-selected labelsUS8655872Oct 15, 2008Feb 18, 2014Yahoo! Inc.Search systems and methods using in-line contextual queriesUS8693764 *Mar 5, 2012Apr 8, 2014Olympus CorporationImage file processing apparatus which generates an image file to include stereo image data and collateral data related to the stereo image data, and information related to an image size of the stereo image data, and corresponding image file processing methodUS8751559Sep 16, 2008Jun 10, 2014Microsoft CorporationBalanced routing of questions to expertsUS8812540Sep 28, 2012Aug 19, 2014Yahoo! Inc.User interfaces for search systems using in-line contextual queriesUS8943535 *Jul 29, 2013Jan 27, 2015Yahoo Japan CorporationAdvertisement distribution apparatus and advertisement distribution methodUS8970669Sep 30, 2009Mar 3, 2015Rovi Guides, Inc.Systems and methods for generating a three-dimensional media guidance applicationUS9055289 *Mar 22, 2012Jun 9, 2015Korea Institute Of Science And Technology3D display systemUS9171011Aug 27, 2013Oct 27, 2015Google Inc.Building search by contentsUS9195739Feb 20, 2009Nov 24, 2015Microsoft Technology Licensing, LlcIdentifying a discussion topic based on user interest informationUS9342602Jul 9, 2014May 17, 2016Yahoo! Inc.User interfaces for search systems using in-line contextual queriesUS9436945 *Apr 14, 2010Sep 6, 2016Redfin CorporationInteractive map-based search and advertisingUS9635066 *Oct 28, 2010Apr 25, 2017Znl Enterprises, LlcMethod and apparatus for entertainment and information services delivered via mobile telecommunication devicesUS20060026013 *Jul 29, 2004Feb 2, 2006Yahoo! Inc.Search systems and methods using in-line contextual queriesUS20060159028 *Nov 14, 2005Jul 20, 2006Martin Curran-GrayMonitoring system, method of sampling datagrams, and apparatus thereforUS20070036444 *Oct 24, 2006Feb 15, 2007Olympus CorporationImage processing apparatus, image processing and editing apparatus, image file reproducing apparatus, image processing method, image processing and editing method and image file reproducing methodUS20070201633 *Sep 23, 2003Aug 30, 2007Delgado Everest BThree dimensional communication meansUS20080262931 *Jun 26, 2008Oct 23, 2008Alwin ChanSystems and methods for presenting advertising content based on publisher-selected labelsUS20080306935 *Jun 11, 2007Dec 11, 2008Microsoft CorporationUsing joint communication and search dataUS20080320021 *Jul 2, 2008Dec 25, 2008Alwin ChanSystems and methods for presenting information based on publisher-selected labelsUS20090070326 *Oct 15, 2008Mar 12, 2009Reiner KraftSearch systems and methods using in-line contextual queriesUS20090132376 *Oct 20, 2008May 21, 2009Jorn KollinMethod for using visible surfaces as advertising surfaces for aerial image and satellite recordingsUS20090265338 *Jun 3, 2008Oct 22, 2009Reiner KraftContextual ranking of keywords using click dataUS20100070484 *Sep 3, 2009Mar 18, 2010Reiner KraftUser interfaces for search systems using in-line contextual queriesUS20100070554 *Sep 16, 2008Mar 18, 2010Microsoft CorporationBalanced Routing of Questions to ExpertsUS20100198684 *Apr 14, 2010Aug 5, 2010Redfin CorporationInteractive map-based search and advertisingUS20100228777 *Feb 20, 2009Sep 9, 2010Microsoft CorporationIdentifying a Discussion Topic Based on User Interest InformationUS20100293035 *May 15, 2009Nov 18, 2010Athellina AthsaniMap advertising system interfaceUS20100312771 *Jun 8, 2010Dec 9, 2010Microsoft CorporationAssociating Information With An Electronic DocumentUS20110040615 *Oct 28, 2010Feb 17, 2011Weisman Jordan KMethod and apparatus for entertainment and information services delivered via mobile telecommunication devicesUS20110074918 *Sep 30, 2009Mar 31, 2011Rovi Technologies CorporationSystems and methods for generating a three-dimensional media guidance applicationUS20110137727 *Dec 7, 2009Jun 9, 2011Rovi Technologies CorporationSystems and methods for determining proximity of media objects in a 3d media environmentUS20120163705 *Mar 5, 2012Jun 28, 2012Olympus CorporationImage file processing apparatus which generates an image file to include stereo image data and collateral data related to the stereo image data, and information related to an image size of the stereo image data, and corresponding image file processing methodUS20130127832 *Mar 22, 2012May 23, 2013Ho Dong LEE3D DlSPLA Y SYSTEMUS20140059588 *Jul 29, 2013Feb 27, 2014Yahoo Japan CorporationAdvertisement distribution apparatus and advertisement distribution methodUS20140129395 *Jan 13, 2014May 8, 2014Accenture Global Services LimitedDetection of stock out conditions based on image processingUS20160335670 *Jul 29, 2016Nov 17, 2016Redfin CorporationInteractive map-based search and advertisingCN102804120A *Nov 30, 2010Nov 28, 2012联合视频制品公司Systems and methods for determining proximity of media objects in a 3D media environmentCN103177230A *Nov 7, 2012Jun 26, 2013株式会社东芝Reader and reading methodWO2004063950A1 *Jan 15, 2004Jul 29, 2004Super Internet Site System Pty LtdSpatial classified advertising directoryWO2005029415A1 *Sep 23, 2003Mar 31, 2005Delgado Everest BarjauThree-dimensional communication meansWO2007121741A1 *Apr 26, 2007Nov 1, 2007Kollin JoernMethod for utilizing visible areas as advertising areas for aerial photographs and satellite pictures* Cited by examinerClassifications U.S. Classification382/154International ClassificationG06Q30/06, G06Q50/00, G06F17/30, G06T3/00, H04N7/173, G06F13/00Cooperative ClassificationG06Q30/02European ClassificationG06Q30/02Legal EventsDateCodeEventDescriptionDec 21, 2001ASAssignmentOwner name: NEC CORPORATION, JAPANFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SHIMAZU, HIDEO;REEL/FRAME:012397/0715Effective date: 20011218RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services