Aerial image collection

In one embodiment, an aerial collection system includes an image collection field vehicle that travels at street level and an image collection aerial vehicle that travels in the air above the street. The aerial vehicle collects image data including at least a portion of the field vehicle. The field vehicle includes a marker, which is identified from the collected image data. The marker is analyzed to determine an operating characteristic of the aerial vehicle. In one example, the operating characteristic in the marker includes information for a flight instruction for the aerial vehicle. In another example, the operating characteristic in the marker includes information for the three dimensional relationship between the vehicles. The three dimensional relationship is used to combine images collected from the air and images collected from the street level.

FIELD

The following disclosure relates to image collection from an aerial vehicle, or more particularly, an analysis of a relationship between a field vehicle and image collection from an aerial vehicle.

BACKGROUND

Road level imagery may be collected using video cameras in a vehicle. Road level imagery provides adequate images of building facades around eye level or the level of the video cameras in the vehicle. However, larger buildings cannot be adequately imaged from vehicle-mounted cameras. Also, vegetation, road signs, or other obstacles may occlude building facades from the view of vehicle-mounted cameras.

Aerial photography from satellites and airplanes provide another view. Satellites orbit hundreds of miles to thousands of miles above the surface of the Earth and can provide only low detail images of wide geographic areas. Airplanes fly a minimum of four hundred feet above the surface of the Earth and also provide relatively low detail images. Airplanes are also incapable of adequately collecting images of the sides of buildings because of the angle of the line of sight. Images may be manually collected by static cameras as an operator moves along the road facade carefully ensuring that the road level imagery is adequately collected. However, this technique is too labor intensive. Another type of imagery collection is needed to provide the level of detail available in images collected by vehicle mounted cameras but from another perspective.

SUMMARY

In one embodiment, an aerial collection system includes an image collection field vehicle that travels at street level and an image collection aerial vehicle that travels in the air above the street. The aerial vehicle collects image data including at least a portion of the field vehicle. The field vehicle includes a marker, which is identified from the collected image data. The marker is analyzed to determine an operating characteristic of the aerial vehicle. In one example, the operating characteristic in the marker includes information for a flight instruction for the aerial vehicle. In another example, the operating characteristic in the marker includes information for the three dimensional relationship between the vehicles. The three dimensional relationship is used to combine images collected from the air and images collected from the street level.

DETAILED DESCRIPTION

Street side imagery is collected by field vehicles equipped with cameras. The imagery is limited by the range of the cameras. For example, some portions of the building may be blocked by shrubs, trees, parked cars, people, or other objects between the field vehicle and the building facade. Also, high portions of the building facade may be out of range or out of the field of view of vehicle mounted cameras. An aerial vehicle may be used in cooperation with the field vehicle. The aerial vehicle may include a camera with a field of view that overlaps or otherwise supplements the camera in the vehicle. The aerial vehicle may track a field vehicle and follow the field vehicle. The image data collected by the aerial vehicle may be combined with the image data collected by the field vehicle. The aerial vehicle may be an autonomous vehicle such as an unmanned aerial vehicle (UAV). The aerial vehicle may be radio controlled helicopter (e.g., quadcopter or other helicopter).

The tracking of the field vehicle may utilize a marker on the top or the side of the field vehicle. The marker may be an augmented reality marker. The aerial vehicle includes a camera or other image detection device configured to detect the marker. The marker may provide a path or target for the aerial vehicle to follow. The marker may provide an instruction for the flight operation of the aerial vehicle. For example, the marker may define a distance from the field vehicle to the aerial vehicle or a speed of the aerial vehicle.

The combination of the image data of the aerial vehicle with the image data collected by the field vehicle to create a single image or related images may also use information stored in the marker. The image data for the marker collected by the aerial vehicle may be analyzed to identify a size and/or orientation of the marker. The size of the marker relates to the distance between the field vehicle and the aerial vehicle. The relative proportions and orientation of the marker relates to the angle between the field vehicle and the aerial vehicle. Therefore, the spatial relationship between the aerial vehicle and the field vehicle may be calculated from the marker. The image data of the aerial vehicle and the image data of the field vehicle may be combined according to the spatial relationship.

FIG. 1illustrates an exemplary system100for collecting and generating street side imagery. The system100includes a developer system121, a field vehicle141, an aerial vehicle129, and optionally, a network127. Additional, different, or fewer components may be provided. The developer system121includes a server125and a database123. The developer system121may include computer systems and networks of a system operator (e.g., NAVTEQ or Nokia Corp.). The field vehicle141includes a terrestrial mobile device10and the aerial vehicle129includes an aerial mobile device11.

The server125or the aerial mobile device11may receive image data of the field vehicle141that is collected by the aerial vehicle129. The marker from the field vehicle141is identified from the image data and analyzed to determine an operating characteristic of the aerial vehicle129. The marker may be an augmented reality code or a quick response (QR) code. The analysis of the marker may allow the aerial vehicle129to follow an appropriate flight path. For example, the mobile device11of the aerial vehicle129may compare subsequent images of the marker to determine the direction of travel of the field vehicle141. The analysis of the marker may allow the image data collected by the aerial vehicle129to be combined with another image. For example, the mobile device11or the server125may align images from the aerial vehicle129and images from the field vehicle141according to the orientation of the marker.

The mobile devices10and11may include a data collection component and a data processing component. The data collection component may include one or any combination of a camera, a light distance and ranging (LIDAR) device, an inertial measurement unit (IMU), and a global positioning system (GPS). The data processing component includes a computer for processing data collected by the data collection component. The mobile devices10and11may be a smart phone, a mobile phone, a personal digital assistant (PDA), a tablet computer, a notebook computer, a personal navigation device (PND), a portable navigation device, and/or any other known or later developed portable or mobile computing device including or coupled to a camera. The camera may include an array of cameras pointing in multiple directions. The array of camera may include 2 to 8 cameras for each side as well as one or more cameras angled up and/or down.

The developer system121and the mobile devices10and11are coupled with the network127. The phrase “coupled with” is defined to mean directly connected to or indirectly connected through one or more intermediate components. Such intermediate components may include hardware and/or software-based components.

The computing resources may be divided between the server125and either of the mobile devices10and11. In some embodiments, the server125performs a majority of the processing. In other embodiments, the mobile device10or11performs a majority of the processing. In addition, the processing is divided substantially evenly between the server125and the mobile device10or11. In another embodiment, the mobile devices10and11operation without connection to the server125other than for storage of image data.

FIG. 2illustrates an example guidance system for aerial image collection. The guidance system includes an aerial vehicle129and a terrestrial vehicle (e.g., field vehicle141). The terrestrial vehicle141may include a marker142. The terrestrial vehicle141may be an automobile traveling along the road133and include data collection equipment. The data collection equipment may include one or any combination of an array of cameras, an inertial measurement unit (IMU), and a global positioning system (GPS).

The building facade131and other objects are imaged by cameras coupled to mobile device10and11in the aerial vehicle129and the terrestrial vehicle141. In one example, one or more cameras in the aerial vehicle129have an aerial field of view137and one or more cameras in the terrestrial vehicle141have a ground field of view135. The aerial field of view137and the ground field of view135may overlap or be adjacent to one another. Either or both of the aerial field of view137and the ground field of view135may be applied to both sides of road133. In addition, panoramic cameras or wide angle cameras may be used. A panoramic camera may be any camera having a field of view wider than that of the human eye (e.g., greater than 80 degrees). A 360 degree camera may also be used.

The mobile device10may control the overlap and/or intersection of the aerial field of view137and the ground field of view135. The mobile device10may generate a command for the one or more cameras in the aerial vehicle129. The command may control the position, orientation, and zoom to set the aerial field of view137. The mobile device10may also generate a command for the position, orientation, and/or zoom of the one or more cameras in the terrestrial vehicle141to set the ground field of view135.

In addition or in the alternative, the command may describe the flight path for the aerial vehicle129to use. The command may provide individual directional commands to the aerial vehicle129. The command may include position data describing the position of the terrestrial vehicle141. The aerial vehicle129may be configured to follow the position of the terrestrial vehicle141. The flight path may be designed to maintain a substantially constant distance between the aerial vehicle129and the terrestrial vehicle141.

The flight command may instruct the aerial vehicle129to land on the terrestrial vehicle141. For example, the mobile device10may receive data from a geographic database that a tunnel, overpass, or other obstruction is approaching. The aerial vehicle129may be secured to the terrestrial vehicle141by a magnetic landing pad or hook.

The commands may be sent from the mobile device10to the aerial vehicle129through a variety of techniques. For example, marker142may relay the command to the aerial vehicle. The marker142may be encoded with data for the command. The data142may be encoded as a quick response (AR) code, a universal product code (UPC), an alphanumeric code, a hexadecimal code, a binary code, a shape, or another code. The aerial vehicle129includes a camera or another imaging device to read and decode the marker142. The camera may be the same or a different camera as the camera configured to collect images of the building facade131.

Alternatively, the command may be transmitted from the mobile device10to the aerial vehicle129through direct or indirect radio communication. The direct radio communication may include wireless communication though protocols known as Wi-Fi, the protocols defined by the IEEE 802.11 standards, the protocols defined by the Bluetooth standards, or other protocols. The indirect radio communication may include signals or data packets through the network127. The network127may include any combination of a cellular network, the Internet, or a local computer network.

The mobile device11, the mobile device10, or the server125may be configured to interpolate and/or stitch together images collected by cameras in the aerial vehicle129and images collected by cameras in the terrestrial vehicle141. Both images may be panoramic images. The images may be stitched together based on the content of the images. For example, common objects in one of the images may be aligned through image rectification or registration. Image rectification identifies lines of object outlines in the images and stretches or shrinks the two images to maximize the number of lines in the images that line up. The object outlines may be vertical (e.g., building edges) horizontal (e.g., road stripes or windows), or at other angles. Other techniques such as feature recognition, reverse image extraction, and multipoint distortion may be used to stitch the images together The images may be stitched together based on the marker142. The shape and size of the marker142depends on the relative orientation and position of the aerial vehicle129. For example, the smaller the marker142appears in images collected by the aerial vehicle129, the farther away the marker142, and the greater the distance between the aerial vehicle129and the field vehicle141. Likewise, an angle of the marker142with respect to the aerial vehicle129depends on the orientation of the aerial vehicle129. Through the angle and distance relating the aerial vehicle129and the field vehicle129, the images may be aligned.

The images may be combined through interpolation. For example, after the images from the aerial vehicle129and the terrestrial vehicle141are aligned, corresponding pixels that overlap the two images are averaged or otherwise mathematically combined or adjusted. The pixels may be averaged in intensity, color, brightness or another attribute. Through interpolation, a single image at a single perspective is generated by the server125or the mobile device11.

In one example, the two images are interpolated at with different weighting as a function of space. Portions of the images are given a weight that favors the images collected by the aerial vehicle129and portions of the images are given a weight that favors images collected by the terrestrial vehicle141. For example, the weighting may vary according to elevation such that only images from the aerial vehicle129are used at a first height in the combined image and only images from the terrestrial vehicle141are used at a second height in the combined image. The first height is greater than the second height. In between the first height and the second height, portions of both images are used. The portions of the respective images may be a linear relationship such that the percentage weight for the images from the aerial vehicle129plus the percentage weight for the images from the terrestrial vehicle141equals a constant (e.g., 100%).

FIG. 3illustrates another example guidance system for aerial image collection through collaboration between an aerial vehicle150and a terrestrial vehicle141. The terrestrial vehicle154includes a tracking marker155. The tracking marker155may be generated by an electronic display mounted on the roof of the terrestrial vehicle154and changed in time based on control by the mobile device10. The tracking marker155may be painted on the terrestrial vehicle154or affixed to the terrestrial vehicle154as a sticker, magnetic sign, or decal. Alternatively, the tracking marker155may be a sign that is interchangeably installed into a guide or slot on the roof of the terrestrial vehicle154so that the signs may be easily removed and interchanged.

The aerial vehicle150may include horizontal cameras151and at least one vertical camera152. The horizontal cameras151are configured to collect images of the building facades131. The vertical camera152is configured to collect images of the tracking marker155and/or the terrestrial vehicle154. The vertical camera152may be an infrared camera and the tracking marker155may include infrared paint. The vertical camera152may be a scanner or a low resolution camera configured to identify specific patterns or shapes. A single wide angle camera or rotatable camera may be used in places of the horizontal cameras151and the vertical camera152.

The terrestrial vehicle154also includes one or more cameras to collect images of the building facade131. Both sets of images may be sent to another location for processing by a computer. The computer may be configured to derive a three dimensional model. The computer may derive the relative positions of the aerial vehicle150and the terrestrial vehicle154using the size and shape of the tracking marker155. The three dimensional model may be augmented using range data. The range data may be collected using a LIDAR device in one or both of the aerial vehicle150and the terrestrial vehicle154. The LIDAR device is an optical sensor that detects distances of objects. The distances may be stored in a point cloud with each point includes a spatial position and/or intensity. The point cloud may be used to align objects in the multiple images collected by the aerial vehicle150and the terrestrial vehicle154. The LIDAR point cloud provides three-dimensional positions for the objects in the images collected by the aerial vehicle150and the terrestrial vehicle154. The mobile device11or the server125is configured to identify a common object in the images from the LIDAR point clouds. Based on the relative positions of the common object in the LIDAR point clouds, one of the images may be rotated and scaled to be aligned with the other images.

The images of the building facade131collected by the aerial vehicle150and the terrestrial vehicle154may be time coded. The time codes may include the time that the images are collected. A time differential may be calculated that indicates a distance between (e.g., a horizontal distance) the aerial vehicle150and the terrestrial vehicle154. The time differential may be an amount of time that the aerial vehicle150is expected to travel the horizontal distance between the aerial vehicle150and the terrestrial vehicle154. The server125may be configured to select images based on the time codes and may translate time codes from one set of images by adding or subtracting the corresponding time differential such that substantially aligned images are selected.

FIGS. 4A-Cand5A-C illustrate example markers for the guidance systems ofFIGS. 2 and 3.FIG. 4Aillustrates a marker160including a cross-pattern161. The cross pattern includes horizontal and vertical lines. Angles between the horizontal lines and the vertical lines may be calculated through image processing by the mobile device11, the mobile device10, or the server125. The relative dimensions of the cross pattern171may be measured by the mobile device11, mobile device10, or the server125. The orientation of the cross pattern171based on the angles and relative dimensions may define the orientation of the terrestrial vehicle or the aerial vehicle. The orientation defines the spatial relationship between images collected at the terrestrial vehicle and images collected at the aerial vehicle.

FIG. 4Billustrates a marker162including a QR code163. The QR code163encodes data as a matrix barcode. The data may include a flight instruction for the aerial vehicle. The QR code may also be measured to determined angles and relative sizes to determine the orientation and spatial relationship of the terrestrial vehicle and the aerial vehicle.

FIG. 4Cillustrates a marker164including augmented reality shapes163. The augmented reality shapes are basic shapes with easily measurable geometries. The shapes may correspond to specific instructions (e.g., speed up, slow down, turn left, turn right, or other commands). The specific instructions may include a flight path appropriate for a geographic position of the aerial vehicle (e.g., urban settings, rural settings, utility lines, tunnels, overpasses, or other potential interference) or environmental factors (e.g., weather, rain, sunshine, visibility, fog, light, darkness, time of day, season, or another factor). The reality shapes163may be a set of commands sequenced in time. For example, one of the shapes may convey a current flight operation of the aerial vehicle and the other shapes convey upcoming commands. The aerial vehicle may image all four commands to determine upcoming maneuvers. The shapes may rotate and disappear as time passes.

FIG. 5Aillustrates a marker166including a grid167. The grid167is made up of horizontal and vertical lines. The mobile device11or the server125is configured to process images of the grid167to calculate angles between the lines.FIG. 5Billustrates a marker168including a polygon169. The polygon169is sized such that another shape appears when the aerial vehicle flies at a specific height and distance from the terrestrial vehicle. For example, the polygon169shown inFIG. 5Bmay appear as a rectangle in an image collected at the aerial vehicle when the aerial vehicle is at the specified height and distance from the terrestrial vehicle.FIG. 5Cillustrates a marker170including a plurality of shapes. The marker170may include a QR code163defining a flight command and a shape161for measuring the relative distance and orientation between vehicles. In one example, the marker includes an AR code at the corners (e.g., four corners).

FIG. 6illustrates another example guidance system for aerial image collection. The terrestrial vehicle141is in communication with multiple aerial vehicles such that multiple roads133are imaged simultaneously. A first aerial vehicle190is configured to track the marker142on the terrestrial vehicle141. A second aerial vehicle191is configured to track an aerial vehicle marker182on the first aerial vehicle190. Optionally, a third aerial vehicle192is configured to track another aerial vehicle marker183on the second aerial vehicle191. The second aerial vehicle191and the third aerial vehicle192may track and detect orientation using any of the implementations discussed above. In addition, the aerial vehicle190may receive commands intended for the second aerial vehicle191from the marker142and relay the commands using the aerial vehicle marker182. The aerial markers182and183may be constant (e.g., signs) or variable (e.g., electronic display). The second aerial vehicle191and the third aerial vehicle192may omit horizontal cameras and/or aerial markers. In other embodiments, the multiple aerial vehicles track a common marker or different markers on a common vehicle (e.g., terrestrial vehicle). Wireless communications may be used to guide the multiple aerial vehicles in yet other embodiments.

FIG. 7illustrates an exemplary server of the system ofFIG. 1. The server125includes a processor300, a communication interface305, and a memory301. The server125may be coupled to a database123and a workstation310. The workstation310may be used as an input device for the server125. In addition, the communication interface305is an input device for the server125. The communication interface305receives data indicative of use inputs made via the workstation310or the mobile devices10or11.

The communication interface305is also configured to receive image data from mobile device10associated with the terrestrial vehicle141and image data from the mobile device11associated with the aerial vehicle129. The communication may be through a direct connection or through network127.

The processor300, which may be any type of controller, is configured to calculate a spatial relationship between the terrestrial vehicle and the aerial vehicle based on the image data from the mobile device11. The image data from the mobile device11includes a stored predetermined shape. The processor300analyzes the shape to determine how far the shape was from the aerial vehicle129when the image was taken. The processor300also analyzes the shape to determine an orientation and/or spatial relationship of the aerial vehicle129with respect to the terrestrial vehicle141.

The processor300is configured to combine the image data collected by the aerial vehicle129and the image data collected by the terrestrial vehicle141according to the orientation and/or spatial relationship between the terrestrial vehicle and the aerial vehicle. The processor300may be configured to translate, rotate, and scale one image to the three dimensional frame of reference of the other image so that the images can be stitched or spliced together. The processor300may also be configured to identify a registration object or registration line in the images, and translate, rotate, or scale one of the image based on the registration object or registration line in order to align the images.

The processor300may also be configured to analyze the image data collected by the aerial vehicle129to identify a flight command for the aerial vehicle129. The processor300identifies the flight command and sends the command to the mobile device11through radio communication. The flight command may include a height, a speed, or a turn for the aerial vehicle129. The flight command may instruct the aerial vehicle129to land on the terrestrial vehicle141.

FIG. 8illustrates an exemplary mobile device of the system ofFIG. 1. The mobile device10or11includes a controller200, a memory204, an input device203, a communication interface205, position circuitry207, an imaging device213, and a display211. Optionally, a range finding device215is coupled to or integrated with the mobile device10or11. In the following endpoint-based embodiments, the mobile device10or11performs a majority of the processing.FIG. 9illustrates an example flowchart for aerial image collection and processing, which is described in relation to the mobile device10or11but may be performed by another device. Additional, different, or fewer acts may be provided.

At act S101, the controller205is configured receive image data of a terrestrial vehicle collected by an aerial vehicle. The image data may also depict the surrounding buildings and other objects. The image data may be received directly from the imaging device213or from memory204.

At act S103, the controller205is configured to identify a marker from the image data. The marker may be an image of a physical mark on the terrestrial vehicle. The marker may be encoded with information. The marker may be a predetermined shape such that the controller205is configured to compare the captured image of the marker to a template. Based on the comparison, the controller205may identify changes in the marker and/or the orientation in the marker.

At act S105, the controller205is configured to analyze the marker to determine an operating characteristic of the aerial vehicle. As shown at act S107, the operating characteristic may be a flight instruction for the aerial vehicle. For example, the marker may instruct the aerial vehicle to fly at a specific altitude, at a specific distance from the terrestrial vehicle, or at a specific speed. The marker may instruct the vehicle to turn right, turn left, fly under a tunnel, or fly over an overpass.

The operating characteristic may be the orientation of the aerial vehicle or camera. The controller205may be configured to analyze the marker to determine the spatial relationship between the aerial vehicle and the terrestrial vehicle and/or the orientation of the aerial vehicle. As shown as act S109, the controller205may be configured to combine the image data from the aerial vehicle with an image collected by the terrestrial vehicle (or previously collected in any fashion) according to the spatial relationship and/or orientation of the aerial vehicle. Acts S107and S109may be performed in succession, simultaneously, or in the alternative. The processor300of the server125may perform one or more of acts S101through S109.

The imaging device213may include a single camera or an array of cameras. The array of cameras may be arranged to provide a 360 degree field of view. The image device213may additionally include a low resolution camera or scanner for imaging the marker.

The range finding device215may be an optical ranging device configured to send a laser or other signal and receive the signal back after the signal reflects off objects. The range finding device215generates distance data or point cloud data of position based on the timing of the reflected signal. The data may be referred to as depth map. The range finding device215may be a LIDAR sensor. The LIDAR sensor may generate binary data (e.g., on or off). An intensity value may be included in place of the on or off value. Alternatively, the data points may include a distance value and two angle values to signify a location of a point in the point cloud. The point cloud may be stored in ASCII or LIDAR exchange format. One or more lasers of the LIDAR device may be in a near infrared spectrum (such as about 700 nm to about 5000 nm or about 800 nm to about 2500 nm) or another light spectrum.

The input device203is configured to receive a selection for manually entering flight commands for the aerial vehicle. The input may include a distance between vehicles or a target altitude for the aerial vehicle. The input may instruct an angle for the cameras of either vehicle or an amount of overlap for the cameras. The input device203may be one or more buttons, keypad, keyboard, mouse, stylist pen, trackball, rocker switch, touch pad, voice recognition circuit, or other device or component for inputting data to the mobile device10or11. The input device203and the display211may be combined as a touch screen, which may be capacitive or resistive. The display211may be a liquid crystal display (LCD) panel, light emitting diode (LED) screen, thin film transistor screen, or another type of display. Alternatively, the display211of the mobile device10may be mounted on the terrestrial vehicle or viewable through a window of the terrestrial vehicle. The display211may be configured to display the markers described above.

The positioning circuitry207is optional and may be excluded for map-related functions. The positioning circuitry207may include a Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), or a cellular or similar position sensor for providing location data. The positioning system may utilize GPS-type technology, a dead reckoning-type system, cellular location, or combinations of these or other systems. The positioning circuitry207may include suitable sensing devices that measure the traveling distance, speed, direction, and so on, of the mobile device10or11. The positioning system may also include a receiver and correlation chip to obtain a GPS signal. Alternatively or additionally, the one or more detectors or sensors may include an accelerometer built or embedded into or within the interior of the mobile device10or11. The accelerometer is operable to detect, recognize, or measure the rate of change of translational and/or rotational movement of the mobile device10or11. The mobile device10or11receives location data from the positioning system. The location data indicates the location of the mobile device10or11. Location data may be used for guiding the aerial vehicle rather than or in addition to the marker.

The controller200and/or processor300may include a general processor, digital signal processor, an application specific integrated circuit (ASIC), field programmable gate array (FPGA), analog circuit, digital circuit, combinations thereof, or other now known or later developed processor. The controller200and/or processor300may be a single device or combinations of devices, such as associated with a network, distributed processing, or cloud computing.

The memory204and/or memory301may be a volatile memory or a non-volatile memory. The memory204and/or memory301may include one or more of a read only memory (ROM), random access memory (RAM), a flash memory, an electronic erasable program read only memory (EEPROM), or other type of memory. The memory204and/or memory301may be removable from the mobile device10or11, such as a secure digital (SD) memory card.

The memory204, the memory301, or database123may be a geographic database. The geographic database may be used to guide the terrestrial vehicle and predict upcoming maneuvers of the terrestrial vehicle. Position data in the geographic database may be used to index the images collected by the terrestrial vehicle and the aerial vehicle. The geographic database123includes information about one or more geographic regions. Each road in the geographic region is composed of one or more road segments. A road segment represents a portion of the road. Each road segment is associated with two nodes (e.g., one node represents the point at one end of the road segment and the other node represents the point at the other end of the road segment). The node at either end of a road segment may correspond to a location at which the road meets another road, i.e., an intersection, or where the road dead-ends. The road segment data record may include data that indicate a speed limit or speed category (i.e., the maximum permitted vehicular speed of travel) on the represented road segment. The road segment data record may also include data that indicate a classification, such as a rank of a road segment that may correspond to its functional class. The road segment data may include data identifying what turn restrictions exist at each of the nodes which correspond to intersections at the ends of the road portion represented by the road segment, the name or names by which the represented road segment is known, the length of the road segment, the grade of the road segment, the street address ranges along the represented road segment, the permitted direction of vehicular travel on the represented road segment, whether the represented road segment is part of a controlled access road (such as an expressway), a ramp to a controlled access road, a bridge, a tunnel, a toll road, a ferry, and so on. Further, the represented road segment may include data indicative of the positions of tunnels, overpasses or other obstacles.

Navigation-related features, including a route calculation application, may display the street side images generated in the above embodiments. End users may access a route from an origin to a destination. The route calculation application determines the route for the end user to travel along the road segments to reach the desired destination. In order to calculate a route, the route calculation application is provided with data identifying a starting location (origin) and a desired destination location. In one embodiment, the starting location may be the end user's current position and the destination may be entered by the end user. Given at least the identification of the starting location (origin) and the desired destination location, the route calculation application determines one or more solution routes between the starting location and the destination location. A solution route is formed of a series of connected road segments over which the end user can travel from the starting location to the destination location. When the route calculation application calculates a route, the application accesses the geographic database123and obtains data that represent road segments around and between the starting location and the destination location. The road calculation application uses the data to determine at least one valid solution route from the starting location to the destination location. The at least one valid solution route may be displayed to the user in the rendering of the geographic region such that addresses or points of interest along the route may be selected to display street side imagery.

In one embodiment, the route calculation application may attempt to find a solution route that takes the least time to travel. The segment cost or travel time for the particular represented road segment considers the type of road, such as freeway or residential street, speed limit and distance of the segment. In one embodiment, the route calculation application may consider traffic conditions to more accurately reflect actual travel time over the connected road segments. When the route calculation application determines one or more solution routes comprising the series of connected road segments, the travel times for each of the included connected road segments is summed to provide an estimated route travel time. Based on the route travel time, the route calculation application selects the quickest route. Once the route calculation application has selected the route, the route calculation application provides an output in the form of an ordered list identifying a plurality of road segments that form the continuous navigable route between the origin and the destination. In addition, the route calculation program provides an output of an estimated route travel time.

Map-related features may also display the street side imagery generated in the embodiments above. The map-related features may be any of the navigation-related features provided to the user without reference to the current location of the user or the device. In addition, map-related features may include display and manipulation of a map of a geographic region. The map-related features may be provided without navigation-related features.

The communication interface205and/or communication interface305may include any operable connection. An operable connection may be one in which signals, physical communications, and/or logical communications may be sent and/or received. An operable connection may include a physical interface, an electrical interface, and/or a data interface. The communication interface205and/or communication interface305provides for wireless and/or wired communications in any now known or later developed format.

The memory401and/or memory204are a non-transitory computer-readable medium configured to store instructions for any of the implementation discussed above. While the non-transitory computer-readable medium is shown to be a single medium, the term “computer-readable medium” includes a single medium or multiple media, such as a centralized or distributed database, and/or associated caches and servers that store one or more sets of instructions. The term “computer-readable medium” shall also include any medium that is capable of storing, encoding or carrying a set of instructions for execution by a processor or that cause a computer system to perform any one or more of the methods or operations disclosed herein.