Patent Description:
The present disclosure relates generally to determining geographic orientation. More particularly, the present disclosure relates to determining geographic orientation based at least in part on imagery.

Mobile computing devices (e.g., smartphones, tablet computers, and/or the like) are ubiquitous and often include a panoply of sensors (e.g., cameras, global positioning system (GPS) receivers, proximity sensors, ambient-light sensors, accelerometers, magnetometers, gyroscopic sensors, radios, fingerprint sensors, barometers, facial-recognition sensors, and/or the like). Many modem mobile computing devices can accurately determine their geographic location (e.g., based on data generated by GPS receivers, received via wireless-network interfaces, and/or the like). Accurately determining geographic orientation of mobile devices, however, remains an intractable challenge. For example, while it is possible to determine geographic orientation based on data received via GPS receivers, such determinations are often inaccurate, particularly when the device is stationary and/or subject to interference from surrounding structures (e.g., located in an "urban canyon," and/or the like). Similarly, while geographic orientation can be determined using magnetometers, such determinations are frequently imprecise or erroneous because the devices themselves can interfere with magnetometers.

The document entitled "<NPL>, discloses a method for determining the direction of a road based on camera images using a learning-machine model.

One aspect of the present invention is directed to a computer-implemented method according to claim <NUM>.

Another aspect of the present invention is directed to a system according to claim <NUM>.

A further aspect of the present invention is directed to one or more non-transitory computer-readable media according to claim <NUM>.

Example aspects of the present disclosure are directed to determining geographic orientation based at least in part on imagery. In particular, a physical real-world environment can include a travelway. A travelway can include, for example, a street, road, avenue, lane, boulevard, highway, freeway, parkway, railway tracks, and/or the like, with or without one or more adjacent sidewalks, walkways, paths, curbs, shoulders, and/or the like. A user located in the environment can utilize a camera of a user device (e.g., a camera system, mobile computing device, smartphone, wearable device, and/or the like) to generate data representing imagery that includes at least a portion of the environment. In accordance with aspects of the disclosure, a computing system (e.g., the user device, a computing system remotely located from the user device, and/or the like) can receive such data and can determine, based at least in part on the data and a machine-learning model, a geographic orientation of the camera (e.g., the camera, the user device, the user, and/or the like) with respect to the travelway.

In some embodiments, a geographic orientation of the travelway with respect to the environment (e.g., with respect to the world, magnetic north, and/or the like) can be known, predetermined, determined by the computing system, and/or the like. In some of such embodiments, the computing system can determine, based at least in part on the geographic orientation of the camera with respect to the travelway and the geographic orientation of the travelway with respect to the environment, a geographic orientation of the camera (e.g., the camera, the user device, the user, and/or the like) with respect to the environment (e.g., with respect to the world, magnetic north, and/or the like).

According to the invention, the computing system determines (based at least in part on the data representing the imagery, the machine-learning model) two possible geographic orientations of the camera with respect to the travelway, which differ by one hundred and eighty degrees. For example, the computing system can determine the camera is facing directly up or down the travelway (e.g., oriented at a zero-degree angle or one-hundred-eighty-degree angle with respect to the travelway), perpendicular to the travelway one way or the other (e.g., oriented at a ninety-degree angle or two-hundred-seventy-degree angle with respect to the travelway), askew to the travelway one way or the other (e.g., oriented at a forty-five-degree angle or two-hundred-twenty-five-degree angle with respect to the travelway), and/or the like. According to the invention, the computing system selects the geographic orientation of the camera with respect to the travelway from amongst the two possible geographic orientations.

In some embodiments, the computing system can select the geographic orientation of the camera with respect to the travelway based at least in part on illumination variance in the imagery. For example, the computing system can determine (e.g., based at least in part on illumination variance in the imagery, and/or the like) a position of a light source (e.g., the sun, the moon, an artificial-light source, and/or the like) in the environment with respect to the camera; an orientation of the light source with respect to the travelway can be known, predetermined, determined by the computing system, and/or the like; and the computing system can select the geographic orientation of the camera with respect to the travelway based at least in part on the position of the light source with respect to the camera, the orientation of the light source with respect to the travelway, and/or the like. In some of such embodiments, the computing system can select the geographic orientation of the camera with respect to the travelway based at least in part on a time at which the camera generated the data representing the imagery. For example, such data can include a timestamp indicating a time at which the camera generated the data representing the imagery, and the computing system can determine the orientation of the light source with respect to the travelway based at least in part on the time at which the camera generated the data representing the imagery, and/or the like.

In some embodiments, the computing system can identify, in the imagery, at least a portion of a building, different travelway, and/or the like. In some of such embodiments, the computing system can select the geographic orientation of the camera with respect to the travelway based at least in part on the at least a portion of the building, different travelway, and/or the like. For example, the computing system can determine (e.g., based at least in part on the imagery, and/or the like) a position of the at least a portion of the building, different travelway, and/or the like with respect to the camera; an orientation of the at least a portion of the building, different travelway, and/or the like with respect to the travelway can be known, predetermined, determined by the computing system, and/or the like; and the computing system can select the geographic orientation of the camera with respect to the travelway based at least in part on the position of the at least a portion of the building, different travelway, and/or the like with respect to the camera, the orientation of the at least a portion of the building, different travelway, and/or the like with respect to the travelway, and/or the like.

In some embodiments, the computing system can recognize, in the imagery, text associated with the at least a portion of the building, different travelway, and/or the like. For example, signage on the building can include text indicating a street address of the building, a name of an organization associated with the building, and/or the like. Similarly, signage can include text indicating a name of the different travelway, and/or the like. In some of such embodiments, the computing system can identify the at least a portion of the building, different travelway, and/or the like based at least in part on the recognized text. For example, the computing system can store, access, and/or the like a database including information indicating street addresses of buildings, street addresses of organizations, organization names, travelway names, associations between one or more portions of the information, and/or the like; and the computing system can identify the at least a portion of the building, different travelway, and/or the like based at least in part on identifying one or more entries in the database that include at least a portion of the recognized text, and/or the like.

In some embodiments, the user device can include a wireless-network interface, one or more sensors in addition to the camera, for example, a magnetometer, a global positioning system (GPS) receiver, and/or the like. In some of such embodiments, the computing system can determine, based at least in part on data generated by the wireless-network interface, additional sensor(s), and/or the like, a geographic orientation of the user device with respect to the environment (e.g., with respect to the world, magnetic north, and/or the like); and the computing system can select the geographic orientation of the camera with respect to the travelway based at least in part on the determined geographic orientation of the user device with respect to the environment.

In some embodiments, the computing system can determine (e.g., based at least in part on data generated by the wireless-network interface, additional sensor(s), and/or the like) a geographic location of the camera, the user device, the user, and/or the like. In some of such embodiments, the computing system can identify the travelway based at least in part on the geographic location of the camera, the user device, the user, and/or the like. Additionally or alternatively, the computing system can determine a geographic orientation of the travelway with respect to the environment based at least in part on the geographic location of the camera, the user device, the user, and/or the like.

According to the invention, the computing system selects, based at least in part on the determined geographic location of the camera, and optionally the user device, the user, and/or the like, the machine-learning model from amongst multiple different machine-learning models for determining geographic orientations of cameras with respect to travelways. Such models are based at least in part on training data comprising imagery from different corresponding geographic regions, and the selected model is based at least in part on training data comprising imagery from a geographic region comprising the determined geographic location of the camera, and optionally the user device, the user, and/or the like. According to the invention, the imagery from such geographic region does not include imagery that comprises the travelway.

In some embodiments, the machine-learning model can be based at least in part on training data cropped from panoramic imagery generated by a camera mounted on a vehicle. Such a vehicle can include one or more sensors (e.g., magnetometers, GPS receivers, and/or the like) for determining a geographic orientation of the camera mounted on the vehicle with respect to a travelway upon a portion of which the vehicle is traveling while the camera mounted on the vehicle captures the panoramic imagery, a physical real-world environment comprising the vehicle and the travelway upon the portion of which the vehicle is traveling, and/or the like. For each image of the images, such training data can include a geographic orientation of the image with respect to a travelway upon a portion of which the vehicle was traveling when the camera mounted on the vehicle captured panoramic imagery from which the image was cropped. For example, such a geographic orientation of the image can be determined based at least in part on data generated by the sensor(s) of the vehicle when the camera mounted on the vehicle captured the panoramic imagery from which the image was cropped.

In some embodiments, the computing system can communicate, to an application, data based at least in part on the geographic orientation of the camera with respect to the travelway, the environment, and/or the like. Such an application can include, for example, a geographic-mapping application, a geographic-navigation application, an augmented reality (AR) application, and/or the like. For example, the computing system can receive a request for such data made by the application via an application programming interface (API), and/or the like, and the computing system can communicate (e.g., return, and/or the like) the data to the application via the API, and/or the like.

The operations, functions, and/or the like described herein can be performed by the user device, a computing system remotely located from the user device, a combination of the user device and the computing system remotely located from the user device, and/or the like. For example, in some embodiments, the user device can locally receive (e.g., from the camera, and/or the like) the data representing the imagery, can locally determine the geographic orientation(s), and/or the like. Additionally or alternatively, a computing system remotely located from the user device can receive (e.g., via one or more networks, and/or the like) the data representing the imagery from the user device, can determine the geographic orientation(s), and/or the like.

The methods and systems described herein can provide a number of technical effects and benefits. For example, the methods and systems described herein can enable a computing system to accurately and efficiently determine an orientation of a camera, user device, user, and/or the like with respect to their environment. In particular, because the camera, data representing the imagery, and methodologies described herein for determining an orientation with respect to a travelway are typically not subject to interference (e.g., magnetic interference, radio interference, and/or the like) from the user device, the environment, and/or the like, the methods and systems described herein can enable a computing system to determine an orientation of a camera, user device, user, and/or the like with respect to their environment more efficiently and accurately than conventional approaches, which are often susceptible to such interference, and/or the like.

<FIG> depicts an example computing environment according to example embodiments of the present disclosure. Referring to <FIG>, environment <NUM> can include user device <NUM>, one or more networks <NUM>, and computing system <NUM>. Network(s) <NUM> (e.g., one or more wired networks, wireless networks, and/or the like) can interface user device <NUM> and computing system <NUM>.

User device <NUM> can include one or more devices cable of performing one or more of the operations, functions, and/or the like described herein, for example, a camera system, laptop computing device, desktop computing device, mobile computing device, tablet computing device, smartphone, media device, wearable device, a combination of one or more of such devices, and/or the like. User device <NUM> can include one or more processors <NUM>, communication interfaces <NUM>, sensors <NUM>, and memory <NUM> (e.g., one or more hardware components for storing executable instructions, data, and/or the like). Communication interface(s) <NUM> can enable user device <NUM> to communicate (e.g., via network(s) <NUM>, and/or the like) with computing system <NUM>. For example, network(s) <NUM> can include one or more wireless networks, and communication interface(s) <NUM> can include one or more wireless-network interfaces configured to enable user device <NUM> to communicate (e.g., via the wireless network(s) of network(s) <NUM>, and/or the like) with computing system <NUM>. Sensor(s) <NUM> can include one or more devices configured to generate data based at least in part on a physical real-world environment in which user device <NUM> is located. For example, sensor(s) <NUM> can include one or more cameras <NUM>, global positioning system (GPS) receivers <NUM>, magnetometers <NUM>, and/or the like. Memory <NUM> can include (e.g., store, and/or the like) instructions <NUM>, which when executed by processor(s) <NUM>, can cause user device <NUM> to perform one or more operations, functions, and/or the like described herein.

Computing system <NUM> can be remotely located from user device <NUM> and can include one or more devices cable of performing one or more of the operations, functions, and/or the like described herein, for example, a desktop computing device, a server, a mainframe, a combination of one or more of such devices, and/or the like. Computing system <NUM> can include one or more processors <NUM>, communication interfaces <NUM>, and memory <NUM> (e.g., one or more hardware components for storing executable instructions, data, and/or the like). Communication interface(s) <NUM> can enable computing system <NUM> to communicate (e.g., via network(s) <NUM>, and/or the like) with user device <NUM>. Memory <NUM> can include (e.g., store, and/or the like) instructions <NUM>, which when executed by processor(s) <NUM>, can cause computing system <NUM> to perform one or more operations, functions, and/or the like described herein.

Irrespective of attribution described or implied herein, unless explicitly indicated otherwise, the operations, functions, and/or the like described herein can be performed by user device <NUM> and/or computing system <NUM> (e.g., by user device <NUM>, by computing system <NUM>, by a combination of user device <NUM> and computing system <NUM>, and/or the like).

<FIG> depicts an example event sequence according to example embodiments of the present disclosure. Referring to <FIG>, at (<NUM>), user device <NUM> can determine its geographic location within a physical real-world environment, its geographic orientation with respect to such environment, and/or the like.

In some embodiments, user device <NUM> can determine its geographic location, orientation, and/or the like based at least in part on data generated by communication interface(s) <NUM>, sensor(s) <NUM>, and/or the like. For example, user device <NUM> can determine its geographic location, orientation, and/or the like based at least in part on data generated by a wireless-network interface of communication interface(s) <NUM> (e.g., indicating a position, orientation, and/or the like of user device <NUM> with respect to one or more radio-signal sources for which locations, orientations, and/or the like are known), GPS receiver(s) <NUM> (e.g., indicating a position, orientation, and/or the like of user device <NUM> with respect to one or more satellite-signal sources for which locations, orientations, and/or the like are known), and/or magnetometer(s) <NUM> (e.g., indicating a position, orientation, and/or the like of user device <NUM> with respect to one or more magnetic-field sources for which locations, orientations, and/or the like are known). It will be appreciated that such determinations are estimates, not absolutes. It will further be appreciated that with respect to geographic orientation, such estimations can be inaccurate, imprecise, erroneous, and/or the like.

<FIG> depicts an example map of an example geographic area according to example embodiments of the present disclosure. Referring to <FIG>, map <NUM> can depict a geographic region that includes location <NUM>, where user device <NUM> can be located. The region can include travelways <NUM> and <NUM>, as well as buildings <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM>. Buildings <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> can be associated with one or more organizations (e.g., occupants, owners, tenants, and/or the like). For example, building <NUM> can include portion <NUM> associated with an organization (e.g., THIP KHAO, and/or the like) and portion <NUM> associated with a different organization (e.g., allegro, and/or the like). The region depicted by map <NUM> can be part of a physical real-world environment (e.g., the world, and/or the like) in which user device <NUM> is located.

<FIG> depicts an example scene of a portion of an example physical real-world environment according to example embodiments of the present disclosure. Referring to <FIG>, scene <NUM> can depict a portion of the physical real-world environment in which user device <NUM> is located (e.g., at location <NUM>, and/or the like). For example, scene <NUM> can include portions of travelways <NUM> and <NUM> and portions <NUM> and <NUM> of building <NUM>. Building <NUM> can include text <NUM> (e.g., a street address of building <NUM>, and/or the like), portion <NUM> can include text <NUM> (e.g., a name of the organization associated with portion <NUM>, and/or the like), and portion <NUM> can include text <NUM> (e.g., a name of the organization associated with portion <NUM>, and/or the like). Scene <NUM> can also include objects <NUM> (e.g., a tree, and/or the like) and <NUM> (e.g., an artificial-light source, signage with text (not illustrated), such as a name of travelway <NUM>, and/or the like).

Returning to <FIG>, at (<NUM>), user device <NUM> can generate data representing imagery of at least a portion of the environment in which it is located. For example, user device <NUM> can include (e.g., store, execute, and/or the like) one or more applications configured to utilize data based at least in part on a geographic orientation of user device <NUM> (e.g., a geographic-mapping application, a geographic-navigation application, an augmented reality (AR) application, and/or the like); such application(s) can prompt a user of user device <NUM> to capture imagery from location <NUM>; the user can utilize one or more of camera(s) <NUM> to capture such imagery; and camera(s) <NUM> can generate data representing imagery of a portion of the environment depicted by scene <NUM>.

<FIG> depicts an example image of a portion of an example physical real-world environment according to example embodiments of the present disclosure. Referring to <FIG>, the data generated by camera(s) <NUM> can include data representing image <NUM>, which, as illustrated, can include at least a portion of travelway <NUM>, object <NUM>, and building <NUM>, including text <NUM> and <NUM>.

Returning to <FIG>, at (<NUM>), user device <NUM> can receive a request regarding its geographic orientation (e.g., for data based at least in part on its geographic orientation, and/or the like). For example, in some embodiments, the application(s) included on user device <NUM> can make such a request via an application programming interface (API), and/or the like.

At (<NUM>), user device <NUM> can communicate (e.g., via network(s) <NUM>, as indicated by the cross-hatched box over the line extending downward from network(s) <NUM>, and/or the like) data to computing system <NUM>, which can receive the data. For example, such data can include data indicating the geographic location (e.g., location <NUM>, and/or the like), orientation, and/or the like of user device <NUM> determined at (<NUM>), the data representing image <NUM>, and/or the like.

At (<NUM>), user device <NUM> and/or computing system <NUM> can determine, based at least in part on the data representing image <NUM>, one or more possible geographic orientations of camera(s) <NUM> (e.g., of camera(s) <NUM>, user device <NUM>, the user, and/or the like) with respect to travelway <NUM> (e.g., at location <NUM>, and/or the like). For example, at (210A), user device <NUM> can determine the possible geographic orientation(s) (e.g., based at least in part on data indicating the geographic location (e.g., location <NUM>, and/or the like), orientation, and/or the like of user device <NUM> determined at (<NUM>), the data representing image <NUM>, and/or the like). Additionally or alternatively, at (210B), computing system <NUM> can determine the possible geographic orientation(s) (e.g., based at least in part on the data communicated at (<NUM>), and/or the like).

The possible geographic orientation(s) can be determined based at least in part on a machine-learning model (e.g., a neural network, and/or the like). For example, such a model can be configured (e.g., trained, optimized, and/or the like) to determine geographic orientations of cameras with respect to travelways, and user device <NUM> and/or computing system <NUM> can utilize the model to determine the possible geographic orientation(s) based at least in part on the data representing image <NUM>, and/or the like (e.g., one or more positions, orientations, and/or the like of the portions of travelway <NUM>, building <NUM>, object <NUM>, and/or the like within image <NUM>, and/or the like).

According to the invention, user device <NUM> and/or computing system <NUM> selects, based at least in part on the geographic location determined at (<NUM>) (e.g., location <NUM>, and/or the like), the machine-learning model from amongst multiple different machine-learning models for determining geographic orientations of cameras with respect to travelways. Such models are based at least in part on training data comprising imagery from different corresponding geographic regions, and the selected model is based at least in part on training data comprising imagery from the region depicted by map <NUM>, and/or the like. According to the invention, the imagery from the region depicted by map <NUM>, and/or the like does not include imagery that comprises travelway <NUM>.

Additionally or alternatively, the machine-learning model can be based at least in part on training data that includes imagery generated by one or more cameras carried by one or more users traveling (e.g., walking, and/or the like) along (e.g., in the middle of, alongside, and/or the like) one or more travelways (e.g., within the region depicted by map <NUM>, and/or the like).

In some embodiments, prior to utilizing the machine-learning model to determine the possible geographic orientation(s), user device <NUM> and/or computing system <NUM> can crop, compress, resize, reorient, and/or the like image <NUM> in accordance with imagery included in the training data. In some embodiments, the machine-learning model can be configured to generate a probability map of the possible orientation(s), and/or the like. In some embodiments, the machine-learning model can be configured to determine the possible orientation(s) based at least in part on one or more geometries (e.g., expressed as one or more line equations, and/or the like) of one or more portions of one or more travelways included in the imagery, and/or the like. In some embodiments, each of the possible orientation(s) can be expressed as a four-dimensional quaternion, and/or the like. In some of such embodiments, the machine-learning model can determine, for each possible orientation, a four-dimensional log variance of the quaternion, and/or the like.

In some embodiments, the machine-learning model can comprise a convolutional neural network as the basis for a regression network, and/or the like. In some embodiments, the machine-learning model can comprise one or more fully connected layers, final regression layers, and/or the like (e.g., on top of the basis network, and/or the like).

In some embodiments, the machine-learning model can comprise an L2 loss function (e.g., on normalized quaternions, and/or the like). For example, the machine-learning model can comprise the following function: <MAT>
wherein: q̂ can correspond to a ground truth unit quaternion (e.g., a four-dimensional vector, and/or the like); and q can correspond to a predicted quaternion, for example, not normalized, (e.g., a four-dimensional vector, and/or the like).

In some embodiments, the machine-learning model can comprise a confidence loss, for example, a Gaussian log likelihood that incorporates variance of predictions. For example, the machine-learning model can comprise the following function: <MAT>
wherein, logv can correspond to a predicted log variance of q (e.g., a four-dimensional vector, and/or the like).

Such loss can assume the ground truth lies on a normal distribution given the observed input and can attempt to predict the mean <MAT>.

and log variance logv of such distribution, and/or the like. The normal distribution can be assumed to be axis-aligned isocontours (e.g., a diagonal covariance matrix with different diagonal elements, and/or the like).

According to the invention, the possible geographic orientation(s) includes two orientations differing by one hundred and eighty degrees. For example, user device <NUM> and/or computing system <NUM> can determine camera(s) <NUM> are facing directly up or down travelway <NUM> (e.g., oriented at a zero-degree angle or one-hundred-eighty-degree angle with respect to travelway <NUM>), perpendicular to travelway <NUM> one way or the other (e.g., oriented at a ninety-degree angle or two-hundred-seventy-degree angle with respect to travelway <NUM>), askew to travelway <NUM> one way or the other (e.g., oriented at a forty-five-degree angle or two-hundred-twenty-five-degree angle with respect to travelway <NUM>), and/or the like.

<FIG> depicts example orientations according to example embodiments of the present disclosure. Referring to <FIG>, space <NUM> can represent a plane within the physical real-world environment in which user device <NUM> is located (e.g., at location <NUM>, and/or the like). Space <NUM> can include reference orientation <NUM> of the environment (e.g., magnetic north, and/or the like). At location <NUM>, travelway <NUM> can be at orientation <NUM> (e.g., offset by angle a from orientation <NUM>, and/or the like). The determined possible orientation(s) can include orientations <NUM> (e.g., offset by angle b from orientation <NUM>, angle c from orientation <NUM>, and/or the like) and <NUM> (e.g., offset by <NUM>° from orientation <NUM>, angle b plus <NUM>°from orientation <NUM>, and angle c plus <NUM>°from orientation <NUM>, and/or the like). In determining the possible orientations, user device <NUM> and/or computing system <NUM> can determine (e.g., based at least in part on one or more positions, orientations, and/or the like of the portions of travelway <NUM>, building <NUM>, object <NUM>, and/or the like within image <NUM>, and/or the like) orientations <NUM> and/or <NUM> based at least in part on an offset (e.g., angle b, angle b plus <NUM>°, and/or the like) of travelway <NUM> within image <NUM>, for example, with respect to camera(s) <NUM> (e.g., a center line of image <NUM>, and/or the like).

Returning to <FIG>, at (<NUM>), user device <NUM> and/or computing system <NUM> can select an orientation of camera(s) <NUM> with respect to travelway <NUM> from amongst the possible orientation(s). For example, at (212A), user device <NUM> can select (e.g., based at least in part on data indicating the geographic location (e.g., location <NUM>, and/or the like), orientation, and/or the like of user device <NUM> determined at (<NUM>), the data representing image <NUM>, and/or the like) orientation <NUM> from amongst orientations <NUM> and <NUM>. Additionally or alternatively, at (212B), computing system <NUM> can select (e.g., based at least in part on the data communicated at (<NUM>), and/or the like) orientation <NUM> from amongst orientations <NUM> and <NUM>.

In some embodiments, the orientation can be selected from amongst the possible orientation(s) based at least in part on the orientation of user device <NUM> determined at (<NUM>). As indicated above, it will be appreciated that the orientation of user device <NUM> determined at (<NUM>) can be inaccurate, imprecise, erroneous, and/or the like (e.g., unreliable for determining an accurate orientation of user device <NUM>, and/or the like). It will further be appreciated, however, that despite its shortcomings, the orientation of user device <NUM> determined at (<NUM>) can be useful in accurately selecting an orientation of camera(s) <NUM> with respect to travelway <NUM> from amongst the possible orientation(s) (e.g., for selecting from amongst orientations <NUM> and <NUM>, and/or the like).

In some embodiments, user device <NUM> and/or computing system <NUM> can select the orientation based at least in part on illumination variance in image <NUM>. For example, user device <NUM> and/or computing system <NUM> can determine (e.g., based at least in part on illumination variance in image <NUM>, and/or the like) a position of a light source, for example, an artificial-light source (e.g., object <NUM>, and/or the like), the sun, the moon, and/or the like in the environment with respect to camera(s) <NUM>; an orientation of the light source with respect to travelway <NUM> can be known, predetermined, determined by user device <NUM> and/or computing system <NUM>, and/or the like; and user device <NUM> and/or computing system <NUM> can select the orientation based at least in part on the position of the light source with respect to camera(s) <NUM>, the orientation of the light source with respect to travelway <NUM>, and/or the like. In some of such embodiments, user device <NUM> and/or computing system <NUM> can select the orientation based at least in part on a time at which camera(s) <NUM> generated the data representing image <NUM>. For example, such data can include a timestamp indicating a time at which camera(s) <NUM> generated the data representing image <NUM>, and user device <NUM> and/or computing system <NUM> can determine the orientation of the light source with respect to travelway <NUM> based at least in part on the time at which camera(s) <NUM> generated the data representing image <NUM>, and/or the like.

In some embodiments, user device <NUM> and/or computing system <NUM> can identify, in image <NUM>, at least a portion of a building, different travelway, and/or the like. In some of such embodiments, user device <NUM> and/or computing system <NUM> can select the orientation based at least in part on the at least a portion of the building, different travelway, and/or the like. For example, user device <NUM> and/or computing system <NUM> can determine (e.g., based at least in part on image <NUM>, and/or the like) a position of the portion of building <NUM> in image <NUM>, and/or the like with respect to camera(s) <NUM>; an orientation of the portion of building <NUM>,and/or the like with respect to travelway <NUM> can be known, predetermined, determined by user device <NUM> and/or computing system <NUM>, and/or the like; and user device <NUM> and/or computing system <NUM> can select the orientation based at least in part on the position of the portion of building <NUM>, and/or the like with respect to camera(s) <NUM>, the orientation of the portion of building <NUM>, and/or the like with respect to travelway <NUM>, and/or the like. It will be appreciated, for example, that had image <NUM> included one or more portions of buildings <NUM>, <NUM>, <NUM>, <NUM>, and/or <NUM>, travelway <NUM>, and/or the like, user device <NUM> and/or computing system <NUM> could select a different orientation.

In some embodiments, user device <NUM> and/or computing system <NUM> can recognize, in image <NUM>, text associated with the at least a portion of the building, different travelway, and/or the like. For example, signage on building <NUM> can include text <NUM> indicating the street address of building <NUM>, text <NUM> indicting the name of the organization associated portion <NUM>, text <NUM> indicting the name of the organization associated portion <NUM>, and/or the like. Similarly, object <NUM> can include signage with text (not illustrated) indicating the name of travelway <NUM>, and/or the like. In some of such embodiments, user device <NUM> and/or computing system <NUM> can identify the at least a portion of the building, different travelway, and/or the like based at least in part on the recognized text. For example, user device <NUM> and/or computing system <NUM> can store, access, and/or the like a database including information indicating street addresses of buildings (e.g., the street address of building <NUM>, and/or the like), street addresses of organizations (e.g., street addresses of the organizations associated with portions <NUM> and <NUM>, and/or the like), organization names (e.g., the names of the organizations associated with portions <NUM> and <NUM>, and/or the like), travelway names (e.g., the name of travelway <NUM>, and/or the like), and/or associations between one or more portions of the information (e.g., associations between the street addresses of the organizations associated with portions <NUM> and <NUM> and the names of the organizations associated with portions <NUM> and <NUM>, and/or the like); and user device <NUM> and/or computing system <NUM> can identify portions <NUM> and/or <NUM>, travelway <NUM>, and/or the like based at least in part on identifying one or more entries in the database that include at least a portion of the recognized text (e.g., text <NUM> and <NUM>, the text included on the signage of object <NUM>, and/or the like).

At (<NUM>), user device <NUM> and/or computing system <NUM> can determine (e.g., based at least in part on the selected orientation, and/or the like) a geographic orientation of camera(s) <NUM> (e.g., of camera(s) <NUM>, user device <NUM>, the user, and/or the like) with respect to the physical real-world environment (e.g., with respect to orientation <NUM>, and/or the like). For example, at (214A), user device <NUM> can determine (e.g., based at least in part on the orientation of camera(s) <NUM> (e.g., orientation <NUM>, and/or the like) with respect to travelway <NUM> (e.g., with respect to orientation <NUM>, and/or the like), data indicating the geographic location (e.g., location <NUM>, and/or the like), orientation, and/or the like of user device <NUM> determined at (<NUM>), the data representing image <NUM>, and/or the like) a geographic orientation of camera(s) <NUM> (e.g., orientation <NUM>, and/or the like) with respect to the environment (e.g., with respect to orientation <NUM>, and/or the like). Additionally or alternatively, at (214B), computing system <NUM> can determine (e.g., based at least in part on the orientation of camera(s) <NUM> (e.g., orientation <NUM>, and/or the like) with respect to travelway <NUM> (e.g., with respect to orientation <NUM>, and/or the like), the data communicated at (<NUM>), and/or the like) a geographic orientation of camera(s) <NUM> (e.g., orientation <NUM>, and/or the like) with respect to the environment (e.g., with respect to orientation <NUM>, and/or the like).

In some embodiments determining the geographic orientation of camera(s) <NUM> with respect to the environment can include determining a geographic orientation of travelway <NUM> (e.g., orientation <NUM>, and/or the like) with respect to the environment (e.g., with respect to orientation <NUM>, and/or the like). For example, user device <NUM> and/or computing system <NUM> can identify (e.g., based at least in part on map <NUM>, location <NUM>, and/or the like) travelway <NUM>; an orientation of travelway <NUM> (e.g., orientation <NUM>, and/or the like) with respect to the environment (e.g., with respect to orientation <NUM>, and/or the like) can be (e.g., based at least in part on map <NUM>, location <NUM>, and/or the like) known, predetermined, determined by user device <NUM> and/or computing system <NUM>, and/or the like; and user device <NUM> and/or computing system <NUM> can determine an orientation of camera(s) <NUM> (e.g., orientation <NUM>, and/or the like) with respect to the environment (e.g., with respect to orientation <NUM>, and/or the like) based at least in part on the orientation of travelway <NUM> (e.g., orientation <NUM>, and/or the like) with respect to the environment (e.g., with respect to orientation <NUM>, and/or the like) and the orientation of camera(s) <NUM> (e.g., orientation <NUM>, and/or the like) with respect to travelway <NUM> (e.g., with respect to orientation <NUM>, and/or the like), for example, by combining an offset (e.g., angle a, and/or the like) of the orientation of travelway <NUM> (e.g., orientation <NUM>, and/or the like) with respect to the environment (e.g., with respect to orientation <NUM>, and/or the like) with an offset (e.g., angle b, and/or the like) of the orientation of camera(s) <NUM> (e.g., orientation <NUM>, and/or the like) with respect to travelway <NUM> (e.g., with respect to orientation <NUM>, and/or the like) to determine an offset (e.g., angle c, and/or the like) of the orientation of camera(s) <NUM> (e.g., orientation <NUM>, and/or the like) with respect to the environment (e.g., with respect to orientation <NUM>, and/or the like).

At (<NUM>), computing system <NUM> can communicate data to user device <NUM>, which can receive the data. For example, such data can indicate the orientation of camera(s) <NUM> (e.g., orientation <NUM>, and/or the like) with respect to travelway <NUM> (e.g., with respect to orientation <NUM>, and/or the like) and/or the orientation of camera(s) <NUM> with respect to the environment (e.g., with respect to orientation <NUM>, and/or the like).

At (<NUM>), user device <NUM> can communicate (e.g., return, and/or the like) data based at least in part on the orientation of camera(s) <NUM> (e.g., orientation <NUM>, and/or the like) with respect to travelway <NUM> (e.g., with respect to orientation <NUM>, and/or the like) and/or the orientation of camera(s) <NUM> with respect to the environment (e.g., with respect to orientation <NUM>, and/or the like).

<FIG> depicts an example method according to example embodiments of the present disclosure. Referring to <FIG>, at (<NUM>), a computing system can receive data generated by a camera and representing imagery that includes at least a portion of a physical real-world environment comprising the camera and a travelway. For example, user device <NUM> and/or computing system <NUM> can receive data generated by camera(s) <NUM> and representing image <NUM>.

At (<NUM>), the computing system can determine, based at least in part on the data and a machine-learning model, a geographic orientation of the camera with respect to the travelway. For example, user device <NUM> and/or computing system <NUM> can determine, based at least in part on the data representing image <NUM> and a machine-learning model, a geographic orientation of camera(s) <NUM> (e.g., orientation <NUM>, and/or the like) with respect to travelway <NUM> (e.g., with respect to orientation <NUM>, and/or the like).

At (<NUM>), the computing system can determine a geographic orientation of the travelway with respect to a physical real-world environment including the camera and the travelway. For example, user device <NUM> and/or computing system <NUM> can determine a geographic orientation of travelway <NUM> (e.g., orientation <NUM>, and/or the like) with respect to the physical real-world environment including the portion depicted by scene <NUM> (e.g., with respect to orientation <NUM>, and/or the like).

At (<NUM>), the computing system can determine, based at least in part on the geographic orientation of the camera with respect to the travelway and the geographic orientation of the travelway with respect to the physical real-world environment, a geographic orientation of the camera with respect to the physical real-world environment. For example, user device <NUM> and/or computing system <NUM> can determine, based at least in part on the geographic orientation of camera(s) <NUM> (e.g., orientation <NUM>, and/or the like) with respect to travelway <NUM> (e.g., orientation <NUM>, and/or the like) and the geographic orientation of travelway <NUM> (e.g., orientation <NUM>, and/or the like) with respect to the physical real-world environment including the portion depicted by scene <NUM> (e.g., with respect to orientation <NUM>, and/or the like), a geographic orientation of camera(s) <NUM> (e.g., orientation <NUM>, and/or the like) with respect to the physical real-world environment including the portion depicted by scene <NUM> (e.g., with respect to orientation <NUM>, and/or the like).

The technology discussed herein makes reference to servers, databases, software applications, and/or other computer-based systems, as well as actions taken and information sent to and/or from such systems. The inherent flexibility of computer-based systems allows for a great variety of possible configurations, combinations, and/or divisions of tasks and/or functionality between and/or among components. For instance, processes discussed herein can be implemented using a single device or component and/or multiple devices or components working in combination. Databases and/or applications can be implemented on a single system and/or distributed across multiple systems. Distributed components can operate sequentially and/or in parallel.

Various connections between elements are discussed in the above description. These connections are general and, unless specified otherwise, can be direct and/or indirect, wired and/or wireless. In this respect, the specification is not intended to be limiting.

The depicted and/or described steps are merely illustrative and can be omitted, combined, and/or performed in an order other than that depicted and/or described; the numbering of depicted steps is merely for ease of reference and does not imply any particular ordering is necessary or preferred.

The functions and/or steps described herein can be embodied in computer-usable data and/or computer-executable instructions, executed by one or more computers and/or other devices to perform one or more functions described herein. Generally, such data and/or instructions include routines, programs, objects, components, data structures, or the like that perform particular tasks and/or implement particular data types when executed by one or more processors in a computer and/or other data-processing device. The computer-executable instructions can be stored on a computer-readable medium such as a hard disk, optical disk, removable storage media, solid-state memory, read-only memory (RAM), or the like. As will be appreciated, the functionality of such instructions can be combined and/or distributed as desired. In addition, the functionality can be embodied in whole or in part in firmware and/or hardware equivalents, such as integrated circuits, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or the like. Particular data structures can be used to more effectively implement one or more aspects of the disclosure, and such data structures are contemplated to be within the scope of computer-executable instructions and/or computer-usable data described herein.

Although not required, one of ordinary skill in the art will appreciate that various aspects described herein can be embodied as a method, system, apparatus, and/or one or more computer-readable media storing computer-executable instructions. Accordingly, aspects can take the form of an entirely hardware embodiment, an entirely software embodiment, an entirely firmware embodiment, and/or an embodiment combining software, hardware, and/or firmware aspects in any combination.

As described herein, the various methods and acts can be operative across one or more computing devices and/or networks. The functionality can be distributed in any manner or can be located in a single computing device (e.g., server, client computer, user device, or the like).

Aspects of the disclosure have been described in terms of illustrative embodiments thereof. Numerous other embodiments, modifications, and/or variations within the scope of the appended claims can occur to persons of ordinary skill in the art from a review of this disclosure.

Claim 1:
A computer-implemented method comprising:
receiving, by a computing system (<NUM>), data generated by a camera (<NUM>) and representing imagery that includes at least a portion of a physical real-world environment comprising the camera (<NUM>) and a travelway (<NUM>); and
determining, by the computing system (<NUM>) and based at least in part on the data and a machine-learning model, a geographic orientation of the camera (<NUM>) with respect to the travelway (<NUM>);
wherein determining the geographic orientation of the camera (<NUM>) with respect to the travelway (<NUM>) comprises:
determining a geographic location of the camera (<NUM>);
selecting, based at least in part on the geographic location of the camera (<NUM>), the machine-learning model from amongst a plurality of different machine-learning models for determining geographic orientations of cameras with respect to travelways, the plurality of different machine-learning models being based at least in part on training data comprising imagery from different corresponding geographic regions, the machine-learning model being based at least in part on training data comprising imagery from a geographic region comprising the geographic location of the camera (<NUM>), the imagery from the geographic region not including imagery that comprises the travelway (<NUM>);
determining, based at least in part on the machine-learning model, two possible geographic orientations of the camera (<NUM>) with respect to the travelway (<NUM>), the two possible geographic orientations differing by one hundred and eighty degrees; and
selecting, from amongst the two possible geographic orientations, the geographic orientation of the camera (<NUM>) with respect to the travelway (<NUM>).