Patent Description:
An AV can be included in a fleet of AVs that are at least partially coordinated by a centralized server to provide AV transportation to users in an operational region. By way of example, the AV can be included in a rideshare transportation system whereby a user places an order for transportation from an origin to a destination by way of an Internet-connected mobile computing device, and a server in communication with the mobile computing device and the AV dispatches the AV to convey the user from the origin to the destination.

Conventionally, rideshare services provide conveyance all the way from a pickup location at which a user boards a vehicle to a destination of the user. For many trips, however, a single modality of transport may not be the fastest or least expensive way for the user to get from the user's origin to her destination. By way of example, during times of high traffic volume the fastest way for a user to get from his place of employment to his home may be to take a train for at least part of his journey. In a conventional transportation system, however, a user would have to estimate or guess whether using multiple modalities of transportation would be faster or cheaper in connection with reaching his destination. Furthermore, the user would have to manually order or otherwise make arrangements for each of the modalities individually which can potentially eliminate any time or cost savings associated with taking multiple-modality transport.

<CIT>discloses a multi-mode transportation system in which a scheduler service (SS) generates an initial transportation schedule (TS) including a list of transport legs where transport over one or more transport legs is via an autonomous vehicle (AV) from an initial scheduled pick-up location (ISPUL) at an initial scheduled pick-up time (ISPUT). The SS monitors for any changes to the initial TS, and if any are detected, updates the initial TS based on the changes to generate a current TS that includes a currently scheduled pick-up time (CSPUT) and a currently scheduled pick-up location (CSPUL) for the transport leg. The CSPUT/CSPUL can be the same as or different than the ISPUT/ISPUL depending on whether or not there were changes to the ISPUT/ISPUL. An AV can be selected from the fleet of AVs and automatically controlled to arrive at the CSPUL to pick-up the passenger by the CSPUT.

The following is a brief summary of subject matter that is described in greater detail herein. This summary is not intended to be limiting as to the scope of the claims.

Various technologies that facilitate providing multiple-modality transportation to a user are described herein. With more particularity, technologies disclosed herein facilitate identification of an optimal trip route from an origin of a user to a desired destination of the user, wherein the optimal trip route includes multiple transportation modalities, at least one of which is an AV, and dispatching of the AV to a pickup location for the user in connection with providing transportation to the user along a portion of the identified trip route.

In an exemplary embodiment, a multiple-modality transportation system includes a client computing device, a server computing device, and an AV, wherein the server computing device is in communication with the client computing device and the AV. A user of the client computing device can interact with the client computing device to order transportation from a present location of the user (or other trip origin desired by the user) to a trip destination.

In example operations of the multiple-modality transportation system, the user provides input to the client computing device that indicates a desired trip destination of the user. Responsive to receipt of the user input, the client computing device outputs a trip request that is indicative of the desired trip destination to the server computing device. The server computing device generates a route plan for the trip request by performing an optimization over routing information pertaining to a region that includes the trip origin and the trip destination. For example, the server computing device can execute an optimization algorithm that identifies a fastest route from the trip origin to the trip destination taking into consideration multiple transportation modalities. Stated differently, the server computing device executes an optimization algorithm that determines the fastest way to travel from the trip origin to the trip destination using different transportation modalities to traverse different partial segments of the total trip route. The different transportation modalities considered by the optimization algorithm include transport by way of an AV. The different transportation modalities considered by the optimization algorithm can further include public transportation such as trains, buses, streetcars, etc. In other examples, the different transportation modalities can include walking, cycling, travel by way of scooter (e.g., using a scooter-share service), or substantially any other mode of transportation.

The server computing device transmits data indicative of the route plan to the client computing device, whereupon the client computing device displays graphical indicia of the route plan to the user. The user can provide further input to the client computing device indicating that the user desires to order transportation according to the route plan indicated in the data received from the server computing device. Responsive to receiving such user input the client computing device outputs a trip confirmation to the server computing device. The server computing device can then issue a pickup instruction to an AV in communication with the server computing device to cause the AV to navigate to a pickup location included in the route plan. Upon rendezvous with the user at the pickup location, the AV can convey the user from the pickup location to a drop off location in accordance with the route plan.

In some exemplary embodiments, the multiple-modality transportation system can include or be in communication with a transportation information server to facilitate provision of single-interaction ordering of multiple-modality transportation by way of the client computing device. By way of example, the server computing device can be in communication with a transportation information server that accepts orders of transportation fares for one or more public transportation modalities. Responsive to receipt of a trip confirmation from the client computing device, the server computing device can communicate with the transportation information server to process payment for a public transportation fare. Responsive to receipt of a proof of payment for the public transportation fare from the transportation information server, such as an electronic fare card, the server computing device can transmit the proof of payment to the client computing device. Subsequently, the user of the client computing device can use the proof of payment to gain admittance to public transportation (e.g., by displaying a graphical indication of the proof of payment on the client computing device to a bus operator). The multiple-modality transportation system can therefore enable a user to order and pay for access to multiple transportation modalities for conveyance along a trip route using a single user input.

In further embodiments of a multiple-modality transportation system, the system can be configured to dynamically update a route plan in real time to account for unforeseen changes in conditions affecting conveyance of a user from her trip origin to her trip destination. The client computing device can be configured to transmit updates pertaining to a location of the user to the server computing device in real time. The server computing device can then issue an updated pickup instruction to an AV based upon the updated location of the user to cause the AV to navigate to a pickup location other than a location indicated in the original route plan. In an illustrative example, the client computing device can be configured to transmit an update to a location of the user that indicates that the user has disembarked from a train at an earlier stop than a stop indicated in an original route plan generated by the server computing device. Responsive to receiving the update of the location of the user, the server computing device can issue a pickup instruction to an AV to cause the AV to navigate to the earlier stop in order to pick up the user. Thus, rather than requiring the user to board the train again and disembark at the stop indicated in the original route plan, the multiple-modality transportation system can coordinate AVs to adapt to movements of the user in connection with conveying the user to her destination.

The above summary presents a simplified summary in order to provide a basic understanding of some aspects of the systems and/or methods discussed herein. This summary is not an extensive overview of the systems and/or methods discussed herein. It is not intended to identify key/critical elements or to delineate the scope of such systems and/or methods. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is presented later.

Various technologies pertaining to facilitating multiple-modality transportation are described herein. With more specificity, disclosed herein are technologies for identifying optimal multiple-modality trip routes and coordinating operation of autonomous vehicles (AVs) to facilitate providing multiple-modality transportation to users. Technologies set forth herein are now described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In other instances, well-known structures and devices are shown in block diagram form in order to facilitate describing one or more aspects. Further, it is to be understood that functionality that is described as being carried out by certain system components may be performed by multiple components. Similarly, for instance, a component may be configured to perform functionality that is described as being carried out by multiple components.

As used herein, the terms "client", "server", "component" and "system" are intended to encompass computer-readable data storage that is configured with computer-executable instructions that cause certain functionality to be performed when executed by a processor. The computer-executable instructions may include a routine, a function, or the like. It is also to be understood that a component or system may be localized on a single device or distributed across several devices. Further, as used herein, the term "exemplary" is intended to mean "serving as an illustration or example of something.

With reference now to the drawings, <FIG> illustrates an exemplary multiple-modality transportation system <NUM> that facilitates presenting a multiple-modality transportation route from a point of origin to a destination to a user, and coordinating operation of one or more AVs to facilitate transportation of the user along the multiple-modality transportation route. Further, the multiple-modality transportation system <NUM> coordinates operation of one or more AVs in real-time responsive to updated locations of a user that are indicative of a potential change in the route plan of the user.

The exemplary multiple-modality transportation system <NUM> includes a client computing device <NUM>, a server computing device <NUM>, and at least one AV <NUM>. The server computing device <NUM> is in communication with each of the client computing device <NUM> and the AV <NUM> by way of a network <NUM>. The network <NUM> can include one or more intermediate network connections that facilitate communication among the elements of the system <NUM> connected to the network <NUM>, and the network <NUM> can further be part of a larger network that includes the network <NUM>. By way of example, and not limitation, the network <NUM> can include connections that facilitate communication between the server computing device <NUM> and the client computing device <NUM> and/or between the server computing device <NUM> the AV <NUM> by way of the Internet. In various embodiments the multiple-modality transportation system <NUM> can further include a transportation information server <NUM> that is in communication with at least the server computing device <NUM> by way of the network <NUM>.

The multiple-modality transportation system <NUM> is configured to enable a user of the client computing device <NUM> to plan and order transportation along a multiple-modality transportation route within a single user interface (e.g., within a single application executing on the client computing device <NUM>). In exemplary embodiments, a user of the client computing device <NUM> can view available multiple-modality transportation routes to a desired destination, and can order transportation along one of the multiple-modality transportation routes such that one or more AVs (e.g., the AV <NUM>) is caused to convey the user along at least a portion of a multiple-modality transportation route.

Conventional rideshare services and other on-demand transportation services (such as bikeshare, scootershare, and the like) are generally limited to a single modality. By way of example, rideshare services that facilitate transportation of passengers by automobiles operated by human drivers conventionally only allow users to arrange for transport by a single automobile from a trip origin to a desired destination, albeit potentially making multiple stops to pick up additional passengers. Similarly, bikeshare and scootershare services generally only allow a user to rent a single bike or scooter for a period of time. To reach a destination using multiple transportation modalities using conventional transportation services, a user would have to manually determine whether using multiple modalities of transportation would be faster or cheaper in connection with reaching his destination. Furthermore, the user would have to manually order or otherwise make arrangements for each of the modalities individually. For example, conventionally a user could hire a first taxi to take him to the train station, pay for a train ticket to take the train some distance, and then hire a second taxi to take him the remaining distance to his origin.

In contrast with such conventional transportation systems, the multiple-modality transportation system <NUM> identifies a multiple-modality transportation route from an origin to a desired destination for a user, and can determine whether the multiple-modality route is faster, cheaper, or otherwise preferable to a single-modality route. Further, the multiple-modality transportation system <NUM> facilitates use of the multiple-modality transportation route by the user by coordinating one or more AVs to convey the user along portions of the route without requiring the user to place multiple transportation orders. In a non-limiting example, the multiple-modality transportation system <NUM> can cause the AV <NUM> to navigate to a terminus of a first portion of the route corresponding to a first modality, such that the AV <NUM> is waiting for the user when the user reaches the terminus. The AV <NUM> can then convey the user along a second portion of the route, the AV <NUM> being a second modality of transportation. A user can therefore order multiple-modality transportation wherein AVs and other transportation modalities are used in a combined manner to convey a user to her destination faster or cheaper than would be possible using single-mode transportation. Further, the system <NUM> can be configured such that if the user deviates from the original multiple-modality transportation route to arrive at an updated location (e.g., a location other than the terminus of the first portion of the route), the system <NUM> can cause an AV to meet the user at the updated location without requiring additional input from the user at the client computing device <NUM>. The AV can then convey the user from the updated location at least partway to the destination of the user. For example, the AV can convey the user from the updated location to the destination of the user, or to an origin point for another modality of transportation, such as a train station. Conventional transportation systems generally require a user to cancel an existing transportation order and place a new transportation order if the user deviates from an existing transportation plan.

Various details of the multiple-modality transportation system <NUM> are now set forth. The client computing device <NUM> includes a processor <NUM> and memory <NUM> that is operably coupled to the processor <NUM> and that stores computer-executable instructions that are executed by the processor <NUM>. The client computing device <NUM> further comprises a display <NUM> on which can be displayed a graphical user interface (GUI) <NUM>. In exemplary embodiments, the client computing device <NUM> can be a mobile computing device such as a smartphone, a tablet computing device, a laptop personal computer, or the like. In other exemplary embodiments, the client computing device <NUM> can be a desktop computing device, a computing device included in a kiosk for providing transportation services, etc. Still further, while the client computing device <NUM> is depicted in <FIG> as a separate component of the system <NUM>, it is to be understood that the client computing device <NUM> can be a computing device included on the AV <NUM>.

The server computing device <NUM> also includes a processor <NUM> and memory <NUM> operably coupled to the processor <NUM>, the memory <NUM> including computer-executable instructions that are executed by the processor <NUM>. The server computing device <NUM> further includes a data store <NUM> that stores routing information <NUM> pertaining to an operational region of the multiple-modality transportation system <NUM>.

The transportation information server <NUM> similarly includes a processor <NUM>, memory <NUM> operably coupled to the processor <NUM>, and a data store <NUM> that includes transportation data <NUM> relating to one or more transportation modalities operative in the operational region of the system <NUM>. As described in greater detail below, the server computing device <NUM> can communicate with the transportation server <NUM> to receive real time updates pertaining to one or more transportation modalities operable in a region. The server computing device <NUM> can further communicate with the transportation server <NUM> to facilitate payment of a fare for a transportation service and to provide an authorization purchased with the fare to a user operating the client computing device <NUM>.

Referring now to <FIG>, an exemplary AV <NUM> is illustrated. In exemplary embodiments, the AV <NUM> of the multiple-modality transportation system <NUM> can be the AV <NUM>. The AV <NUM> can navigate about roadways without human conduction based upon sensor signals outputted by sensor systems of the AV <NUM>. The AV <NUM> includes a plurality of sensor systems, namely, a sensor system <NUM><NUM>,. , and a sensor system N <NUM>, where N can be substantially any integer greater than <NUM> (collectively referred to herein as sensor systems <NUM>-<NUM>). The sensor systems <NUM>-<NUM> are of different types and are arranged about the AV <NUM>. For example, the sensor system <NUM><NUM> may be a lidar sensor system and the sensor system N <NUM> may be a camera sensor (image) system. Other exemplary sensor systems included in the sensor systems <NUM>-<NUM> can include radar sensor systems, GPS sensor systems, sonar sensor systems, infrared sensor systems, and the like.

The AV <NUM> further includes several mechanical systems that are used to effectuate appropriate motion of the AV <NUM>. For instance, the mechanical systems can include, but are not limited to, a vehicle propulsion system <NUM>, a braking system <NUM>, and a steering system <NUM>. The vehicle propulsion system <NUM> can be an electric motor, an internal combustion engine, or a combination thereof. The braking system <NUM> can include an engine brake, brake pads, actuators, and/or any other suitable componentry that is configured to assist in decelerating the AV <NUM>. The steering system <NUM> includes suitable componentry that is configured to control the direction of movement of the AV <NUM>.

The AV <NUM> additionally includes a computing system <NUM> that is in communication with the sensor systems <NUM>-<NUM>, the vehicle propulsion system <NUM>, the braking system <NUM>, and the steering system <NUM>. The computing system <NUM> includes a processor <NUM> and memory <NUM>. The memory <NUM> includes computer-executable instructions that are executed by the processor <NUM>. Pursuant to various examples, the processor <NUM> can be or include a graphics processing unit (GPU), a plurality of GPUs, a central processing unit (CPU), a plurality of CPUs, an application-specific integrated circuit (ASIC), a microcontroller, a programmable logic controller (PLC), a field programmable gate array (FPGA), or the like.

The memory <NUM> of the computing system <NUM> includes an operation management system <NUM> that is configured to receive sensor signals from the sensor systems <NUM>-<NUM> and to output data pertaining to objects detected in the driving environment of the AV <NUM>. By way of example, the operation management system <NUM> can be configured to generate perception data indicative of a position, velocity, type, etc. for each of a plurality of objects in the driving environment of the AV <NUM> based upon the sensor signals. The operation management system <NUM> is further configured to control at least one of the mechanical systems of the AV <NUM> (e.g., at least one of the vehicle propulsion system <NUM>, the braking system <NUM>, and/or the steering system <NUM>). By way of example, the operation management system <NUM> can be configured to output control signals to any of the vehicle propulsion system <NUM>, the braking system <NUM>, or the steering system <NUM> to cause such systems <NUM>-<NUM> to direct the AV <NUM> along a trajectory. The operation management system <NUM> can further be configured to determine a planned trajectory for the AV <NUM> based upon the perception data.

With reference once again to <FIG>, exemplary operations of the system <NUM> are described in connection with providing multiple-modality transportation to a user of the client computing device <NUM>. The client computing device <NUM> executes a transportation client application <NUM> that is configured to allow a user to order multiple-modality transportation. The transportation client <NUM> is configured to communicate with a transportation server application <NUM> executing on the server computing device <NUM>. The transportation client application <NUM> presents data to the user of the client computing device <NUM> and receives input from the user by way of the GUI <NUM>.

In an initial interaction with the client computing device <NUM> in connection with ordering transportation for a trip, the user provides input to the client computing device <NUM> that indicates a desired destination for the trip. By way of various examples, the user can type a name of a desired destination, the user can select a desired destination on a map displayed on the GUI <NUM> by the transportation client <NUM>, or the user can select the desired destination from a list of potential destinations displayed on the GUI <NUM> by the transportation client <NUM>. The user can further optionally input a desired origin for the trip by way of the GUI <NUM>. In other exemplary embodiments, the transportation client <NUM> can receive location information indicating a present location of the client computing device <NUM> and uses the present location as the origin for the trip. By way of example, the transportation client <NUM> can receive the present location of the client computing device <NUM> from a sensor included on the client computing device <NUM> (e.g., a GPS sensor). In other examples, the client computing device <NUM> can determine the present location of the client computing device <NUM> based on IP address information, WiFi location information, or other data that is indicative of a location of the client computing device <NUM>. Responsive to receiving the user input indicating the desired destination of the user, the transportation client <NUM> generates a trip request that comprises data indicative of the trip origin and the trip destination, and outputs the trip request to the server computing device <NUM> by way of the network <NUM>.

The transportation client <NUM> can further be configured to generate the trip request based upon a user selection of a desired optimization parameter. For example, in connection with inputting the desired destination of the user, the user can further indicate whether the user wants to take a fastest route, a least expensive route, or a route having the shortest distance. In further embodiments, a user can adjust a relative preference among multiple optimization parameters. In a non-limiting example, the user can indicate a relative preference for time versus money. By way of further example, the user can indicate that a route that is a first threshold amount longer in duration is acceptable to the user provided that the route is at least a second threshold amount cheaper to take. The transportation client <NUM> can generate the trip request to include data indicating desired optimization parameters of the user and optionally a relative preference among multiple optimization parameters.

Referring now briefly to <FIG>, an exemplary GUI <NUM> for display on the display <NUM> of the client computing device <NUM> is shown. The GUI <NUM> includes selection regions <NUM>, <NUM> corresponding to single-mode transportation, which does not fall under the scope of the claims, and multiple-mode transportation, respectively.

The transportation client <NUM> generates the trip request based upon user indications of particular modalities of transportation that the user is willing to take. For instance, the user can provide input to the transportation client <NUM> that indicates transportation modalities that the user is willing to take along a multiple-modality transportation route. Modalities indicated by such user input can be stored as a modality setting of the transportation client <NUM>. In a non-limiting example, the modality setting can indicate that the user is willing to take AVs, trains and buses but that the user is not willing to walk or use a scooter for parts of a multiple-modality trip. The modality setting is region dependent such that the transportation client <NUM> stores different modality settings for different operational regions. By way of example, the modality setting of the transportation client <NUM> can indicate that the user is willing to use AVs, trains, buses, and scooters in San Francisco, whereas the modality setting can indicate that in Cleveland the user is only willing to use scooters and AVs for multiple-modality transportation. In still further embodiments, the modality setting can be time dependent. By way of example, the modality setting of the transportation client <NUM> can indicate that the user is willing to use a first set of modalities during a first set of times of day, and a second set of modalities during a second set of times of day. By way of further illustration, the modality setting of the transportation client <NUM> can indicate that the user does not want to walk or take bicycles after a certain time of night.

The transportation server application <NUM> receives the trip request from the transportation client <NUM> by way of the network <NUM>. Responsive to receiving the trip request, the transportation server <NUM> generates a route plan for the trip request. The transportation server <NUM> generates the route plan by performing an optimization over the routing information <NUM>. The transportation server <NUM> is configured to perform the optimization over the routing information <NUM> such that the transportation server <NUM> considers routes that include multiple transportation modalities. Stated differently, the transportation server <NUM> performs an optimization over the routing information <NUM> without constraining the route to occur along a single transportation modality. In exemplary embodiments, the transportation server <NUM> can be configured to perform the optimization over the routing information considering transportation modalities indicated in the trip request. In other embodiments, the transportation server <NUM> can be configured to perform the optimization over the routing information considering all available transportation modalities except those modalities specifically indicated in the trip request as being excluded.

The routing information <NUM> includes a map of a region that includes the trip origin and the trip destination indicated in the trip request. The routing information <NUM> can also include other data pertaining to various transportation modalities operating in the region or various conditions that can affect transportation through the region. By way of example, the routing information <NUM> can include routes and schedules for other modalities of transportation such as trains and buses, locations of personal transportation devices such as bicycles and scooters available on rideshare services, data indicating traffic conditions on streets in the region (e.g., updated in real time), and the like. The routing information <NUM> can further include locations of AVs in a fleet of AVs operating in the region. Still further, the routing information <NUM> can include real time updated weather and road conditions such as precipitation, outside temperature, road surface conditions (e.g., dry, wet, icy, etc.), and the like.

The server computing device <NUM> can be configured to periodically request updated information pertaining to transportation modalities operating in a region, and to further update the routing information <NUM> based upon such updated information. In an exemplary embodiment, the transportation server <NUM> can transmit a request to the transportation information server <NUM> for various data pertaining to a transportation modality operative in a region. The transportation information server <NUM> includes transportation data <NUM> that relates to one or more transportation modalities operative in the region. By way of example, the transportation information server <NUM> can be a computing device that is operated by a transportation service (e.g., a public transportation service or a privately operated transportation service such as a bikeshare service or a scooter-share service) and that provides information relating to transportation offered by the transportation service. Responsive to receiving the request from the transportation server <NUM>, the transportation information server provides at least a portion of the transportation data <NUM> to the server computing device <NUM>. In non-limiting examples, the transportation data <NUM> includes data pertaining to routes, schedules, available transportation resources (e.g., locations of shared personal transportation devices such as bicycles, scooters, and the like), current traffic conditions etc. for one or more transportation modalities. The server computing device <NUM> updates the routing information <NUM> to include the transportation data received from the transportation information server <NUM>. By way of example, the server computing device <NUM> can update the routing information <NUM> in real time with data pertaining to a public transportation service such as schedules, service outages, transportation delays, locations of transportation resources of the public transportation service (e.g., trains, buses, streetcars, etc.). In other exemplary embodiments, the transportation server <NUM> can update the routing information <NUM> responsive to receiving new information relating to locations of AVs in a fleet of AVs.

In an exemplary embodiment, the transportation server <NUM> can be configured to perform the optimization to find each of a cheapest route, a fastest route, and a shortest route. In other embodiments, the transportation server <NUM> can be configured to perform the optimization to find one of the least expensive route, the fastest route, or the shortest route based upon a user selection indicated in the trip request received from the transportation client <NUM>. In further embodiments, the transportation server <NUM> can be configured to perform the optimization to find various routes that satisfy desired optimization parameters and optionally a relative preference between desired optimization parameters, as indicated in the trip request received from the transportation client <NUM>.

It is to be understood that the optimization can be performed subject to various additional constraints. For example, the optimization can be performed over the routing information <NUM> to find a least expensive route subject to an additional maximum time constraint for the trip. In another example, the optimization can be performed over the routing information <NUM> to find a fastest route subject to an additional constraint that defines a maximum number of modality transfers for the trip. In still another example, an additional constraint can be a location constraint that defines an operational region within which an entirety of the route plan should be situated. In another example, the location constraint can define an operational region in which no part of the route plan should enter. Such additional constraints can be defined by the user by way of input at the client computing device <NUM> and the constraints included in the trip request transmitted by the transportation client <NUM>. In further embodiments, additional constraints on the optimization can be applied as default constraints by the transportation server <NUM>. By way of example, the transportation server <NUM> can exclude certain transportation modalities such as walking, scooters, and bicycles in areas where the routing information <NUM> indicates that severe weather such as heavy rainfall is present.

As noted above, the transportation server <NUM> generates a route plan for a trip request received from the transportation client <NUM> by performing the optimization over the routing information <NUM>. Depending on current travel conditions as indicated in the routing information <NUM>, a route plan that includes multiple transportation modalities can be the route plan that best satisfies the various optimization constraints (e.g., that has a lowest cost according to a cost function used to perform the optimization).

By way of example, and referring now to <FIG>, a conceptual diagram of various exemplary route plans is shown. <FIG> depicts a region <NUM> that includes a transportation origin <NUM> and a transportation destination <NUM>. For example, the transportation origin <NUM> can be the place of work of a user, whereas the transportation destination <NUM> can be a home of a user. The region <NUM> includes a plurality of roadways <NUM>-<NUM> and a passenger railway <NUM>. Three exemplary route plans are depicted in <FIG>. Each of the three route plans begin at the origin <NUM> and include a first modality of transport from the origin <NUM> along the roadway <NUM> to a first terminus point <NUM>. By way of example, the first modality of transport can be an AV that conveys the user from the origin to at least the first modality terminus point <NUM>. At the first terminus point <NUM>, the three route plans diverge. A first route plan that is a single-modality route plan continues on along the roadways <NUM>, <NUM>, and <NUM> all the way to the origin destination <NUM>. Thus, for example, the first route plan is a route plan that conveys the user all the way from the origin <NUM> to the destination <NUM> in a single AV.

Whereas traveling along the first route plan may be the fastest way to reach the destination <NUM> from the origin <NUM> under some travel conditions, under other travel conditions it may be faster to travel along different routes. For example, during rush hour periods of high traffic, traversing the roadway <NUM> in an AV may be slower than taking a train that runs on the railway <NUM>. Therefore, according to a second route plan, the user exits the first modality of transport at the first terminus point <NUM> and continues to a first train station <NUM> that lies along the railway <NUM>. In the second route plan the user takes a second transport modality, the train, along the railway <NUM> to a second train station <NUM>. According to the second route plan the user then takes a third transport modality, such as a bus or a second AV, from a second terminus point <NUM> to the destination <NUM>. The second route plan therefore avoids potential traffic along roadway <NUM>. In another example, the third route plan can direct a user to exit a first AV at the first terminus point <NUM> and to walk to a third terminus point <NUM> at the roadway <NUM> that runs parallel to the roadway <NUM>. In such example, walking from the first terminus point <NUM> to the third terminus point <NUM> can be considered to be using a second transportation modality. The third route plan can then provide that the user is to take a second AV from the third terminus point <NUM> to the destination <NUM> along the roadway <NUM>. The third route may be the fastest route when, for example, the roadway <NUM> is congested with traffic while another roadway, the roadway <NUM>, allows more rapid movement of vehicles.

Upon generating the route plan, or multiple route plans (e.g., satisfying different optimization constraints), the transportation server <NUM> outputs data indicative of the route plan, or plans, to the transportation client <NUM>. Responsive to receiving the data, the transportation client <NUM> can cause to be displayed on the display <NUM> at least a portion of a route plan indicated in the data. The user can then indicate confirmation of a route plan by providing confirmation input by way of the GUI <NUM>. By way of example, and referring again to <FIG>, the GUI <NUM> can include a region <NUM> that displays graphical indicia of one or more route plans. The graphical indicia can include a path taken by a route plan including indications of terminus points at which the route plan switches from one modality of transport to another modality of transport. For example, the region <NUM> shown in the GUI <NUM> depicts a single-mode route plan <NUM> that extends from an origin <NUM> to a destination <NUM>. The region <NUM> also depicts a multiple-mode route plan <NUM> that extends from the origin <NUM> to a first terminus point <NUM>, from the first terminus point <NUM> to a second terminus point <NUM>, and from the second terminus point <NUM> to the origin <NUM>.

Referring once again to <FIG>, the transportation client <NUM> outputs a confirmation indication to the transportation server <NUM> responsive to receiving confirmation input from the user at the GUI <NUM>. The confirmation indication indicates to the transportation server <NUM> that the user has confirmed the route plan. In exemplary embodiments wherein the transportation server <NUM> outputs multiple route plans to the transportation client <NUM> responsive to receiving a trip request, the confirmation indication can include both a selection of one of the multiple route plans and a confirmation of the route plan. By way of example, and referring again to <FIG>, the GUI <NUM> can include a selection region <NUM> that functions as a confirmation button for receipt of confirmation input. Responsive to receipt of user input in the selection region <NUM> (e.g., by way of a mouse click, tapping of the selection region <NUM> on a touchscreen, or the like), a client computing device on which the GUI <NUM> is displayed outputs a confirmation indication to a transportation server. Prior to providing the confirmation input at the selection region <NUM>, the user can select one of the routes <NUM>, <NUM> by selecting graphical indicia corresponding to the routes <NUM>, <NUM> in the GUI <NUM>.

Referring yet again to <FIG>, the server computing device <NUM> receives the confirmation indication from the client computing device <NUM>. Responsive to receiving the confirmation indication, the server computing device <NUM> can issue a pickup instruction to the AV <NUM>. In exemplary embodiments, the server computing device <NUM> includes an AV coordination component <NUM> that is configured to direct operation of the AV <NUM> and other AVs that may be operating in a fleet of AVs in an operational region that includes the trip origin and the trip destination. The AV coordination component <NUM> can issue a pickup instruction to the AV <NUM> to cause the AV <NUM> to navigate to a pickup location included in the route plan selected and confirmed by the user. The pickup instruction can be configured to cause the AV <NUM> to arrive at a location along the route prior to the user arriving at the location, so that the AV <NUM> can stand ready to continue conveying the user along the route when the user arrives. By way of example, and referring briefly to <FIG>, the pickup instruction can cause an AV to navigate to the second terminus <NUM> in sufficient time that the AV will be waiting at the second terminus <NUM> when the user arrives at the second train station <NUM>.

The server computing device <NUM> can further be configured to dynamically update a pickup location for an AV that is indicated in a route plan while the user is following the route plan. In an exemplary embodiment, subsequent to confirmation of the route plan with the user, the AV <NUM> can be traveling to a pickup location indicated in the route plan. The transportation client <NUM> can be configured to periodically output location data to the transportation server <NUM>, the location data indicating an updated location of a user of the client computing device <NUM> as the user follows the route plan. The AV <NUM> can also be configured to periodically output location data that is indicative of an updated location of the AV <NUM> to the server computing device <NUM>. Based upon the updated locations of the user and the AV <NUM>, the transportation server <NUM> can update the route plan to indicate an updated pickup location that is different than the pickup location that was originally indicated in the route plan. By way of example, the transportation server <NUM> can output an indication of the updated pickup location to the transportation client <NUM>, and the transportation client <NUM> can cause the updated pickup location to be presented to the user on the display <NUM>. By way of further example, the AV coordination component <NUM> can issue an updated pickup instruction to the AV <NUM> that causes the AV <NUM> to navigate to the updated pickup location.

The transportation server <NUM> can update a pickup location according to any of various criteria. By way of example, and not limitation, as the user is following the route plan the transportation server <NUM> can perform an optimization over at least a portion of the routing information <NUM> based upon the updated location of the user and the updated location of the AV <NUM>. The optimization can be configured to determine whether an updated pickup location would result in a faster travel time from the updated location of the user to the trip destination. In other embodiments, the transportation server <NUM> can update the pickup location based upon determining that either the user or the AV <NUM> will reach the original pickup location a threshold amount of time before the other. For example, the transportation server <NUM> can update the pickup location responsive to determining that one of the user or the AV <NUM> will reach the origin pickup location greater than one minute, greater than two minutes, or greater than five minutes before the other of the user or the AV <NUM>. The transportation server <NUM> can update the pickup location such that the user and the AV <NUM> reach the updated pickup location within the threshold amount of time.

In further embodiments, the transportation server <NUM> can be configured to update the pickup location based upon determining that the user has deviated from the route plan. The transportation server <NUM> can determine that the user has deviated from the route plan based upon the updated location of the user. In an example, the transportation server <NUM> can determine that the user has deviated from the route plan based upon the updated location of the user being greater than a threshold distance away from any location along the route plan. In another example, the transportation server <NUM> can determine that the user has deviated from the route plan based upon an amount of time taken for the user to reach the updated location of the user. In still another example, the transportation server <NUM> can determine that the user has deviated from the route plan based upon a modality transition occurring at a location other than a terminus point indicated in the route plan. By way of further illustration, if the updated location of the user indicates that the user has exited the AV <NUM> prior to the AV <NUM> reaching its terminus point along the route plan, the transportation server <NUM> can determine that the user has deviated from the route plan. In another illustrative example, if the updated location of the user indicates that the user has disembarked from a train one or more stops earlier or later than indicated in the route plan, the transportation server <NUM> can determine that the user has deviated from the route plan.

In various embodiments, the transportation server <NUM> can be configured to process payment for transportation along at least a portion of the trip route responsive to receiving the confirmation indication. For instance, the transportation server <NUM> can process payment for portions of the trip route that are traversed by the AV <NUM>. In an exemplary embodiment, the transportation server <NUM> can be configured to charge an account based upon stored account information associated with the user of the client computing device <NUM>. In another example, the transportation server <NUM> can transmit a request for payment information to the client computing device <NUM>. The user of the client computing device <NUM> can then input payment information (e.g., a credit card number) by way of the GUI <NUM>, and the transportation client <NUM> can transmit the payment information to the server computing device <NUM>.

The transportation server <NUM> can further be configured to communicate with the transportation information server <NUM> in order to pay for a fare of the user along one or more of the modalities included in the multiple-modality route plan and receive an authorization for use of a transportation resource, a transportation resource being a means of conveyance such as a train, an AV, a bus, a bicycle, a scooter, and so forth. In an exemplary embodiment, the transportation information server <NUM> includes a fares component <NUM> that is configured to process fare payments and to output an authorization for use of a transportation resource. By way of example, the fares component <NUM> can receive payment information and output an electronic ticket that is usable as proof of payment for admittance to a train or other public conveyance. The transportation server <NUM> can output a fare request that includes payment information to the transportation information server <NUM>. The fares component <NUM> of the transportation information server <NUM> determines whether the payment information indicates a valid payment for a transportation resource indicated in the fare request (e.g., that the payment is sufficient for the transportation resource). Responsive to determining that the payment is valid, the fares component <NUM> outputs an indication of authorization for use of the transportation resource to the server computing device <NUM>. The transportation server <NUM> can then provide the indication of authorization to the transportation client <NUM>, whereupon the transportation client <NUM> can store the indication of authorization in the memory <NUM> for later use. By way of example, the authorization indication can include an electronic ticket that is usable for admittance to the transportation resource, and the transportation client <NUM> can subsequently display the electronic ticket on the display <NUM>.

In various exemplary embodiments, the transportation client <NUM> and/or the AV <NUM> can be configured to assist the user of the multiple-modality transportation system <NUM> by providing guidance relating to a multiple-modality transportation route. In a non-limiting example, the AV <NUM> can be configured to display guidance to a user relating to the route plan. In exemplary embodiments, the AV <NUM> can include a display <NUM> that is positioned facing a passenger compartment of the AV <NUM>. The display can display a GUI <NUM>. The GUI <NUM> can indicate various information relating to the route plan including a location of the user along the route indicated in the route plan, a remaining time to reach the destination of the user, a remaining time until the route plan indicates that the user is to transfer to another modality of transport, or other desirable information. In another example, when the AV <NUM> reaches a terminus point at which the route plan indicates that the user is to switch to a different modality of transport, the AV <NUM> can display an indication on the display <NUM> that indicates information pertaining to the next modality of transport. By way of example, when the AV <NUM> reaches a terminus point where the route plan indicates that the user is to leave the AV <NUM> and board a train, the AV <NUM> can cause the display <NUM> to display an indication that the user should board the train. In further embodiments, the transportation client <NUM> can be similarly configured to display any of the data described above with respect to the display <NUM> on the display <NUM> of the client computing device <NUM>.

In still other embodiments, responsive to the transportation server <NUM> determining that the user of the client computing device <NUM> has deviated from the route plan, the transportation server <NUM> can output a query to at least one of the client computing device <NUM> or the AV <NUM>. Responsive to receipt of the query, the client computing device <NUM> can display a prompt on the display <NUM> (or the AV <NUM> can display the prompt on the display <NUM>) to request user input indicating whether the user wishes to update the multiple-modality route plan to continue to the destination of the user, or if the user wishes to cancel the trip. The client computing device <NUM> can output an indication to the server computing device <NUM> that the trip is to be cancelled responsive to receipt of user input indicating cancellation. The client computing device <NUM> can output an indication to the server computing device <NUM> to update the multiple-modality route plan responsive to receipt of user input indicating that the user wishes to update the multiple-modality route plan. Similarly, the AV <NUM> can output such indications responsive to receipt of the user input (e.g., by way of the GUI <NUM>).

It is to be understood that while an exemplary multiple-modality transportation system <NUM> has been described herein, various modifications to the exemplary system <NUM> are possible. By way of example, the transportation information server <NUM> can be one of many server computing devices that correspond to different transportation modalities, or transportation modalities owned by different entities. In another example, multiple server computing devices can be configured to communicate with the server computing device <NUM> in connection with performing functionality described herein as being performed by the transportation information server <NUM>. By way of example, a first server computing device can include the data store <NUM> that stores the transportation data <NUM> whereas a second server computing device can include the memory <NUM> that includes the fares component <NUM>. Other potential modifications to the exemplary system <NUM> are also contemplated.

In view of the disclosure set forth above, it is to be appreciated that in various embodiments a multiple-modality transportation system includes a client device, a server, and one or more AVs, wherein the client device and the AVs are in communication with the server by way of a network. In an exemplary embodiment, functionality described as being performed by the server can be performed by a plurality of computing devices such as in a cloud computing system. In exemplary operations the client can send a trip request to the server, and the server can perform an optimization over routing information to determine potential route plans for the trip request, where one or more of the route plans can include multiple transportation modalities. The routing information can include real time traffic, weather, public alert, and transportation information pertaining to one or more modalities operating in a region. In an exemplary embodiment, the server can receive real time updates to the routing information from any of various sources such as weather, traffic, and transportation information servers. The server can output potential route plans to the client device. The server can then output instructions to one or more AVs to carry out a route plan in the potential route plans responsive to receiving an indication that a user of the client device has selected the route plan.

<FIG> illustrate exemplary methodologies relating to multiple-modality transport. While the methodologies are shown and described as being a series of acts that are performed in a sequence, it is to be understood and appreciated that the methodologies are not limited by the order of the sequence. For example, some acts can occur in a different order than what is described herein. In addition, an act can occur concurrently with another act. Further, in some instances, not all acts may be required to implement a methodology described herein.

Moreover, the acts described herein may be computer-executable instructions that can be implemented by one or more processors and/or stored on a computer-readable medium or media. The computer-executable instructions can include a routine, a sub-routine, programs, a thread of execution, and/or the like. Still further, results of acts of the methodologies can be stored in a computer-readable medium, displayed on a display device, and/or the like.

With reference to <FIG>, an exemplary methodology <NUM> performed by a server computing device in a multiple-modality transportation system is illustrated. The methodology <NUM> begins at <NUM>, and at <NUM>, a trip request that comprises data indicative of a trip origin and a trip destination is received. In an example, the trip request can be received from a client computing device that is operated by a user in connection with ordering transportation from the trip origin to the trip destination. At <NUM> a route plan that includes multiple transportation modalities is generated for the trip request. The route plan is generated by performing an optimization over routing information that includes a map of a region that includes the trip origin and the trip destination indicated in the trip request. The multiple transportation modalities include transport by way of an AV and a second transportation modality. A pickup instruction is issued to an AV at <NUM>, the pickup instruction configured to cause the AV to navigate to a pickup location included in the route plan generated at <NUM>. The methodology <NUM> completes at <NUM>.

With reference now to <FIG>, an exemplary methodology <NUM> performed by a client computing device in a multiple-modality transportation system is illustrated. The methodology <NUM> begins at <NUM>, and at <NUM> a trip request that comprises data indicating a trip origin and a trip destination is generated. The trip request can be generated responsive to receiving user input that is indicative of the desired trip destination of a user of the client computing device. The trip request is output to a server computing device at <NUM>. The trip request is configured to cause the server computing device to generate a route plan for the trip request that includes multiple transportation modalities. As described in greater detail above, the server computing device can generate the route plan by performing an optimization over routing information that includes a map of the trip origin and the trip destination indicated in the trip request. At <NUM>, responsive to receiving data indicative of a route plan generated by the server computing device for the trip request, a graphical indication of at least a portion of the route plan is displayed on a display of the client computing device, whereupon the methodology <NUM> ends <NUM>.

With reference now to <FIG>, an exemplary methodology <NUM> performed by an AV in a multiple-modality transportation system is illustrated. The methodology <NUM> begins at <NUM> and at <NUM> the AV navigates to a pickup location indicated in a pickup instruction, the pickup instruction being transmitted to the AV by a server computing device in connection with a route plan that includes multiple transportation modalities. In an exemplary embodiment, the route plan can be a route plan generated by the server in response to a trip request from a client computing device. At <NUM>, a first graphical indication that indicates at least a portion of the multiple modality route plan is displayed on a display included in the AV. A second graphical indication is displayed at <NUM> responsive to arriving at a terminus point in the route plan, the terminus point being a point at which the route plan indicates that a passenger of the AV is to change from using a first modality of transportation to a second modality of transportation. The second graphical indication indicates the second modality of transportation, thereby providing the user guidance in transferring from the first modality of transportation (e.g., the AV) to the second modality of transportation. The methodology <NUM> ends at <NUM>.

With reference now to <FIG>, a methodology <NUM> performed by a server computing device of a transportation system in connection with dynamically updating a route plan. The methodology <NUM> begins at <NUM> and at <NUM> a trip request that indicates a trip origin and a trip destination is received. By way of example, the trip request is received from a client computing device that is configured to communicate with the server computing device. At <NUM> a route plan is generated for the trip request based upon the trip origin, the trip destination, and routing information for a region that includes the trip origin and the trip destination. A pickup instruction is issued to an AV at <NUM>, the pickup instruction configured to cause the AV to navigate to a first pickup location included in the route plan. In an example, the first pickup location can be a location at which the AV is intended to meet a passenger at a particular time. At <NUM> a determination is made whether an updated location of a user (e.g., a user of a client computing device from which the trip request is received at <NUM>) has been received. If an updated location of the user has been received, an update to the pickup instruction is transmitted to the AV at <NUM> and the methodology <NUM> ends <NUM>. The update to the pickup instruction is configured to cause the AV to navigate to a second pickup location. For example, the second pickup location can be a pickup location that is closer to the updated location of the user in order to avoid having the AV or the user wait for the other. If it is determined that an updated location of the user has not been received at <NUM>, the methodology <NUM> completes at <NUM>.

With reference now to <FIG>, another methodology <NUM> performed by a server computing device of a transportation system in connection with dynamically updating a route plan is illustrated. The methodology <NUM> begins at <NUM> and at <NUM> a trip request that indicates a trip origin and a trip destination is received. At <NUM> a route plan is generated for the trip request based upon the trip origin, the trip destination, and routing information relating to a region that includes the trip origin and the trip destination. A pickup instruction is issued to an AV at <NUM>, the pickup instruction configured to cause the AV to navigate to a pickup location included in the route plan. At <NUM>, a determination is made whether an updated location of a user of a client computing device (e.g., a client computing device from which the trip request is received at <NUM>) has been received. If an updated location of the user has been received, a query is transmitted to the client computing device at <NUM>, whereupon the methodology <NUM> completes <NUM>. The query is configured to cause the client computing device to prompt the user for input that indicates whether the route plan is desirably updated by the user. Subsequently, the user can provide input to the client computing device indicating that the route is desirably updated or, for example, that the user wishes to cancel transportation. If the route is desirably updated, the server computing device can transmit an update to the pickup instruction as described above with respect to the methodology <NUM>. If it is determined at <NUM> that an updated location of the user has not been received, the methodology <NUM> ends at <NUM>.

With reference now to <FIG>, a methodology <NUM> performed by a client computing device of a transportation system in connection with dynamically updating a route plan is illustrated. The methodology <NUM> begins at <NUM> and at <NUM> the client computing device receives data indicative of a route plan generated by a server computing device. The route plan can indicate, for example, a path to be traveled from an origin to a destination, modalities of transport to be taken by a user for various portions of the path, etc. At <NUM> a graphical indication of a first pickup location is displayed on a display of the client computing device, the first pickup location being included in the route plan. At <NUM>, an updated location of a user of the client computing device is output to the server computing device. A graphical indication of a second pickup location that is determined by the server computing device based on the updated location of the user is displayed by the client computing device at <NUM>. By way of example, the graphical indication of the second pickup location can be displayed in response to receiving an update from the server computing device that indicates the second pickup location as an update to the route plan. The methodology <NUM> completes at <NUM>.

Referring now to <FIG>, a high-level illustration of an exemplary computing device <NUM> that can be used in accordance with the systems and methodologies disclosed herein is illustrated. For instance, the computing device <NUM> may be or include any of the computing systems <NUM>, <NUM>, <NUM>, <NUM>. The computing device <NUM> includes at least one processor <NUM> that executes instructions that are stored in a memory <NUM>. The instructions may be, for instance, instructions for implementing functionality described as being carried out by one or more systems discussed above or instructions for implementing one or more of the methods described above. The processor <NUM> may be a GPU, a plurality of GPUs, a CPU, a plurality of CPUs, a multi-core processor, etc. The processor <NUM> may access the memory <NUM> by way of a system bus <NUM>. In addition to storing executable instructions, the memory <NUM> may also store data pertaining to transportation modalities available in a region, user account information, and so forth.

The computing device <NUM> additionally includes a data store <NUM> that is accessible by the processor <NUM> by way of the system bus <NUM>. The data store <NUM> may include executable instructions, routing information pertaining to a region, data relating to transportation modalities in a region, user account information, etc. The computing device <NUM> also includes an input interface <NUM> that allows external devices to communicate with the computing device <NUM>. For instance, the input interface <NUM> may be used to receive instructions from an external computer device, from a user, etc. The computing device <NUM> also includes an output interface <NUM> that interfaces the computing device <NUM> with one or more external devices. For example, the computing device <NUM> may transmit control signals to the vehicle propulsion system <NUM>, the braking system <NUM>, and/or the steering system <NUM> by way of the output interface <NUM>. In another example, the output interface <NUM> can be or include a display.

Additionally, while illustrated as a single system, it is to be understood that the computing device <NUM> may be a distributed system. Thus, for instance, several devices may be in communication by way of a network connection and may collectively perform tasks described as being performed by the computing device <NUM>.

Various functions described herein can be implemented in hardware, software, or any combination thereof. If implemented in software, the functions can be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes computer-readable storage media. A computer-readable storage media can be any available storage media that can be accessed by a computer. By way of example, and not limitation, such computer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Disk and disc, as used herein, include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and blu-ray disc (BD), where disks usually reproduce data magnetically and discs usually reproduce data optically with lasers. Further, a propagated signal is not included within the scope of computer-readable storage media. Computer-readable media also includes communication media including any medium that facilitates transfer of a computer program from one place to another. A connection, for instance, can be a communication medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio and microwave are included in the definition of communication medium.

Claim 1:
A computing system (<NUM>) comprising:
a processor; and
memory that stores computer-executable instructions that, when executed by the processor, cause the processor to perform acts comprising:
receiving a trip request, the trip request comprising data indicative of a trip origin (<NUM>) and a trip destination (<NUM>), wherein the trip request is based upon different modality settings for different operational regions, wherein a modality setting indicates particular transportation modalities that the user is willing to take, wherein the transportation modalities include transport by way of an autonomous vehicle, AV, and a second transportation modality; and
generating a route plan for the trip request by performing an optimization over routing information (<NUM>), considering the transportation modalities indicated in the trip request, the routing information including a map of a region (<NUM>) that includes the trip origin and the trip destination; and
issuing a pickup instruction to an AV (<NUM>) in communication with the computing system, wherein the pickup instruction is configured to cause the AV to navigate to a pickup location included in the route plan.