Patent ID: 12216672

DETAILED DESCRIPTION

The following disclosure provides many different embodiments, or examples, for implementing different features of the provided subject matter. Specific examples of components, values, operations, materials, arrangements, or the like, are described below to simplify the present disclosure. These are, of course, merely examples and are not intended to be limiting. Other components, values, operations, materials, arrangements, or the like, are contemplated. For example, the formation of a first feature over or on a second feature in the description that follows may include embodiments in which the first and second features are formed in direct contact, and may also include embodiments in which additional features may be formed between the first and second features, such that the first and second features may not be in direct contact. In addition, the present disclosure may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed.

Further, spatially relative terms, such as “beneath,” “below,” “lower,” “above,” “upper” and the like, may be used herein for ease of description to describe one element or feature's relationship to another element(s) or feature(s) as illustrated in the figures. The spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. The apparatus may be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein may likewise be interpreted accordingly.

A vehicle fleet as a distributed database is expensive and time-consuming to duplicate, because all data must be uploaded from each vehicle, and the connectivity and bandwidth thereof is unstable and expensive. Querying a distributed database directly must be done carefully to ensure a query does not crash the system.

To help maintain working condition of the distributed database, i.e. —the working condition of each vehicle in the fleet, at least some embodiments herein screen queries for feasibility, and tailor queries for different models in the fleet.

In at least some embodiments, tailoring the query for a model includes referring to a database of computational resources and running processes for that model. In at least some embodiments, a process is crafted for execution on a specific model to help vehicles of that model avoid resource shortage. In at least some embodiments, even models having lower amounts of computational resources are programmed to perform more complex tasks, and the query creator is notified if the amount of time to gather the data is extreme. In at least some embodiments, a query engine estimates when the results can be obtained based on likelihood of encountering events producing the requested data, bandwidth to the server, and computational resources of the model.

In at least some embodiments, a vehicle is configured to execute the query to gather the requested data. In at least some embodiments, the vehicle is configured to store the requested data in the vehicle until one or more conditions are met for uploading the data. In at least some embodiments, a vehicle is configured to execute multiple queries and store multiple instances of requested data until uploading. In at least some embodiments, a vehicle is configured to compress, filter, or purge requested data, in response to unavailable storage resources, according to corresponding priority values established by the server. At least some embodiments herein enable querying a distributed database directly, and informing those querying of the likelihood and waiting time for receiving results.

A user, such as a software developer, an insurance provider, a market researcher and a law enforcement officer, is able to use an on-demand data retrieval (ODDR) system to enter a data request into a user interface, such as a graphical user interface (GUI). The software developer is a software developer who develops applications, middleware or OS (operating systems) to be run on the vehicle, for example. Example applications include automated-driving system applications, such as an object recognition application, a road recognition application, a sensor fusion application, a localization application, a path planner application, a controller application, etc. The data request is analyzed and stored in a server and then transmitted to a vehicle by the server. On the server side, the data requests are stored in a storage unit and a request queue is generated based on the stored requests. The user is able to see or request updates on a status of the data request. For example, while the data request is still within the server prior to transmission to the vehicle, the status may be indicated as “pending.” Once the server transmits the data request to the vehicle, the status may be updated to “submitted.” This allows the user to see and track the status of data requests made to the vehicle. One of ordinary skill in the art would recognize that the description refers to a vehicle for the sake of clarity; however, the description is applicable to groups of vehicles in addition to a single vehicle.

The user interface for generating the data request includes forms related to vehicle identifying information, data types being requested, start time and end time. In some embodiments, the start time and the end time are absolute times, such as Unix time, that is an elapsed time since a Unix epoch time. In some embodiments, the start time and the end time are relative times to the time that the data request is received by the vehicle. In some embodiments, the start time and the end time are relative times to a trigger event. The trigger event is the occurrence within the vehicle or in the environment surrounding the vehicle about which the user is seeking data or receipt of a data request by the vehicle. For example, a trigger event resulting from an environment surrounding the vehicle includes sudden acceleration, sudden braking, capturing an image of a target of a data request, detecting of a target of a data request or other suitable occurrences. The user information for monitoring a status of data requests includes identifying information of the data request and a status of the data request, such as pending or submitted.

In some embodiments, once the data request is received by the vehicle, the data request is processed to make the data request agnostic as to the source of the data request. In some embodiments, a data request identification (ID) is assigned to the received data request by the vehicle, for example by a request abstractor in the vehicle. In some embodiments, the data request ID is assigned to the data request prior to transmission of the data request to the vehicle. In some embodiments, a data request is generated by an application running in the vehicle and the application assigns the data request ID. In other words, the data is processed in a consistent manner regardless of a program or system that transmits the data request to the vehicle. In some embodiments, a data request is generated by a software component stored within the vehicle, and the data is processed in consistent with a data request received from an external device. This helps to sharing the same data collection software components among trigger data collection, where an application generates a data collection request to the logger, and the ODDR-based external data collection request.

In some embodiments, once the data request is received by the vehicle, the data request is processed to make the data request agnostic to the sensors within the vehicle and the server. In some embodiments, the data request is generated by an application running in the vehicle. In some embodiments, an application programming interface (API) is usable to make the data request from the application agnostic to the sensors within the vehicle or information from the server. This helps to maximize the user's ability to collect data without programming a request for specific sensor models. The data request is then transferred to a data collector and the requested data is collected in response to occurrence of the trigger event. In the situation where the trigger event had already occurred, such as a traffic accident, the data request is fulfilled based on data stored within a storage device within the vehicle. A time frame, i.e., start and end times, of the collected data is determined based on the data request. The collected data is transferred back to the server.

The collected data is then stored in the server and a notification is sent to the user regarding completion of the data request. For example, the status of the data request is updated to “complete” on the user interface.

In some instances, a budget management system or a payment system is implemented on the server side or vehicle side, such that the user is charged a fee for a data request. The fee is payable either at the submission of the request or at completion of data collection. The fee is adjustable based on the type and amount of data requested. In some embodiments, when the total amount of fee that is charged to the user reaches a maximum threshold of user's budget, the data request from the user is rejected.

This ODDR system allows users to access information collected by vehicle in an on-demand style. That is, the data is not necessarily continuously collected, but could be collected to satisfy specific desires of a user. In some embodiments, the ODDR system helps users, such as software developers, collecting data to update the design, implementation and parameter tuning of their software in an exploratory way based on collected data so that the users are able to continuously improve the software by delivering updates from the server to the vehicle via network, for example, as an Over-the-Air (OTA) update. In some embodiments, the ODDR system helps machine learning developers who develops machine learning model for the applications collecting data to train the model with the data which was not available when the model was initially developed so that the machine learning developers are able to update the models to fix weakness and issues of the model continuously. In some instances, an insurance provider would be able collect data related to a traffic accident. In some instances, law enforcement would be able to collect information related to a crime or a traffic accident.

FIG.1is a schematic diagram of a request retrieval system100in accordance with some embodiments. The request retrieval system100includes a user interface (UI)110. The UI110is configured to receive a user request for data from a vehicle140. The request retrieval system100further includes a server120configured to receive the user request from the UI110; transmit the user request to the vehicle140; receive data from the vehicle140; and provide the data to the user via an accessible console150. The server120includes a communication section130for communicating with the UI110and the vehicle140. The request retrieval system100further includes an accessible console150configured to communicate data collected from the vehicle140to the user.

The UI110is configured to receive input instructions from the user. In some embodiments, the user includes a software developer. In some embodiments, the user includes a machine learning model developer. In some embodiments, the user includes an insurance provider. In some embodiments, the user includes law enforcement personnel. In some embodiments, the user includes a market research company. The UI110provides options for the user to select what type of vehicle and what type of data is being requested. In some embodiments, the UI110is capable of generating the data request using forms related to vehicle identifying information, data types being requested, start time and end time. In some embodiments, the start time and the end time are absolute times, such as Unix time, that is an elapsed time since a Unix epoch time. In some embodiments, the start time and the end time are relative times to the time that the data request is received by the vehicle. In some embodiments, the start time and the end time are relative times to a trigger event. In some embodiments, the UI110also provides the user with options for selecting a trigger event and a data collection duration relative to the trigger event. In some embodiments, the UI110includes information related to a type of vehicle from which data is requested. In some embodiments, the UI110includes vehicle ID which is able to uniquely identify a vehicle as a target of the request. For example, the vehicle ID includes a universally unique identifier (UUID) format. In some embodiments, the UI110includes data type that is able to identify the source of the data that user wants to collect. For example, the data type includes sensor ID of sensor that sensor data is collected from, application ID of application that application log is collected from. In some embodiment, the format of the sensor ID and application ID includes a universally unique identifier (UUID) format. In some embodiments, the UI110includes drop down menus. In some embodiments, the UI110includes editable fields for receiving information related to a data request. In some embodiments, the UI110provides information regarding what data option types are available to the user. In some embodiments, the data option types available depend on the user. For example, law enforcement is able to select more data options than an insurance provider in some embodiments.

In some embodiments, the UI110includes a graphical user interface (GUI). In some embodiments, the UI110includes a mobile terminal, such as a mobile telephone, connectable to the server120. In some embodiments, the UI110includes a web interface such as RESTful API. In some embodiments, the UI110includes a computer connectable to the server120. In some embodiments, the UI110is capable of wireless connection to the server120. In some embodiments, the UI is connectable to the server120by a wired connection. The UI110is also able to provide the user with updates regarding a status of a data request. In some embodiments, the UI110provides status updates regarding a data request in response to an additional query by the user. In some embodiments, the UI110provides status updates regarding a data request upon receipt of updated information from the server120automatically without user interaction. In some embodiments, the status update causes the UI110to trigger an alert for the user. In some embodiments, the alert includes an audio or visual alert.

In some embodiments, the UI110includes a means for accepting payment of a fee from the user. In some embodiments, the UI110includes data entry fields to permit the user to enter payment card information. In some embodiments, the UI110includes a reader for detecting payment card information, such as a magnetic stripe reader, a bar code reader, a chip reader, or another suitable reader.

The server120includes a communication section130configured to communicate with the UI110and the vehicle140. The communication section130includes a receiver131configured to receive data requests from the UI110. In some embodiments, the receiver131includes a wireless receiver. In some embodiments, the receiver is configured to receive the data requests via a wired connection. In some embodiments, the receiver131is further configured to perform initial processing on the received data request. In some embodiments, the received data request includes priority level information. In some embodiments, the receiver131is configured to assign a priority level to the data request based on an identity of the user that submitted the data request or a fee paid by the user that submitted the data request. In some embodiments, the receiver131is configured to assign a request identification (ID) number to each received data request. In some embodiments, the server120is configured to limit access to certain sensors within the vehicle140based on an identity the user. For example, a third-party user will not be able to access sensor related to safety functions of the vehicle140in some embodiments.

The communication section130further includes a memory unit132configured to store data requests received by the receiver131. In some embodiments, the memory unit132includes a random access memory, a solid state memory, or another type of memory. In some embodiments, the memory unit132is configured to store the data requests along with a status of the data request. In some embodiments, the status of the data request includes pending (prior to transmission of the data request to the vehicle140); submitted (following transmission of the data request to the vehicle140); and completed (following receipt of the requested data from the vehicle140). In some embodiments, the memory unit132is accessible by the user. In some embodiments, updates to information in the memory unit132trigger notifications of a user associated with the information updated in the memory unit132. In some embodiments, the memory unit132stores data requests in conjunction with time stamp data indicating a time at which the data request was received. In some embodiments, the memory unit132stores data requests in association with a priority level. In some embodiments, the priority level is determined based on an identity of the user. For example, in some embodiments, law enforcement has higher priority than an insurance provider, which has higher priority than a normal user, such as a software developer. In some embodiments, the priority level is determined based on a fee paid by the user. For example, in some embodiments, a user is able to pay a fee in order to increase a priority level of their request in order to obtain the requested data sooner. In some embodiments, the priority level of a data request is increased as an amount of time between initial storage of the data request and transmission of the data request to the vehicle increases.

The communication section130further includes a transmitter133. The transmitter133is configured to transmit a status of data requests to the UI110. In some embodiments, the status of the data requests is wirelessly transmitted to the UI110. In some embodiments, the status of the data requests is transmitted to the UI110via a wired connection. In some embodiments, the transmitter133is configured to provide an update on a data request automatically in response to an update in the memory unit132. In some embodiments, the transmitter133is configured to provide an update on a data request in response to a received update request from the user. In some embodiments, the transmitter133is configured to automatically transmit a request ID upon initially saving the data request in the memory unit132. In some embodiments, the status of the data request includes a priority level of the data request. In some embodiments, the status of the data request includes an estimated time until the data request is transmitted to the vehicle140.

The communication section130further includes a query queue134configured to store data requests in priority order for transmission to the vehicle140. In some embodiments, the query queue134is integrated into the memory unit132. In some embodiments, the query queue134is separate from the memory unit132. In some embodiments, the query queue134is configured to retrieve data requests from the memory unit132based on priority level and time stamp information. In some embodiments, the query queue134is configured to order data requests based on priority level; and by time since initial saving in the memory unit132in response to data requests having a same priority level.

The communication section130further includes a transmitter135configured to transmit data requests to the vehicle140from the query queue134. The transmitter135is configured to transmit the data requests to the vehicle140based on an order of the data requests in the query queue134. In some embodiments, the data requests are transmitted to the vehicle140wirelessly. In some embodiments, the data requests are transmitted to the vehicle140by a wired connection. The data requests transmitted to the vehicle140include trigger event information, data duration information related to how long before and after the trigger event the data should be collected, and sensor information indicating a type of sensor of the vehicle140should collect the data. In some embodiments, the data requests transmitted to the vehicle140include priority level information. In some embodiments, the transmitter135is configured to transmit data requests to the vehicle140when the vehicle140sends a request to server120to transmit the data requests to the vehicle140. In some embodiments, the transmitter135is configured to transmit data requests to the vehicle140any time the communication section130has sufficient connectivity to the vehicle140to transmit the data request unless the communication section130has received information indicating that the vehicle140is unable to accept a new data request. In some embodiments, the transmitter135is configured to transmit the data requests to the vehicle140periodically so long as the vehicle140is able to receive new data requests and the transmitter135has sufficient connectivity to the vehicle140. In some embodiments, the transmitter135is configured to transmit the data requests to the vehicle140in batches, such as in groups of 5 data requests, 20 data requests or some other number of data requests. In some embodiments, the transmitter135is configured to request confirmation of receipt of the data request from the vehicle140. In response to failing to receive confirmation of receipt from the vehicle for a predetermined time period, the transmitter135is configured to re-transmit the data request. In some embodiments, the status of the data request stored in the memory unit132is updated to indicate submission to the vehicle140in response to the communication section130receiving confirmation of receipt of the data request from the vehicle140.

The communication section130further includes a receiver136configured to receive notification of the occurrence of trigger events from the vehicle140. In some embodiments, the occurrence of a trigger event is receipt of a data request. In some embodiments, the receiver136is configured to receive the notification of the trigger events wirelessly. In some embodiments, the receiver136is configured to receive the notification of the trigger events via a wired connection. In some embodiments, the receiver136is configured to send a signal to the memory unit132to update a status of a data request related to the notified trigger event.

The communication section130further includes a receiver137configured to receive data from the vehicle140responsive to the data requests transmitted by the transmitter135. In some embodiments, the data is split by the vehicle140into data packets that is the unit of transmission from the vehicle140to the server120, and the receiver137receives the data packet from the vehicle140. In some embodiments, the receiver137is configured to receive the data wirelessly. In some embodiments, the receiver137is configured to receive the data via a wired connection. In some embodiments, the receiver137is configured to send a signal to the memory unit132to update a status of a data request related to the receipt of requested data. In some embodiments, the data responsive a single data request is received in a single packet from the vehicle140. In some embodiments, the data responsive to a single data request is received in multiple packets from the vehicle140. The receiver137transfers the received data to a pre-processor122.

The server120further includes the pre-processor122configured to receive data from the receiver137and perform pre-processing on the data to generate collected data. In some embodiments, the pre-processing includes reforming of data from multiple packets to compile data responsive to a data request. In some embodiments, the pre-processing includes de-serializing of data to compile structured data from a byte array that is received. In some embodiments, the pre-processing includes de-compressing of data if the data is compressed by the vehicle140before sending. In some embodiments, the pre-processing includes error correction by Error Correction Code (ECC) such as Reed-Solomon (RS) Code, Bose-Chaudhuri-Hocquenghem (BCH) code, Low-density parity-check (LDPC) code and the like. In some embodiments, the pre-processing includes smoothing of data by removing outlier values to reduce a risk of report incorrect data to the user. In some embodiments, the pre-processing includes associating data request ID information, priority level information or other suitable information with the received data from the receiver137. In some embodiments, the data is pre-processed so that the information is provided to the user in a format that is easy to understand and does not rely on specialized knowledge or equipment to discern the information.

The server120further includes a data storage126configured to store the collected data generated by the data pre-processor122. In some embodiments, the data storage126is integrated with the memory unit132. In some embodiments, the data storage126is separate from the memory unit132. In some embodiments, the data storage126includes a solid state drive (SSE), a random access memory or another suitable memory. In some embodiments, the data storage126is accessible by the user, e.g., using the UI110or an accessible console150. In some embodiments, the data storage126is configured to notify the user in response to data related to a data request is available. In some embodiments, the notification includes an alert to the user. In some embodiments, the alert includes an audio or visual alert. In some embodiments, the data storage126is configured to cause the UI110or the accessible console150to automatically display the notification of an availability of the collected data. In some embodiments, the data storage126is accessible by a user using the accessible console150without the user submitting a data request. In some embodiments, the data within the data storage126are searchable by the user via the accessible console150. In some embodiments, the collected data is visualized in the console150.

The request retrieval system100further includes a vehicle140. The vehicle140includes sensors to detect both an internal status of the vehicle140as well as an external environment surrounding the vehicle140. In some embodiments, the sensors include a camera, a light distance and ranging (LiDAR) sensor, a radio distance and ranging (RADAR) sensor, a sound navigation and ranging (SONAR) sensor, an accelerometer, a steering wheel position, a speedometer, or another suitable sensor. The vehicle140is capable of receiving data requests, either wirelessly or via a wired connection.

In some embodiments, in response to receiving the data request, the vehicle140is configured to assign a data request ID to the received data request and the data request is processed to be agnostic to an originating system or program of the data request. In another embodiments, the communication section130instead of the vehicle140assigns the data request ID, and the data request ID is included in the data request that is sent from the communication section130to the vehicle140. Making the data request agnostic to the originating system or program of the data request helps with expanding an ability of the vehicle140to receive and process a wide range of data requests from different users and systems. The vehicle140includes a processor for processing the data requests and determining what type of information from which sensors available in the vehicle140are capable of satisfying the data request. In at least some embodiments, the vehicle140includes a mobile computing network, which is a network of processors, controllers, or a combination thereof, such as a Controller Area Network (CAN). In at least some embodiments, each processor is an Electronic Control Unit (ECU). The vehicle140further includes a memory for storing data from the sensors. In some embodiments, the processor accesses the memory to determine whether any stored data is capable of satisfying the data request. The vehicle140is further capable of transmitting the data deemed to satisfy the data request to the server120either wirelessly or via a wired connection. In some embodiments, the processor is configured to attempt to satisfy received data requests in a priority order based on a received priority level of the data request. In some embodiments, the vehicle140is configured to transmit data to the server preferentially based on the received priority level of the data request.

In some embodiments, the memory and the processor of the vehicle140are configured to store and execute software applications in an electronic control unit (ECU) within the vehicle140. In some embodiments, a data request is generated by the software application stored in the ECU. In some embodiments, the data request is generated in response to a trigger event, such as sudden acceleration, sudden braking, capturing sensor data including specific objects or specific scenes that are predefined in the software application, “crashing” of the software application, a detected abnormality in the software application, or another suitable detected occurrence. In some embodiments, the vehicle140is configured to generate a notification to a maintainer, e.g., the user, of the software application in response to detecting a trigger event associated with the software application. In some embodiments, the notification is transmitted, either wirelessly or through a wired connection, directly to the user, e.g., through the UI110. In some embodiments, the notification is transmitted, either wirelessly or through a wired connection, to the user through the server120. In some embodiments, the notification includes an audio or visual notification. In some embodiments, the notification is configured to cause the UI110to automatically display the notification without user interaction.

The request retrieval system100further includes an accessible console150. The accessible console150permits the user to access the collected data stored in the data storage126. In some embodiments, the accessible console150is integrated with the UI110. In some embodiments, the accessible console150is separate from the UI110. In some embodiments, the accessible console150includes another server separate from the server120. In some embodiments, the accessible console150automatically receives collected data related to a data request from the user upon receipt of the collected data by the data storage126. In some embodiments, the accessible console150permits the user to search the data storage126to determine whether any of the collected data stored in the data storage126are useful to the user without the user submitting a data request.

Using the request retrieval system100permits users to obtain information from one or more vehicles140in a format that is easy to understand without relying on specialized equipment to request or read the received data. The ability to prioritize data requests in the request retrieval system100help to ensure that law enforcement or other user is able to obtain data, while also permitting users to pay a fee to obtain data faster. This flexibility helps to improve the usefulness of the request retrieval system100for a wide range of users.

FIG.2is views of a graphical user interface (GUI)200and250for request retrieval system in accordance with some embodiments. In some embodiments, the GUI200is usable as UI110in request retrieval system100(FIG.1). In some embodiments, the GUI200is usable to generate a data request for receipt by the receiver131(FIG.1). The GUI200includes a plurality of information types210which identify a type of information that the GUI200is able to accept from the user. The GUI200further includes a plurality of fields220configured to receive information related to a corresponding information type210of the GUI200. The GUI200further includes a submit button230configured to submit a data request to a server, e.g., server120(FIG.1), based on the information in the fields220. One of ordinary skill in the art would recognize that the names and number of the plurality of information types210is merely exemplary and that different numbers and types of information are also within the scope of this disclosure.

In some embodiments, the fields220includes fields for users to enter the vehicle ID, the data type, the start time and the end time. In some embodiments, the field220further includes a field for users to enter a priority level of the data request. In some embodiments, the GUI200further includes information related to how a user is able to increase a priority level of a data request, such as indicating a fee associated with each available priority level. In some embodiments, the GUI200includes fields220for allowing a user to enter log in information to establish an identity of the user. In some embodiments, the GUI200is configured to display a priority level of the user following receiving log in information. In some embodiments, the GUI200further includes fields220for receiving payment information related to fees for establishing a priority level of a data request.

The GUI250is configured to be displayed to the user after the user has selected the submit button230on GUI200. In some embodiments, the GUI250is usable as the GUI110in the ODDR system100(FIG.1). The GUI250includes information indicating that the data request has been received. The GUI250includes a query ID label260and a query ID field270. Information for populating the query ID field270is received from a server, e.g., server120(FIG.1), following the server receiving and storing the data request. In some embodiments, the GUI250includes information of the vehicle ID. In some embodiments, the GUI250includes information related to a priority level of the data request. In some embodiments, the GUI250includes information regarding a status of the data request, such as pending, submitted, completed, etc. In some embodiments, the GUI250includes information related to an estimated time until the data request is submitted to a vehicle, e.g., vehicle140(FIG.1). In at least some embodiments, the GUI250includes information related to an estimated time until the requested data is received. In at least some embodiments, the GUI250includes information related to an estimated energy expenditure to receive the requested data. In some embodiments, the GUI250is displayed automatically in response to receipt of query ID information from the server. In some embodiments, the GUI250is displayed in response to a user submitting a request for an update on an uploaded data request.

FIG.3is a diagram of a data structure300of a request retrieval command310in accordance with some embodiments. In some embodiments, the request retrieval command310is transmitted from the server120to the vehicle140(FIG.1). The request retrieval command310includes information related to a type of data sought by a data request to a vehicle, e.g., vehicle140(FIG.1).

The request retrieval command310includes a transfer priority parameter311that indicates a priority level of the data request. The request retrieval command310further includes a log level parameter312that indicates what type of data, if any, should be retrieved from other applications on the vehicle. For example, in some embodiments, the request retrieval command310retrieves data from an object recognition application. The log level parameter312determines what type of data to retrieve from the other application, such as error level or critical level. In some embodiments, the log level parameter312is omitted from the request retrieval command310or the log level parameter312is left in a null state. The request retrieval command310further includes a time range to be collected parameter313that indicates a time period before and/or after a trigger event to collect data. The time range is corresponding to the start time and the end time that was entered in GUI200(FIG.2) by the users. The request retrieval command310further includes a uniform resource locator (URL) endpoint parameter314that indicates a destination for the data collected in response to the data request. The request retrieval command310further includes a frequency parameter315that indicates how often, if ever, the data should be sampled from the rime range313. For example, when the event time is t=100 sec, the time range comprises start time=−1 sec and end time=2 sec and the frequency is 10 Hz (100 msec cycle), then the data at t=99.0 sec, 99.1 sec, 99.2 sec, . . . , 101.9 sec, 102.0 sec is collected by the request retrieval command. The request retrieval command310further includes a log ID parameter316that indicates types of sensors and/or applications usable to collect the data requested by the data request. In some embodiments, unique IDs (such as Universally unique identifier (UUID)) are pre-assigned to all the sensors and applications, and the unique IDs which the user want to collect data from is specified in the log ID parameter316. The request retrieval command310further includes a requester ID parameter317that indicates an identity of the user that made the data request. The request retrieval command310further includes an event ID parameter318that indicates a trigger event associated with the data request. The request retrieval command310further includes a budget ID parameter319that indicates how much of the resources of the vehicle, e.g., vehicle140(FIG.1), should be allocated to satisfying the data request. One of ordinary skill in the art would understand that additional parameters are possible in the request retrieval command310. For example, in some embodiments, the request retrieval command310includes vehicle location parameter that indicates a geographic area where the trigger event is capable of occurring. One of ordinary skill in the art would also understand that the request retrieval command310does not always include all of the parameters inFIG.3. For example, in some embodiments, the budget ID parameter319is omitted.

FIG.4is a block diagram of a request retrieval system400, in accordance with some embodiments. In some embodiments, the request retrieval system400is part of the request retrieval system100(FIG.1). In some embodiments, the request retrieval system400is usable in conjunction with the request retrieval system100(FIG.1). In some embodiments, the request retrieval system400is separate from the request retrieval system100(FIG.1).

The request retrieval system400includes a detecting vehicle system410configured to capture information about a vehicle or surroundings of the vehicle. The detecting vehicle system110captures information about the vehicle and the surroundings and transmits the information to a server. The request retrieval system400further includes a server440configured to receive the information, encode the information, and disseminate the information to a user terminal460.

The detecting vehicle system410includes an electronic control unit (ECU)420configured to receive data from a sensor414, a global positioning system (GPS)416and a map418. The ECU420includes a situation detector422, a data specifier432, a log collector434and a log transmitter436. The situation detector422includes a vehicle control monitor424, an object detector426, and a scene detector428.

In some embodiments, the ECU420further includes a localization unit configured to receive data from the GPS416and the map418and determine a position of the vehicle and a pose and state of the vehicle relative to detected and/or known objects and/or road position. A pose is an orientation of the vehicle relative to a reference point, such as a roadway. In some embodiments, the position of the vehicle also refers to a position vector of the vehicle. The pose and state of the vehicle refers to a speed and a heading of the vehicle. In some embodiments, the pose and state of the vehicle also refers to a velocity vector, an acceleration vector and jerk vector of the vehicle. In some embodiments, the position vector, the velocity vector, the acceleration vector, and the jerk vector include angle vector. In some embodiments, the state of the vehicle also refers to whether an engine or motor of the vehicle is running.

The sensor414is configured to capture information, such as images, of an environment surrounding the vehicle. In some embodiments, the sensor414includes a visible light camera, an IR camera. In some embodiments, the sensor414is replaced with or is further accompanied by a light detection and ranging (LiDAR) sensor, a radio detection and ranging (RADAR) sensor, a sound navigation and ranging (SONAR) sensor or another suitable sensor. In some embodiments, the sensor414includes additional cameras located at other locations on the vehicle. For example, in some embodiments, additional cameras are located on sides of the vehicle in order to detect a larger portion of the environment to the left and right of the viewing vehicle. Since vehicle occupants are able to look out of side windows of the vehicle, using additional cameras to detect a larger portion of the environment surrounding the vehicle helps to increase precision of detecting objects or scenes surrounding the vehicle. For example, in some embodiments, additional cameras are located on a back side of the vehicle in order to detect a larger portion of the environment to a rear of the vehicle. This information helps to capture information about objects. In some embodiments, the data from the sensor414includes a timestamp or other metadata in order to help synchronize the data from the sensor414with the data from other components.

The GPS416is configured to determine a location of the vehicle. Knowing the location of the viewing vehicle helps to relate an object or scene with determined locations on the map418.

The map418includes information related to the roadway and known objects along the roadway. In some embodiments, the map418is usable in conjunction with the GPS416to determine a location and a heading of the vehicle. In some embodiments, the map418is received from an external device, such as the server440. In some embodiments, the map418is periodically updated based on information from the sensor414and/or the GPS416. In some embodiments, the map418is periodically updated based on information received from the external device. In some embodiments, the map418is generated from sensor data by simultaneous localization and mapping (SLAM) algorithm. Including the map418helps to determine whether an object is a known object. Including the map118having known objects helps to increase precision of new object detection.

The situation detector422is configured to generate information related to performance of the vehicle and of systems within the vehicle. The situation detector422is able to collect information from components within the vehicle, such as the sensor414, braking systems, acceleration system, and other suitable components. Utilizing this information, the situation detector422is able to determine performance of the vehicle. In some embodiments, the situation detector422is further configured to monitor performance of software and networking operations within the vehicle. For example, in some embodiments, the situation detector422is configured to receive information related to “crashing” of software or applications within the vehicle. In some embodiments, the situation detector422is configured to collect information regarding a storage capacity of a memory device within the vehicle. In some embodiments, the situation detector422is configured to receive information related to a processing capability of a processor within the vehicle.

The vehicle control monitor424is configured to receive sensor data and control logs related to current operation of the vehicle. In some embodiments, the sensor data includes information related to vehicle speed, acceleration, jerk, braking, steering, pitching, rolling, yawing, blinking hazard lamp, horn beeping, or other suitable information. The vehicle control monitor424is configured to determine whether any of the received sensor data indicates the satisfaction of a criteria for fulfilling a request, e.g., a trigger event was detected.

The object detector426is configured to receive sensor data from the sensor414to determine whether any abnormal objects are located in the roadway. In some embodiments, the object detector426is further configured to determine whether any objects are present along or adjacent to the roadway. In some embodiments, the sensor data from the sensor414includes an image and the object detector426is configured to perform image recognition on the received image, e.g., using a trained neural network, to identify abnormal objects. In some embodiments, the object detector426is configured to compare any identified objects with information from the GPS416and the map418to help determine a type of an identified object. In some embodiments, the object detector426is configured to identify objects, e.g., a tire, a car part, etc., an animal, a pothole, a traffic regulation board, an emergency vehicle, a vehicle with hazard lights active, or other suitable objects as objects.

The scene detector428is configured to receive the sensor data from the sensor414to determine whether any scenes are located in an environment surrounding the vehicle that satisfy a condition for fulfilling a request. In some embodiments, the scene detector428is configured to determine that a vehicle accident has occurred in response to detecting that two or more vehicles are in contact with one another or that a vehicle is surrounded by multiple fallen objects. In some embodiments, the scene detector428is configured to determine that construction is occurring based on detecting multiple construction vehicles in close proximity. In some embodiments, the scene detector428is configured to determine that a vehicle is parked on a shoulder of the roadway based on determining that a vehicle is located adjacent to the roadway and is not moving or is moving significantly slower than other vehicles. In some embodiments, the scene detector428is configured to use image recognition, such as through a trained neural network, to determine contents of a scene surrounding the vehicle.

In some embodiments, each of the object detector426and the scene detector428are active during an entire period of operation of the vehicle, e.g., when an engine or motor of the vehicle is running. In some embodiments, at least one of the object detector426or the scene detector428is activated in response to the vehicle control monitor424determining that a specific behavior, e.g., trigger event, was detected.

The data specifier432is configured to receive a determination that a fulfillment of a request was performed or that a trigger event was detected. The data specifier432is configured to analyze the received information to determine what sensor data from the sensor414should be collected based on the received data. For example, in some embodiments where an abnormal steering behavior by the driver is detected, the data specifier432is configured to determine that image data from a front camera of the sensor414should be captured. Further, the data specifier432is configured to determine a time period over which the data from the determine sensor should be collected based on a time of the detected situation. In some embodiments, the data specifier432is configured to determine the sensor414from which to collect data based on instructions in a received request from user.

In some embodiments, the data specifier432is configured to determine a region of the received sensor data that is relevant to the detected situation. In some embodiments, the region of the received sensor data is identified based on object recognition performed on the sensor data, e.g., by the object detector426or the scene detector428. In some embodiments, the data specifier432is configured to crop a received image from the sensor data or remove extraneous data from the sensor data if the sensor data is not an image to reduce an amount of information in a log of the abnormal situation. In some embodiments, the data specifier432is configured to remove personal information such as license plate, human faces, etc. from the sensor data.

The log collector434is configured to receive data from the data specifier432. In some embodiments, the log collector434is configured to receive data directly from the sensor414, the GPS416, or the situation detector422based on information provided by the data specifier432. The log collector434is also configured to determine what information is useful for identifying the type and location of the object, such as location information from the GPS416or the map418, image information from the sensor414, cropped or reduced information from the data specifier432, timestamp information related to a time the object or scene was detected, or other suitable information.

The log collector434generates log data based on the received and correlated data, such as the cropped image and location data. The log collector434also associates timestamp information with the log data in order to assist with synchronization of the collected data and for queue priority within the server440. In some embodiments, the log collector434generates the log data to further include world coordinates associated with the cropped image. In some embodiments, the log collector434generates the log data to further include a map location associated with the cropped image. In some embodiments, the log collector434includes additional information to assist in increasing accuracy of determining the object or scene.

While the above description relates to generating log data based on an image from the sensor414, one of ordinary skill in the art would understand that the log collector434is not limited solely to generating log data based on images. In some embodiments, the log collector434is configured to generate log data based on information from other sensors attached to the vehicle, such as RADAR, LiDAR, or other suitable sensors. In some embodiments where the occupant is wearing smart glasses, the log collector434is further configured to generate the log data based on information received from the smart glasses.

The log transmitter436is configured to receive log data from the log collector434and transmit the log data to the server440. In some embodiments, the log transmitter436is configured to transmit the log data wirelessly. In some embodiments, the log transmitter436is configured to transmit the log data via a wired connection. In some embodiments, the log transmitter436is configured to transmit the log data to the user terminal460directly. In some embodiments, the log transmitter436is configured to transmit the log data to a mobile device accessible by the user, which in turn is configured to transmit the log data to the server440. In some embodiments, the log transmitter436is configured to transmit the log data to the mobile device using Bluetooth® or another suitable wireless technology. In some embodiments, the ECU420is configured to determine whether the data transfer rate from the mobile device to the server440is higher than a transfer rate from the log transmitter436to the server440. In response to a determination that the data transfer rate from the mobile device to the sever440is higher, the log transmitter436is configured to transmit the log data to the mobile device to be transmitted to the server440. In response to a determination that the data transfer rate from the mobile device to the server440is not higher, the log transmitter436is configured to transmit the log data to the server440from the vehicle system410directly without transferring the log data to the mobile device.

In some embodiments, the detecting vehicle system410further includes a memory configured to store sensor data from sensors attached to the vehicle. In some embodiments, the memory is further configured to store information associated with previously detected objects or scenes. In some embodiments, in response to detecting an object or scene that matches a previous object or scene, the data specifier434is configured to provide results based on the matching object or scene. In some embodiments, the detecting vehicle system410is further configured to determine whether the detecting vehicle has received from the server440information related to an object or scene that matches the determined object or scene from the situation detector422. In some embodiments, in response to determining that the detecting vehicle has already received information related to the determined object or scene, the detecting vehicle system410is configured to prevent transmission of the log data to the server440. Avoiding transmission of redundant information to the server440helps to reduce data transmitted to the server440and helps to minimize power consumption by the detecting vehicle system410. In some embodiment, the storing of the previous requests is called caching. One of ordinary skill in the art would understand caching as using hardware or software to store data so that future requests for that data are able to be served faster.

The server440includes a log data receiver442configured to receive the log data from the log transmitter436. In some embodiments, the log data receiver442is configured to receive the log data from the mobile device. The server440further includes a log encoder444configured to encode the log data. The server440further includes a log transferer446configured to transmit the encoded log data to the user terminal160. The server440further includes a request/rule receiver448configured to receive a request or a rule from the user terminal460.

The log data receiver442is configured to receive the log data from the log transmitter436. In some embodiments, the log data receiver442is configured to receive the log data from the mobile device. In some embodiments, the log data receiver442is configured to receive the log data wirelessly. In some embodiments, the log data receiver442is configured to receive the log data via a wired connection. In some embodiments, the log data receiver442is configured to attach a timestamp for a time that the log data was received to the log data.

The log encoder444is configured to encode the received log data according to a predetermined encoding protocol. Encoding the log data according to a predetermined encoding protocol helps to ensure that the user terminal460is able to reliably decode the log data for use by the user terminal460. In some embodiments, the log encoder444is configured to perform compression of the log data, image encoding, thumbnail image creation, or other suitable encoding protocols. In some embodiments, the log encoder444is configured to perform encryption of the log data. In some embodiments, the log encoder444is further configured to perform super-resolution to make the data more visible for the user. One of ordinary skill in the art would understand that super-resolution is a process of receiving a high-resolution image from a low-resolution image. Improving the resolution of the log data helps to reduce false positives or false negatives.

In some embodiments, the server440further includes a database for storing received log data. In some embodiments, the log data is stored in the database prior to and/or after encoding by the log encoder444. In some embodiments, the log data is stored in the database in a priority queue. In some embodiments, the priority of the priority queue is determined based on a time that the object or scene, e.g., a trigger event, was detected, a time that the log data was received by the log data receiver442, a type of the object or scene, an identity of the driver of the detecting vehicle, or other suitable priority criteria.

The log transferer446is configured to receive the encoded log data from the log encoder444. The log transferer446is configured to transmit the encoded to the user terminal460. In some embodiments, the log transferer446is configured to transmit the encoded log data to a mobile device accessible by the user. In some embodiments, the log transferer446is configured to transfer the encoded log data wirelessly. In some embodiments, the log transferer446is configured to transmit the encoded log data via a wired connection. In some embodiments, the log transferer446is configured to transmit encoding protocol information along with the encoded log data. Transmitting the encoding protocol information for the encoded log data helps the mobile device or the user terminal460to accurately decode the encoded log data for use by the user terminal460.

The request/rule receiver448is configured to receive new or updated rules or requests for data from a user. In some embodiments, the request/rule receiver448is configured to receive the new or updated rules or requests wirelessly. In some embodiments, the request/rule receiver448is configured to receive the new or updated rules or request via a wired connection. In some embodiments, the request/rule receiver448from the UI110(FIG.1).

In some embodiments, the server440is configured to receive location in formation from multiple vehicles. In some embodiments, the server440is configured to receive navigation plans from multiple vehicles. In some embodiments, the log transferer446is configured to limit the transmission of encoded log data to only vehicles that are within a predetermined distance of the detected trigger event.

In some embodiments, the server440is configured to transmit only log data associated with a newly detected trigger event. That is, if the trigger event has already been reported by the server440, the trigger event is not reported again. Limiting the repetitive reporting of trigger event helps to reduce redundant data received by user terminals to the server440.

The user terminal460is a user terminal accessible by a user associated with a fulfilled request. In some embodiments, the user terminal460includes a GUI. In some embodiments, the user terminal460is configured to automatically generate an alert in response to received data from the server440. In some embodiments, the alert includes an audio or visual alert.

One of ordinary skill in the art would understand that modifications to the request retrieval system400are within the scope of this disclosure. For example, in some embodiments, the detecting vehicle system410is able to transmit log data directly to the user terminal460over a network, such as a wireless network. In some embodiments, a mobile device of an occupant in the detecting vehicle is able to transmit log data directly to the user terminal460, such as a wireless network.

By automatically identifying and disseminating information related to satisfaction of rule or requests detected within the vehicle or in an environment surrounding a vehicle, the user is able to improve performance of applications or software executed using a processing system of the vehicle, e.g., the ECU420. In some embodiments, the user is able to object information related to events such as accidents.

FIG.5is an operational flow for mobile computing network content query and capture in accordance with at least some embodiments of the subject disclosure. The operational flow provides a method of mobile computing network content query and capture. In at least some embodiments, the method is performed by a server, such as server120shown inFIG.1or server440shown inFIG.4. In at least some embodiments, the method is performed by a processor of the server including sections for performing certain operations, such as processor1002shown inFIG.10, which will be explained hereinafter.

At S550, a query receiving section of the processor receives a query. In at least some embodiments, the query receiving section receives, from a client device, a query for a target content of data capturable by a fleet of mobile computing networks, the query including a target content identifier that identifies the target content. In at least some embodiments, the client device is an accessible console, such as accessible console150ofFIG.1, or a user terminal, such as user terminal460ofFIG.4. In at least some embodiments, the query receiving section performs the operational flow shown inFIG.6, which will be explained hereinafter.

At S552, a programming section of the processor programs a task for a model of vehicles in the fleet. In at least some embodiments, the programming section programs a task to capture the target content, the task programmed to be executed by each mobile computing network using available resources of the mobile computing network. In at least some embodiments, the programming section refers to a database of vehicle models to determine the total resources of the model and the currently executed tasks. In at least some embodiments, the programming section programs the task to include instructions for detecting the target content from at least one sensor. In at least some embodiments, the programming section programs the task to consume available resources and avoid interfering with other concurrently executed tasks. In at least some embodiments, the programming section performs the operational flow shown inFIG.7, which will be explained hereinafter.

At S553, the processor determines whether all models have been programmed with a task. In at least some embodiments, the processor refers to a database of vehicle models to determine whether all models have been programmed. In at least some embodiments, a fleet of mobile computing networks includes only one vehicle model. If the processor determines that models remain without a programmed task, then the operational flow returns to model task programming at S552to program a task for another vehicle model. If the processor determines that all models have been programmed with a task, then the operational flow proceeds to task transmission at S555.

At S555, a transmitting section of the processor transmits the tasks to a vehicle. In at least some embodiments, the transmitting section transmits the content retrieval task and sub-tasks to the vehicle. In at least some embodiments, as iterations of the operations at S555proceed, the transmitting section transmits the task to each mobile computing network. In at least some embodiments, the transmitting section performs the operational flow shown inFIG.8, which will be explained hereinafter.

At S556, the processor determines whether tasks have been transmitted to all vehicles. In at least some embodiments, the processor refers to a database of vehicles in the fleet to determine whether tasks have been transmitted to all vehicles. If the processor determines that vehicles remain without a transmitted task, then the operational flow returns to task transmission at S555to transmit a task to another vehicle. If the processor determines that all vehicles have been transmitted a task, then the operational flow proceeds to data reception at S558.

At S558, a data receiving section of the processor receives queried content data. In at least some embodiments, the data receiving section receives data including the target content from each mobile computing network among the fleet of mobile computing networks. In at least some embodiments, the data receiving section receives instances of target content from a vehicle in order by priority according to a content transmission policy. In at least some embodiments, the data receiving section performs the operational flow shown inFIG.9, which will be explained hereinafter.

FIG.6is an operational flow for receiving a content query in accordance with at least some embodiments of the subject disclosure. The operational flow provides a method of receiving a content query. In at least some embodiments, the method is performed by a server, such as server120shown inFIG.1or server440shown inFIG.4. In at least some embodiments, the method is performed by a query receiving section of a processor of the server, such as processor1002shown inFIG.10, which will be explained hereinafter.

At S660, the query receiving section receives an initial query. In at least some embodiments, the query receiving section receives from a client device, a draft query for the target content of data capturable by a fleet of mobile computing networks.

At S662, the query receiving section estimates a delay. In at least some embodiments, the query receiving section estimates a delay until a target quantity of the target content is received. In at least some embodiments, the query includes the target quantity of the target content. In at least some embodiments, the query receiving section estimates the delay based on, for each mobile computing network, a likelihood of the mobile computing network encountering the target content, a bandwidth of a connection to the mobile computing network, and a number of available resources of the mobile computing network.

At S664, the query receiving section estimates an energy expenditure. In at least some embodiments, the query receiving section estimates an energy expenditure by the fleet of mobile computing networks to capture and transmit the target quantity of the target content. In at least some embodiments, the query receiving section estimates the energy expenditure based on, for each mobile computing network, a complexity of the task, and a type of a connection to the mobile computing network. In at least some embodiments, the query receiving section estimates the energy expenditure of not only the mobile computing network, but also the vehicle, the service provider of the connection, and any other entities expending energy in the process, especially those entities that charge a fee for energy expenditure. In at least some embodiments, the energy expenditure represents the cost of the capture and transmission of the target quantity of the target content.

At S666, the query receiving section transmits the estimates to the client device. In at least some embodiments, the query receiving section transmits the estimated delay and estimated energy expenditure to the client device.

At S668, the query receiving section determines whether the estimates are acceptable. In at least some embodiments, the query receiving section requests confirmation after transmitting the estimated delay and estimated energy expenditure. If the query receiving section determines that the estimates are not acceptable, then the operational flow receives a modified query (S669) and returns to delay estimation at S662. If the query receiving section determines that the estimates are acceptable, then the operational flow ends.

FIG.7is an operational flow for programming a content query task in accordance with at least some embodiments of the subject disclosure. The operational flow provides a method of programming a content query task. In at least some embodiments, the method is performed by a server, such as server120shown inFIG.1or server440shown inFIG.4. In at least some embodiments, the method is performed by a programming section of a processor of the server, such as processor1002shown inFIG.10, which will be explained hereinafter.

At S770, the programming section determines whether the vehicle model includes a built-in detection function. In at least some embodiments, the programming section determines each mobile computing network among the fleet of mobile computing networks having a built-in function for directly detecting the target content. If the programming section determines that the vehicle model does not include a built-in detection function, then the operational flow proceeds to detection sub-task programming at S771. If the programming section determines that the vehicle model does include a built-in detection function, then the operational flow proceeds to retention policy adjustment at S773.

At S771, the programming section programs a detection sub-task. In at least some embodiments, the programming section programs, for each mobile computing network that does not have the built-in function, a detection sub-task within the task to detect the target content. In at least some embodiments, the programming section programs an image recognition algorithm for recognizing the target content from an image sensor.

At S773, the programming section adjusts a retention policy. In at least some embodiments, the query includes a priority value of the target content. In at least some embodiments, the programming section adjusts a retention policy of each mobile computing network based on the priority value relative to another priority value of a concurrently executed task. In at least some embodiments, the retention policy includes a probability of purging each instance of captured target content, the probability of purging inversely related to the priority value. In at least some embodiments, the probability of purging is a function of age and priority value.

At S775, the programming section determines whether the retention policy includes filtering. In at least some embodiments, the retention policy includes filtering as an alternative to purging. In at least some embodiments, the retention policy includes a probability of filtering each instance of captured target content, the probability of filtering being inversely related to the priority value and greater than the probability of purging. In at least some embodiments, the probability of filtering is a function of age and priority value. If the programming section determines that the retention policy includes filtering, then the operational flow proceeds to filtering sub-task programming at S776. If the programming section determines that the retention policy does not include filtering, then the operational flow proceeds to compression determination at S778.

At S776, the programming section programs a filtering sub-task. In at least some embodiments, the programming section programs a filtering sub-task within the task to filter each instance of captured target content, the filtering sub-task programmed to be executed by each mobile computing network using available resources of the mobile computing network.

At S778, the programming section determines whether the retention policy includes compression. In at least some embodiments, the retention policy includes compression as an alternative to purging. In at least some embodiments, the retention policy includes a probability of compressing each instance of captured target content, the probability of compressing being inversely related to the priority value and greater than the probability of purging. In at least some embodiments, the probability of compressing is a function of age and priority value. If the programming section determines that the retention policy includes compression, then the operational flow proceeds to compression sub-task programming at S776. If the programming section determines that the retention policy does not include compression, then the operational flow ends.

At S779, the programming section programs a compression sub-task. In at least some embodiments, the programming section programs a compression sub-task within the task to compress each instance of captured target content, the compression sub-task programmed to be executed by each mobile computing network using available resources of the mobile computing network.

FIG.8is an operational flow for transmitting a content query in accordance with at least some embodiments of the subject disclosure. The operational flow provides a method of transmitting a content query. In at least some embodiments, the method is performed by a server, such as server120shown inFIG.1or server440shown inFIG.4. In at least some embodiments, the method is performed by a transmitting section of a processor of the server, such as processor1002shown inFIG.10, which will be explained hereinafter.

At S880, the transmitting section transmits the task. In at least some embodiments, the transmitting section transmits the task to the respective mobile computing network. In at least some embodiments, as iterations of S555proceed, the transmitting section transmits the task to each mobile computing network.

At S882, the transmitting section transmits a retention policy. In at least some embodiments, the transmitting includes transmitting the task and the adjusted retention policy to the respective mobile computing network. In at least some embodiments, as iterations of S555proceed, the transmitting section transmits the task and the adjusted retention policy to each mobile computing network.

At S884, the transmitting section determines whether the bandwidth of a connection between the mobile computing network and the server is higher than a threshold value. In at least some embodiments, the transmitting section transmits bigger data payloads, such as the sub-tasks, only over a high bandwidth connection. If the transmitting section determines that the bandwidth of the connection between the mobile computing network and the server is higher than a threshold value, then the operational flow proceeds to sub-task transmission at S886. If the transmitting section determines that the bandwidth of the connection between the mobile computing network and the server is not higher than a threshold value, then the operational flow returns bandwidth determination at S884.

At S886, the transmitting section transmits sub-tasks to the vehicle. In at least some embodiments, the transmitting section transmits the detection sub-task to each mobile computing network in response to detecting that a connection to the mobile computing network has a bandwidth that is higher than a threshold bandwidth value. In at least some embodiments, the transmitting section transmits the filtering sub-task to each mobile computing network in response to detecting that a connection to the mobile computing network has a bandwidth that is higher than a threshold bandwidth value. In at least some embodiments, the transmitting section transmits the compression sub-task to each mobile computing network in response to detecting that a connection to the mobile computing network has a bandwidth that is higher than a threshold bandwidth value. In at least some embodiments, the transmitting section stops transmission in response to the bandwidth falling below the threshold value, and resumes transmission in response to the bandwidth exceeding the threshold value.

FIG.9is an operational flow for receiving queried content in accordance with at least some embodiments of the subject disclosure. The operational flow provides a method of receiving queried content. In at least some embodiments, the method is performed by a server, such as server120shown inFIG.1or server440shown inFIG.4. In at least some embodiments, the method is performed by a data receiving section of a processor of the server, such as processor1002shown inFIG.10, which will be explained hereinafter.

At S990, the data receiving section receives metadata. In at least some embodiments, the data receiving section receives a metadata of each instance of captured target content. In at least some embodiments, the metadata provides descriptions or keywords explaining the respective instance of captured target content.

At S993, the data receiving section transmits a content transmission policy. In at least some embodiments, the data receiving section transmits, from the server in response to receiving the metadata of each instance of captured target content, a content transmission policy including a transmission priority value corresponding to each instance of captured target content.

At S996, the data receiving section receives an instance of captured target content. In at least some embodiments, the data receiving section receives a first instance of captured target content among each instance of captured target content in an order based on the transmission priority value. In at least some embodiments, the transmission priority value is based on a transmission priority function of the content transmission policy, the transmission priority function based on a size of the first instance, an age of the first instance, and the bandwidth of the connection to the server

At S999, the data receiving section determines whether all captured target content has been received. If the data receiving section determines that captured target content remains to be received, then the operational flow returns to S996. If the data receiving section determines that all captured target content has been received, then the operational flow ends.

FIG.10is a diagram of a system1000for implementing a request retrieval system in accordance with some embodiments. System1000includes a hardware processor1002and a non-transitory, computer readable storage medium1004encoded with, i.e., storing, the computer program code1006, i.e., a set of executable instructions. Computer readable storage medium1004is also encoded with instructions1007for interfacing with external devices. The processor1002is electrically coupled to the computer readable storage medium1004via a bus1008. The processor1002is also electrically coupled to an I/O interface1010by bus1008. A network interface1012is also electrically connected to the processor1002via bus1008. Network interface1012is connected to a network1014, so that processor1002and computer readable storage medium1004are capable of connecting to external elements via network1014. The processor1002is configured to execute the computer program code1006encoded in the computer readable storage medium1004in order to cause system1000to be usable for performing a portion or all of the operations as described in ODDR system100(FIG.1), ODDR system400(FIG.4) or method600(FIG.6).

In some embodiments, the processor1002is a central processing unit (CPU), a multi-processor, a distributed processing system, an application specific integrated circuit (ASIC), and/or a suitable processing unit.

In some embodiments, the computer readable storage medium1004is an electronic, magnetic, optical, electromagnetic, infrared, and/or a semiconductor system (or apparatus or device). For example, the computer readable storage medium1004includes a semiconductor or solid-state memory, a magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk, and/or an optical disk. In some embodiments using optical disks, the computer readable storage medium1004includes a compact disk-read only memory (CD-ROM), a compact disk-read/write (CD-R/W), and/or a digital video disc (DVD).

In some embodiments, the storage medium1004stores the computer program code1006configured to cause system1000to perform a portion or all of the operations as described in ODDR system100(FIG.1), ODDR system400(FIG.4) or method600(FIG.6). In some embodiments, the storage medium1004also stores information needed for performing a portion or all of the operations as described in ODDR system100(FIG.1), ODDR system400(FIG.4) or method600(FIG.6) as well as information generated during performing a portion or all of the operations as described in ODDR system100(FIG.1), ODDR system400(FIG.4) or method600(FIG.6), such as a priority level parameter1016, a query ID parameter1018, a query status parameter1020, a query data parameter1022and/or a set of executable instructions to perform a portion or all of the operations as described in ODDR system100(FIG.1), ODDR system400(FIG.4) or method600(FIG.6).

In some embodiments, the storage medium1004stores instructions1007for interfacing with manufacturing machines. The instructions1007enable processor1002to generate manufacturing instructions readable by the manufacturing machines to effectively implement method400during a manufacturing process.

System1000includes I/O interface1010. I/O interface1010is coupled to external circuitry. In some embodiments, I/O interface1010includes a keyboard, keypad, mouse, trackball, trackpad, and/or cursor direction keys for communicating information and commands to processor1002.

System1000also includes network interface1012coupled to the processor1002. Network interface1012allows system1000to communicate with network1014, to which one or more other computer systems are connected. Network interface1012includes wireless network interfaces such as BLUETOOTH, WIFI, WIMAX, GPRS, or WCDMA; or wired network interface such as ETHERNET, USB, or IEEE-1394. In some embodiments, a portion or all of the operations as described in ODDR system100(FIG.1), ODDR system400(FIG.4) or method600(FIG.6) is implemented in two or more systems1000, and information such as priority level, query ID, query status and query data are exchanged between different systems1000via network1014.

Mobile computing network programming for queried content capture is performed by receiving, from a client device, a query for a target content of data capturable by a fleet of mobile computing networks, the query including a target content identifier that identifies the target content, programming a task to capture the target content, the task programmed to be executed by each mobile computing network using available resources of the mobile computing network, transmitting the task to each mobile computing network; and receiving data including the target content from each mobile computing network among the fleet of mobile computing networks.

The foregoing outlines features of several embodiments so that those skilled in the art may better understand the aspects of the present disclosure. Those skilled in the art should appreciate that they may readily use the present disclosure as a basis for designing or modifying other processes and structures for carrying out the same purposes and/or achieving the same advantages of the embodiments introduced herein. Those skilled in the art should also realize that such equivalent constructions do not depart from the spirit and scope of the present disclosure, and that they may make various changes, substitutions, and alterations herein without departing from the spirit and scope of the present disclosure.