Patent Description:
In the last several decades, the use of electronic devices has become common. In particular, advances in electronic technology have reduced the cost of increasingly complex and useful electronic devices. Cost reduction and consumer demand have proliferated the use of electronic devices such that they are practically ubiquitous in modern society. As the use of electronic devices has expanded, so has the demand for new and improved features of electronic devices. More specifically, electronic devices that perform new functions, perform richer functions, and/or that perform functions faster, more efficiently, and/or more reliably are often sought after.

Advances in technology have resulted in smaller and more powerful electronic devices. For example, there currently exist a variety of electronic devices such as cellular phones, smartphones, vehicle in-dash electronics, laptop computers, servers, etc..

Users often value convenience, flexibility, and quick service. However, some offerings may be limited. As can be observed from this discussion, improving user experience and/or convenience may be beneficial.

<CIT> discusses a system and method for pattern identification of a learned image, or learned pattern, in a target image. <CIT> discusses an augmented reality based application for use on a smartphone to allow a user to easily identify a rental vehicle for a particular rental by scanning a parking area.

In accordance with the present invention, there is provided a method performed by an electronic device for determining an inventory of auxiliary products in the interior of a vehicle as set out in claim <NUM>, and an electronic device for determining an inventory of auxiliary products in the interior of a vehicle as set out in claim <NUM>. Other aspects of the.

A method performed by an electronic device is described. The method includes receiving, by a processor, an auxiliary product request in conjunction with a vehicle request. The method also includes requesting, by the processor, image data of a scene of an interior of one or more vehicles from one or more image sensors; receiving said image data from the one or more image sensors; performing, by the processor, computer vision analysis on the image data, computer vision analysis comprising performing object recognition and object classification to classify one or more objects within the scene of the interior of the one or more vehicles; determining, by the processor, an auxiliary product inventory for each of the one or more vehicles based on the computer vision analysis, the auxiliary product inventory comprising objects within the scene of the interior of an associated vehicle; determining, by the processor, that one or more auxiliary product inventory comprises at least one object that satisfies the auxiliary product request ; and dispatching, by the processor, one of the one or more vehicles to fulfill the vehicle request wherein the one vehicle is associated with the determined auxiliary product inventory. invention can be found in the dependent claims. Any embodiment referred to and not falling within the scope of the claims is merely an example useful to the understanding of the invention.

The systems and methods disclosed herein may relate to customizing vehicle amenities. For example, some configurations of the systems and methods disclosed herein may enable mobile productivity by providing approaches for customizing classify one or more objects within a scene of the interior of the at least one vehicle. The apparatus additionally includes means for determining, based on the classification, whether the one or more objects satisfy the auxiliary product request or the auxiliary service request within the interior of the at least one vehicle.

The systems and methods disclosed herein may relate to customizing vehicle amenities. For example, some configurations of the systems and methods disclosed herein may enable mobile productivity by providing approaches for customizing auxiliary product and/or auxiliary service options in shared vehicles. The term "auxiliary" as used herein may mean "non-primary. " For example, auxiliary products and/or auxiliary services in the context of vehicle service may exclude the class of vehicle, brand/provider, and/or destination. For instance, an auxiliary service may not include a primary service of taking a passenger from point A to point B in a car of X type by Y provider. Instead, auxiliary products and/or auxiliary services may include variable features within a type of vehicle that can optionally be consumed in conjunction with the primary service.

Many people eat, drink, and/or work in their cars. Whether because of a late start to their commute or long road trip, people often consume beverages, snacks, and/or entire meals while driving or riding in their cars in addition to working. In a world where cars, autonomous, semi-autonomous, or manually driven, are shared and summoned by people, providing food service and/or other productivity services to drivers and/or passengers is a capability that may involve logistical, monetary, hygienic, and/or privacy challenges.

Cars may be summoned that provide food (e.g., a food delivery service, pizza delivery, etc.) or that provide transportation (e.g., an Uber vehicle, taxi, etc.), but not both. Some premium vehicles may provide a small number of predetermined offerings like peanuts and water bottles (e.g., the same offerings in each vehicle). Moreover, there may be no options for summoning cars with productivity equipment like printers or Wi-Fi. While it may be possible to hire a car for transportation and then have the car take the passengers to a location to acquire food or to make copies, for example, this takes time and may involve traveling far off the shortest distance route to the passenger's ultimate destination. There may be no large scale service that provides both transportation and personalized (e.g., on demand) food and/or productivity service.

Some configurations of the systems and methods disclosed herein may provide a framework for providing customized (e.g., on demand) food and/or productivity service in a shared vehicle that addresses the logistical challenges involved in this type of service. For example, the framework may include made-to-order food, beverage service, and/or productivity service within the vehicle. Some configurations of the systems and methods disclosed herein may be utilized in entertainment and productivity applications. Some configurations of the systems and methods disclosed herein may be implemented in land-based vehicles (e.g., automobiles, cars, trucks, non-aircraft, etc.). Other configurations may be implemented in other kinds of vehicles (e.g., aircraft).

In some approaches, one or more vehicles (e.g., cars) may be equipped with food and beverage options and/or or other preparation equipment with a variety of food and beverage options stocked ahead of time. For example, the preparation equipment, a stock of ingredients (e.g., food and/or beverage items), and/or supplies may be provided in each vehicle. The food and/or beverage options may be automatically monitored and/or periodically restocked. It should be noted that stocking is not limitless due to a defined volume/cargo space in each vehicle. Accordingly, no provider may be able to accommodate all possible requests. It should also be noted that in some approaches, autonomous vehicles may be leased out during the day by owners on an individual basis and may be differently stocked than large fleets. In any case, earlier passengers may deplete an entire stock of a common ingredient (e.g., bread), which may result in a significant variance between available vehicles at any given moment.

The preparation equipment may scale from an automatic coffee machine to a full service kitchen. Other productivity equipment may range from a color copier to a business center. The equipment may be operated by the passenger or a dedicated other person or robot within the vehicle.

Some configurations of the systems and methods disclosed herein may utilize computer vision in order to monitor items (e.g., food, beverages, other items, etc.) for inventory control. For example, by mounting one or more cameras in the cabin and/or cargo areas, computer vision may be utilized to determine the contents without the need for additional sensors and/or systems that track inventory or consumption. Accordingly, computer vision may provide a powerful and/or flexible solution for monitoring.

Vehicles (e.g., cars) may be selected based on a passenger's desire to have made-to-order food and/or beverages, and/or to perform certain tasks along their trip. When a passenger summons a vehicle, their food, beverage, and/or productivity preferences may be input as well as their destination. When the dispatch algorithm pairs a vehicle with a passenger, the decision on which vehicle to send may be made based not only on the location-based information, but also on the quantity of the food, beverages, and/or preparation equipment that match the passenger's preferences. Vehicles may be selected based on individual products (e.g., a kind of soda), individual brands (e.g., beverage brands, restaurant names, etc.) or based on a type of cuisine (e.g., vegetarian, Italian, Chinese, Mexican, Indian, etc.). Payment could be in the form of a subscription fee or per-item based on consumption in the car.

For example, a passenger may summon a "bar car" on their way from the office to make themselves a cocktail on their way home. Or, a family may summon a "sandwich car" so they could make sandwiches for themselves on a trip to the beach because they did not have time to make them before leaving the house. Or, a user may need to print out some flyers on their way to a meeting and may summon a "business center car. " In another example, a user may be late for work and may not have time for breakfast and may summon a "coffee car.

In some configurations, the systems and methods disclosed herein may also apply to arbitrary customizations. For example, the framework may apply to vehicle customization in addition to food and/or productivity services. For instance, vehicles with certain types of seats (leather or cloth), scents (through air fresheners), configurations (seats facing each other rather that facing forward), etc., may be selected in accordance with the framework.

Various configurations are now described with reference to the Figures, where like reference numbers may indicate functionally similar elements. The systems and methods as generally described and illustrated in the Figures herein could be arranged and designed in a wide variety of different configurations. Thus, the following more detailed description of several configurations, as represented in the Figures, is not intended to limit scope, as claimed, but is merely representative of the systems and methods.

<FIG> is a block diagram illustrating one example of an electronic device <NUM> in which systems and methods for customizing amenities in shared vehicles is implemented. Examples of the electronic device <NUM> include cellular phones, smart phones, computers (e.g., desktop computers, laptop computers, etc.), servers, tablet devices, media players, televisions, vehicles (e.g., cars, trucks, aircraft, motorcycles, etc.), automobiles, cameras, video camcorders, digital cameras, personal cameras, action cameras, surveillance cameras, mounted cameras, connected cameras, robots, aircraft, gaming consoles, personal digital assistants (PDAs), set-top boxes, etc. The electronic device <NUM> may include one or more components or elements. One or more of the components or elements may be implemented in hardware (e.g., circuitry) or a combination of hardware and software (e.g., a processor with instructions).

In some configurations, the electronic device <NUM> may include a processor <NUM>, a memory <NUM>, a display <NUM>, one or more image sensors <NUM>, one or more optical systems <NUM>, and/or a communication interface <NUM>. The processor <NUM> may be coupled to (e.g., in electronic communication with) the memory <NUM>, display <NUM>, image sensor(s) <NUM>, optical system(s) <NUM>, and/or communication interface <NUM>. It should be noted that one or more of the elements illustrated in <FIG> may be optional. In particular, the electronic device <NUM> may not include one or more of the elements illustrated in <FIG> in some configurations. For example, the electronic device <NUM> may or may not include an image sensor <NUM> and/or optical system <NUM>. Additionally or alternatively, the electronic device <NUM> may or may not include a display <NUM>.

In some configurations, the electronic device <NUM> may present a user interface <NUM> on the display <NUM>. For example, the user interface <NUM> may enable a user to interact with the electronic device <NUM>. In some configurations, the display <NUM> may be a touchscreen that receives input from physical touch (by a finger, stylus, or other tool, for example). Additionally or alternatively, the electronic device <NUM> may include or be coupled to another input interface. For example, the electronic device <NUM> may include a camera facing a user and may detect user gestures (e.g., hand gestures, arm gestures, eye tracking, eyelid blink, etc.). In another example, the electronic device <NUM> may be coupled to a mouse and may detect a mouse click. In yet another example, the electronic device <NUM> may be coupled to a keyboard and may receive key inputs from the keyboard. In some configurations, one or more of the images described herein may be presented on the display <NUM> and/or user interface <NUM>.

The communication interface <NUM> may enable the electronic device <NUM> to communicate with one or more other electronic devices. For example, the communication interface <NUM> may provide an interface for wired and/or wireless communications. In some implementations, the electronic device <NUM> may communicate directly or indirectly (via a network, for instance) with a server device, with a client device, and/or with a vehicle. In some configurations, the communication interface <NUM> may be coupled to one or more antennas <NUM> for transmitting and/or receiving radio frequency (RF) signals. Additionally or alternatively, the communication interface <NUM> may enable one or more kinds of wireline (e.g., Universal Serial Bus (USB), Ethernet, etc.) communication.

In some configurations, multiple communication interfaces <NUM> may be implemented and/or utilized. For example, one communication interface <NUM> may be a cellular (e.g., <NUM>, Long Term Evolution (LTE), CDMA, <NUM>, etc.) communication interface <NUM>, another communication interface <NUM> may be an Ethernet interface, another communication interface <NUM> may be a universal serial bus (USB) interface, and yet another communication interface <NUM> may be a wireless local area network (WLAN) interface (e.g., Institute of Electrical and Electronics Engineers (IEEE) <NUM> interface).

The electronic device <NUM> (e.g., image obtainer <NUM>) obtains one or more images (e.g., digital images, image frames, frames, video, etc.). The one or more images (e.g., frames) are images of a scene (e.g., a vehicle interior, one or more objects, and/or background). For example, the electronic device <NUM> may include one or more image sensors <NUM> and one or more optical systems <NUM> (e.g., lenses). An optical system <NUM> may focus images of objects that are located within the field of view of the optical system <NUM> onto an image sensor <NUM>. The optical system(s) <NUM> may be coupled to and/or controlled by the processor <NUM> in some configurations.

A camera may include at least one image sensor and at least one optical system. Accordingly, the electronic device <NUM> may be one or more cameras, may include one or more cameras, and/or may be coupled to one or more cameras in some implementations. The image sensor(s) <NUM> capture the one or more images (e.g., image frames, video, still images, burst mode images, stereoscopic images, etc.). In some implementations, the electronic device <NUM> may include multiple optical system(s) <NUM> and/or multiple image sensors <NUM>.

Additionally or alternatively, the electronic device <NUM> may request and/or receive the one or more images from another device (e.g., one or more external image sensors coupled to the electronic device <NUM>, a network server, traffic camera, drop camera, vehicle camera, web camera, security camera, remote camera, etc.). In some configurations, the electronic device <NUM> may request and/or receive the one or more images via the communication interface <NUM>. For example, the electronic device <NUM> may or may not include a camera (e.g., an image sensor <NUM> and/or optical system <NUM>) and may receive images from one or more remote devices.

The memory <NUM> stores instructions and/or data. The processor 112accesses (e.g., read from and/or write to) the memory <NUM>. Examples of instructions and/or data that is stored by the memory <NUM> includes request information <NUM>, product data <NUM> (e.g., auxiliary product data), service data <NUM> (e.g., auxiliary service data), vehicle information <NUM>, image obtainer <NUM> instructions, data obtainer <NUM> instructions, computer vision analyzer <NUM> instructions, vehicle matcher <NUM> instructions, request obtainer <NUM> instructions, inventory determiner <NUM> instructions, and/or instructions for other elements, etc..

In some configurations, the electronic device <NUM> (e.g., the memory <NUM>) may include an image data buffer (not shown). The image data buffer may buffer (e.g., store) image data (e.g., image frame(s)) from the image sensor <NUM>. The buffered image data may be provided to the processor <NUM>.

In some configurations, the electronic device <NUM> may include a camera software application and/or a display <NUM>. When the camera application is running, images of scenes and/or objects that are located within the field of view of the optical system <NUM> may be captured by the image sensor(s) <NUM>. The images that are being captured by the image sensor(s) <NUM> may be presented on the display <NUM>. In some configurations, these images may be displayed in rapid succession at a relatively high frame rate so that, at any given moment in time, the objects that are located within the field of view of the optical system <NUM> are presented on the display <NUM>. The one or more images obtained by the electronic device <NUM> may be one or more video frames and/or one or more still images.

The processor <NUM> includes and/or implements a data obtainer <NUM>, a vehicle matcher <NUM>, a request obtainer <NUM>, an image obtainer <NUM>, a computer vision analyzer <NUM>, and/or an inventory determiner <NUM>. Additionally or alternatively, one or more of the elements illustrated in the processor <NUM> may be implemented separately from the processor <NUM> (e.g., in other circuitry, on another processor, on a separate electronic device, etc.).

The processor <NUM> may include and/or implement a request obtainer <NUM>. The request obtainer <NUM> may obtain a vehicle request. For example, the request obtainer <NUM> may obtain information regarding a request for a shared vehicle (e.g., a vehicle for hire, a taxi, an autonomous driving service vehicle, a rental vehicle, etc.). A vehicle request may include a pickup location. The pickup location may specify a requested (e.g., prospective) pickup location of a user. A vehicle request may include a destination. The destination may specify a requested (e.g., prospective) destination of a user. For instance, the destination may indicate one or more places where a user wants to go. In some configurations, the vehicle request may include one or more time constraints. A time constraint may indicate an amount of time that a user is willing to wait (e.g., a maximum time limit) for a ride and/or a required time (e.g., time window) for pick up and/or drop off. In some configurations, the vehicle request may indicate whether the user wants a vehicle with a driver or a driverless (e.g., autonomous) vehicle. In some configurations, the vehicle request may specify one or more conventional vehicle options in addition to an auxiliary product request and/or an auxiliary service request. For example, the vehicle request may specify a vehicle type (e.g., sedan, sub-compact, sport-utility vehicle (SUV), minivan, van, truck, etc.), number of seats, cruise control, power windows, power locks, radio, compact disc (CD) player, digital video disc (DVD) player, backup camera, sunroof, leather seats, cloth seats, etc..

In some cases and/or configurations, the vehicle request may include an auxiliary product request. The auxiliary product request may indicate one or more products requested by a user. Examples of auxiliary products may include one or more of food (e.g., food items), food preparation equipment, beverages, office supplies (e.g., writing utensils, pens, pencils, paper, paperclips, staples, binders, etc.), computers (e.g., laptop computers, netbooks, tablets, etc.), removable storage (e.g., thumb drive, external hard drive, etc.), printers, copiers, reading material (e.g., newspapers, magazines, books, etc.), still cameras, video cameras, clothing, etc. Additionally or alternatively, the auxiliary product request may specify a particular quantity of one or more auxiliary products, a quality level for one or more auxiliary products, and/or one or more other aspects (e.g., price, brand, etc.). In some configurations, auxiliary products may not include one or more conventional vehicle items (e.g., some kinds of vehicle-integrated electronics, seats, climate control unit(s) (e.g., air conditioning, heating, etc.), vehicle adjustment settings (e.g., seat adjustment settings, rear-view mirror settings, steering wheel adjustments, etc.), etc.). In some implementations, the auxiliary product(s) and/or the auxiliary service(s) may increase user productivity (by allowing one or more users to accomplish more tasks while traveling, for example). In some configurations, the term "auxiliary" may mean that a product or service is not integrated into (e.g., built into) a vehicle. Additionally or alternatively, the term "auxiliary" may mean that a product or service is not a conventional product or service for a shared vehicle in some configurations.

In some cases and/or configurations, the vehicle request may include an auxiliary service request. The auxiliary service request may indicate one or more services requested by a user. Examples of auxiliary services may include one or more of food preparation, beverage preparation (e.g., drink mixing), office services (e.g., copying, printing, lamination, document preparation, etc.), massage, etc. Additionally or alternatively, the auxiliary service request may specify a particular quantity of one or more auxiliary services, a quality level for one or more auxiliary services, and/or one or more other aspects (e.g., price, provider, etc.). In some configurations, auxiliary services may not include one or more conventional vehicle services (e.g., driving service, air conditioning, heating, etc.).

In some configurations, the request obtainer <NUM> may obtain the vehicle request from the user interface <NUM>. For example, the user interface <NUM> may receive one or more inputs (e.g., keyboard inputs, touch inputs, voice inputs, gesture inputs, etc.) that indicate the vehicle request. Additionally or alternatively, the request obtainer <NUM> may obtain the vehicle request from a remote device. For example, the electronic device <NUM> may receive the vehicle request via the communication interface(s) <NUM>. For instance, the communication interface(s) <NUM> may receive a wireline and/or wireless signal indicating the vehicle request. The vehicle request may be stored as request information <NUM> in the memory <NUM>.

The processor <NUM> includes and/or implement a data obtainer <NUM>. The data obtainer <NUM> obtains auxiliary product data and/or auxiliary service data associated with one or more vehicles. The auxiliary product data may be stored as all or part of the product data <NUM> in the memory <NUM>. The auxiliary service data may be stored as all or part of the service data <NUM> in the memory <NUM>. In some configurations, the data obtainer <NUM> may obtain other data associated with one or more vehicles. For example, the data obtainer <NUM> may obtain one or more vehicle locations, vehicle availability for the one or more vehicles, and/or one or more aspects (e.g., conventional aspects) of a vehicle. For example, the data obtainer <NUM> may request and/or receive vehicle location(s), vehicle availability, and/or vehicle aspect(s) from one or more remote devices (e.g., vehicles, electronic devices, servers, etc.). Additionally or alternatively, one or more of the vehicle location(s), vehicle availability, and/or vehicle aspect(s) may be obtained locally and/or may be predetermined. For example, the vehicle information <NUM> may include a set (e.g., database) of information for one or more vehicles. Additionally or alternatively, the electronic device <NUM> may include and/or may be in communication with a locating means (e.g., global positioning system (GPS), Wi-Fi-based locating device (e.g., triangulation), inertial navigation, etc.) for locating the electronic device <NUM> (e.g., vehicle).

Auxiliary product data indicates information about one or more auxiliary products in one or more vehicles. For example, the auxiliary product data indicates one or more auxiliary product types, and may indicate one or more auxiliary product quantities, and/or one or more auxiliary product qualities. Additionally or alternatively, the auxiliary product data may indicate one or more features of one or more auxiliary products and/or one or more auxiliary product aspects (e.g., price, brand, etc.). The auxiliary product data is stored in a database (e.g., auxiliary product database, auxiliary product inventory database, etc.).

Auxiliary service data may indicate information about one or more auxiliary services in one or more vehicles. For example, the auxiliary service data may indicate one or more auxiliary service types, auxiliary service availability (e.g., quantity), and/or one or more auxiliary service qualities. Additionally or alternatively, the auxiliary service data may indicate one or more features of one or more auxiliary services and/or one or more auxiliary service aspects (e.g., price, provider, etc.). In some configurations, the auxiliary service data may be stored in a database (e.g., auxiliary service database).

In some configurations, the data obtainer <NUM> includes an image obtainer <NUM> and/or a computer vision analyzer. For example, obtaining the auxiliary product data and/or the auxiliary service data is based on image data from an interior (e.g., cabin, cargo holding space, trunk, and/or compartment) of at least one vehicle and/or based on computer vision analysis of the image data. For instance, the electronic device <NUM> obtains one or more images (e.g., still images, burst images, video, etc.) from one or more vehicles and perform computer vision analysis on the image(s) to determine auxiliary product data and/or the auxiliary service data. Additionally or alternatively, a remote device may obtain one or more images and/or may perform computer vision analysis to determine auxiliary product data and/or the auxiliary service data, which the electronic device <NUM> may request and/or receive. More detail regarding obtaining image(s) and/or performing computer vision analysis is given as follows, which is performed by the electronic device <NUM> and/or one or more remote devices.

The processor <NUM> (e.g., data obtainer <NUM>) includes and/or implement an image obtainer <NUM>. One or more images (e.g., image frames, video, burst shots, etc.) are provided to the image obtainer <NUM>. For example, the image obtainer <NUM> obtains image frames from one or more image sensors <NUM>. For instance, the image obtainer <NUM> receives image data from one or more image sensors <NUM> and/or from one or more external cameras. As described above, the image(s) may be captured from the image sensor(s) <NUM> included in the electronic device <NUM> or may be captured from one or more remote camera(s).

The image obtainer <NUM> requests and/or receives image data (e.g., one or more images, image frames, etc.). For example, the image obtainer <NUM> requests and/or receive one or more images from a remote device (e.g., external camera(s), remote server, remote electronic device, vehicle, etc.) via the communication interface <NUM>. The images obtained from the cameras are utilized by the electronic device <NUM> for computer vision analysis (and/or determining auxiliary product data and/or the auxiliary service data).

The processor <NUM> includes and/or implement a computer vision analyzer <NUM>. The computer vision analyzer <NUM> performs computer vision analysis on the image data (e.g., one or more images). In particular, the computer vision analyzer <NUM> performs object recognition, object tracking, object classification, optical character recognition, scene understanding, etc. The computer vision analysis is utilized to determine the auxiliary product data and/or the auxiliary service data. For example, the computer vision analysis is utilized to determine a product inventory associated with one or more vehicles.

In some configurations, a camera has a view of a vehicle interior (e.g., of one or more stocking locations for one or more products, of a cabin, of one or more cargo holding spaces, of a trunk, and/or of one or more compartments, etc.). The camera captures image data (e.g., one or more images of the vehicle interior). The image data is provided to the image obtainer <NUM>. For example, the electronic device <NUM> may include a camera (e.g., image sensor <NUM> and optical system <NUM>) that may capture the image data and provide the image data to the image obtainer <NUM>. In another example, a remote camera in a vehicle captures image data and/or sends the image data to the electronic device <NUM> (e.g., image obtainer <NUM>). The image obtainer <NUM> provides the image data to the computer vision analyzer <NUM>.

The computer vision analyzer performs computer vision analysis on the image data. For example, the computer vision analyzer performs object detection, object recognition, object tracking, object classification, and/or optical character recognition, etc., on the image data.

In some configurations, the computer vision analyzer <NUM> performs object detection to produce all or part of the auxiliary product data. For example, the computer vision analyzer <NUM> detects one or more objects in the image data (e.g., in one or more images, in one or more subsets of the image(s), etc.). Object detection isbased on a detection model. For example, motion may be used to separate moving objects from a background. Additionally or alternatively, the image data (e.g., one or more images and/or one or more subsets of one or more images) is searched for one or more objects using a detection model. For instance, one or more subsets (e.g., windows, sliding window positions, etc.) of the image data are compared with the detection model to determine whether an object represented by the detection model is in the image data. For example, the computer vision analyzer <NUM> may correlate the image data with the detection model, may determine a similarity between the image data and the detection model, and/or may determine a probability of a match between the image data and the detection model, etc. Object detection may be utilized to classify one or more auxiliary products in a vehicle, determine a number of auxiliary products in the vehicle, determine whether an auxiliary product is still present in the vehicle, determine whether the auxiliary product has been taken from the vehicle, etc..

In some approaches, the computer vision analyzer <NUM> recognizes one or more objects (e.g., objects, characters, faces, etc.) to produce all or part of the auxiliary product data. For example, the computer vision analyzer <NUM> determines recognition data (e.g., one or more keypoints, corners, edges, and/or feature vectors, etc.) from the image data. For instance, the recognition data may represent one or more aspects of one or more objects. In some approaches, the computer vision analyzer <NUM> may search the image data for keypoints, corners, edges, etc..

The computer vision analyzer <NUM> has access to one or more recognition models corresponding to one or more objects. For example, the memory <NUM> stores the one or more recognition models. Examples of the recognition model(s) includes images, keypoint sets, point clouds, point models (e.g., deformable models, non-deformable models), mesh models, feature vectors, etc..

The computer vision analyzer <NUM> compares the recognition data (e.g., keypoints, corners, edges, and/or feature vectors, etc.) with the recognition model(s) to determine whether the recognition data matches one or more recognition models. For example, the computer vision analyzer <NUM> may correlate the recognition data with the recognition model(s), may determine one or more distance measures between the recognition data and the recognition model(s), may determine a transformation between the recognition data and the recognition model(s), may determine a matching probability between the recognition data with the recognition model(s), etc. Comparing the recognition data with the recognition model(s) may produce one or more matching metrics, which may indicate a degree of similarity (and/or difference) between the recognition data and the recognition model(s). A match may be indicated by the matching metric(s). For example, a matching metric that meets a threshold (e.g., similarity threshold, probability threshold, distance threshold, recognition threshold, etc.) may indicate that the object in the image is the kind of object corresponding to the recognition model(s). Object recognition may be utilized to classify one or more objects, to determine what auxiliary product is being used, to determine what auxiliary product is being consumed, what auxiliary product has been taken from the vehicle, etc..

It should be noted that object recognition may be more specific than object detection in some configurations. For example, object detection may detect that an object is indicated in image data, while object recognition may recognize the object. In some approaches, object detection may indicate the presence of an object, while object recognition may be used to classify an object (e.g., identify an object type). In other approaches, object detection may be used to classify an object (e.g., identify an object type), while object recognition may be used to recognize one or more characteristics of the object. For example, object detection may detect an aluminum beverage can, while object recognition may determine a brand of beverage, whether the can is open, the expiration date of the beverage, etc..

Additionally or alternatively, the computer vision analyzer <NUM> may perform object tracking. For example, the computer vision analyzer <NUM> may track an object over two or more frames. In some approaches, object tracking may be performed by searching a subsequent frame for an object detected and/or tracked in a previous frame. For instance, the computer vision analyzer <NUM> may track keypoints, corners, edges, and/or feature vectors of an object between frames. Optical flow is one example of object tracking that may be performed. Object tracking may be utilized to determine whether an auxiliary product is being used, is being consumed, has been taken from the vehicle, etc. Object tracking may additionally or alternatively be utilized to determine an amount of time an object has been in a vehicle. For example, some items (e.g., perishable goods, food, produce, beverages, etc.) may degrade over time. Object tracking may be utilized to monitor an amount of time that an object has been in a vehicle in order to rotate and/or replace inventory as needed. For example, one or more items may expire and/or lose freshness after a period of time. If an expiration and/or freshness threshold is reached for a particular item, the item may be removed, discarded, discounted, and/or replaced, etc..

In some configurations, the computer vision analyzer <NUM> may perform analysis to produce computer analysis data, such as bounding boxes around individual items, segmentation of individual items, and/or one or more derived descriptions of an item's identity and/or location, etc. For example, an object's center may be derived from a bounding box or segmentation representation. The computer analysis data may be utilized to determine an inventory. For instance, item location data may be utilized to determine a number of items (e.g., whether any items have been removed and/or added). Additionally or alternatively, item identity data may be utilized to track item (e.g., product) type in an inventory. For instance, the inventory determiner <NUM> may compare a detected item identity data to any item types currently in the inventory. If the item type is not already in the inventory, the inventory determiner <NUM> may add the item type. If the item type is in the inventory, the inventory determiner <NUM> may add to the number of that existing item type (in the case that an item is added).

In some configurations, computer vision may be utilized for monitoring a quantity of supplies and/or ingredients as follows. One or more cameras may be mounted in a vehicle with a view of food and/or supply storage locations. Each time an auxiliary product (e.g., item, food, supply, ingredient, etc.) is removed or added, the camera(s) may be used to detect objects through object detection, object classification, or a mix of both. Other attributes of the object, such as expiration date, quantity in a bottle, etc., may also be ascertained through computer vision. For example, if the computer vision analysis indicates a passed expiration date or a partially consumed beverage, the electronic device <NUM> may discard the corresponding auxiliary product and/or may alert a person (e.g., driver, fleet service worker, etc.) to remove the expired or consumed item. Computer vision may provide a flexible approach for determining auxiliary product data and/or auxiliary service data. For example, computer vision may not require specialized storage containers for the auxiliary products. Computer vision may also enable flexibly adding and/or changing auxiliary products and/or auxiliary services for detection, recognition, tracking, and/or classification.

The computer vision analyzer <NUM> may perform object detection, object tracking, object recognition, and/or object classification to determine auxiliary service data in some configurations. For example, the computer vision analyzer <NUM> may detect, track, recognize, and/or classify an auxiliary service provider (e.g., a person, a worker, a robot, etc.). For instance, the computer vision analyzer <NUM> may perform facial recognition on a worker and/or may detect and/or recognize a uniform of an auxiliary service provider.

Additional or alternative approaches may be implemented to determine auxiliary product data and/or auxiliary service data in some configurations. For example, one or more optical and/or contact solutions may be implemented. For instance, dedicated optical (e.g., near-infrared LED) and/or mechanical (e.g., pressure) sensors may be located under and/or near each individual item. The sensors may measure presence or absence of an object. A stocker may ensure that the correct items are in each location. Dedicated optical and/or mechanical sensors may be implemented in addition to or alternatively from cameras with computer vision.

Additionally or alternatively, one or more manual scanning solutions may be implemented. For example, each auxiliary product (e.g., item) may have a barcode or other uniquely identifying symbol. Each time the auxiliary product is added or removed, the code/symbol may be scanned by a dedicated code reader. For instance, a user may manually scan (like at a grocery store checkout) the auxiliary product. Additionally or alternatively, one or more scanners may be fixed scanners so that scanning is part of the action of removing an auxiliary product.

The processor <NUM> (e.g., data obtainer <NUM>) may include and/or implement an inventory determiner <NUM> in some configurations. The inventory determiner <NUM> may determine an auxiliary product inventory of one or more vehicles based on the auxiliary product data. The auxiliary product inventory may indicate one or more auxiliary product types, one or more auxiliary product quantities, and/or one or more auxiliary product qualities.

In some configurations, the inventory determiner <NUM> determines the auxiliary product inventory of one or more vehicles based on the computer vision analysis. In some approaches, the computer vision analysis may indicate a quantity of one or more types of auxiliary products. For example, the computer vision analysis indicates detected objects (e.g., auxiliary products) and/or recognized objects (e.g., auxiliary products). The inventory determiner <NUM> may interpret and/or organize the computer vision analysis to produce the auxiliary product inventory. For example, the inventory determiner <NUM> may count a number of detected auxiliary products and/or may sort the detected auxiliary products into product types. For instance, the inventory determiner <NUM> may total a number of auxiliary products for each auxiliary product type. Additionally or alternatively, the inventory determiner <NUM> may add to the quantity of an auxiliary product when the computer vision analyzer <NUM> detects a new instance of that auxiliary product (when stocking, for example). Or, the inventory determiner <NUM> may subtract from the quantity of an auxiliary product when the computer vision analyzer <NUM> detects that an auxiliary product has been taken and/or consumed. The auxiliary product inventory may be stored as all or part of the product data <NUM> in the memory <NUM>.

In some configurations, the electronic device <NUM> may manage charging a user for any requested and/or consumed auxiliary product and/or auxiliary service. For example, if the electronic device <NUM> detects that a user has removed and/or consumed an auxiliary product and/or an auxiliary service (based on computer vision or otherwise, for example), the electronic device <NUM> may charge the user for the auxiliary product and/or auxiliary service. For instance, the electronic device <NUM> may generate and/or present a bill for the auxiliary product. Additionally or alternatively, the electronic device <NUM> may charge a user account (e.g., credit card, debit account, etc.) for the auxiliary product and/or auxiliary service. In some approaches, the user may be charged for obtaining a requested auxiliary product and/or a requested auxiliary service. For example, if the vehicle obtains a perishable food item for the user at the user's request, the electronic device <NUM> may charge the user for the perishable food item, whether or not the user consumes the perishable food item.

The auxiliary product data and/or the auxiliary service data may be dynamic. For example, one or more auxiliary product types may be changeable. More specifically, the type(s) of one or more auxiliary products in a vehicle may change. In one example, a vehicle may be requested with hamburgers as the auxiliary product for one trip. In another trip, the vehicle may be requested with tacos as the auxiliary product for the trip. In some configurations, the stock of a vehicle may not be from a predetermined menu of items.

In some configurations, the electronic device <NUM> may obtain and/or store vehicle information <NUM>. The vehicle information <NUM> may indicate one or more aspects (e.g., conventional aspects) of a vehicle. Examples of vehicle information <NUM> may include vehicle type (e.g., sedan, sub-compact, sport-utility vehicle (SUV), minivan, van, truck, etc.), number of seats, cruise control, power windows, power locks, radio, compact disc (CD) player, digital video disc (DVD) player, backup camera, sunroof, leather seats, cloth seats, etc. The vehicle information <NUM> may indicate the aspects of one or more (e.g., a set of, a fleet of) vehicles. In some configurations, the vehicle information <NUM> may indicate the location of a vehicle and/or a vehicle status (e.g., available or unavailable). For example, the electronic device <NUM> may request and/or receive information regarding the one or more vehicles.

The processor <NUM> may include and/or implement a vehicle matcher <NUM>. The vehicle matcher <NUM> may determine whether one or more vehicles satisfy the vehicle request. For example, the vehicle matcher <NUM> may determine whether one or more vehicles have an auxiliary product inventory and/or a service availability to satisfy the auxiliary product request and/or the auxiliary service request. In some cases and/or configurations, if the auxiliary product inventory of a vehicle satisfies the auxiliary product request, the vehicle matcher <NUM> may indicate a match and/or assign (and/or dispatch) the vehicle to satisfy the vehicle request. In some cases and/or configurations, if the auxiliary service data of a vehicle satisfies the auxiliary service request, the vehicle matcher <NUM> may indicate a match and/or assign (and/or dispatch) the vehicle to satisfy the vehicle request. As described in relation to <FIG>, it should be noted that the electronic device <NUM> may be implemented in a client device, in a vehicle, and/or in a cloud device (e.g., one or more remote devices, servers, network devices, etc.). For example, determining whether an auxiliary product inventory would satisfy an auxiliary product request and/or determining whether auxiliary product data would satisfy an auxiliary service request may be performed by a client device, by a vehicle, and/or by a cloud device. Additionally or alternatively, indicating a match and/or assigning a vehicle to fulfill a request may be performed by a client device, by a vehicle, and/or by a cloud device.

In some configurations, the vehicle matcher <NUM> may utilize multiple criteria to determine whether a vehicle is a match and/or whether to assign (and/or dispatch) the vehicle to fulfill the vehicle request. For example, the vehicle matcher <NUM> may determine whether the aspects of one or more vehicles match the vehicle request, whether an auxiliary product request would be satisfied based on the auxiliary product data associated with the one or more vehicles, whether an auxiliary service request would be satisfied based on the auxiliary service data associated with the one or more vehicles, and/or whether the one or more vehicles can meet a time constraint (for user location pick up and/or for destination arrival, for instance). In some approaches, the vehicle matcher <NUM> may determine an amount of time to pick up a user at the pickup location based on the pickup location of the vehicle request and the location(s) of one or more vehicles.

If all of the criteria would be satisfied for one or more vehicles, the vehicle matcher <NUM> may indicate the one or more matches and/or may assign (and/or dispatch) a vehicle to fulfill the vehicle request. In some configurations, if multiple vehicles would satisfy the vehicle request, the vehicle matcher <NUM> may rank (e.g., prioritize) the vehicles. The vehicle matcher <NUM> may rank the vehicles in some approaches based on auxiliary product data (e.g., auxiliary product inventory), auxiliary product request, destination, vehicle location, pickup location, time constraint, auxiliary service data, and/or auxiliary service request. For example, the vehicle matcher <NUM> may calculate (based on the vehicle location, pickup location, and/or the destination, for instance) an amount of travel time for pick-up and travel. Vehicles that would take longer and/or would not meet the time constraint may be ranked lower and/or eliminated. In some approaches, the vehicle that is nearest to the pickup location and/or that can most quickly arrive at the pickup location may be ranked highest and/or assigned. In some approaches, ranking the vehicles may be based on the auxiliary product inventory. For example, the vehicle that has the most stock of the one or more requested auxiliary products may be ranked higher than other vehicles with less stock. Additionally or alternatively, ranking the vehicles may be based on the auxiliary service availability. For example, the vehicle that has the one or more requested auxiliary services may be ranked higher than other vehicles.

If not all of the criteria would be satisfied for any vehicle (within a geographical range, for example), the vehicle matcher <NUM> may indicate (e.g., offer, provide, etc.) one or more other options. For example, the vehicle matcher <NUM> may indicate one or more options. The options may mostly match the criteria of the vehicle request. For example, the vehicle matcher <NUM> may provide options that satisfy the auxiliary product request and/or auxiliary service request, but with a longer wait time than could satisfy the time constraint. In another example, the vehicle matcher <NUM> may provide options that satisfy the time constraint but that do not satisfy all of the auxiliary product request(s) and/or auxiliary service request(s). In some approaches, potential vehicles may be limited based on geographical area (e.g., within a city, county, state, distance from the pickup location, distance from the drop-off location, etc.). In some configurations, the electronic device <NUM> may assign a vehicle to obtain one or more auxiliary products and/or one or more auxiliary services to fulfill the vehicle request. For example, the electronic device <NUM> may indicate an option to send a vehicle to obtain the auxiliary product(s) and/or the auxiliary service(s). The option may be presented contingent on an additional fee in some configurations. For example, a vehicle may be assigned and/or dispatched to obtain one or more requested auxiliary products and/or one or more requested auxiliary services if a user assents to paying a fee for the additional travel of the vehicle.

Assigning a vehicle may include indicating the assignment to a vehicle, a driver, and/or a remote device (e.g., a computer of a fleet manager). For example, the electronic device <NUM> may present an assignment on the display <NUM> (e.g., user interface <NUM>), may send the assignment to a device (e.g., smartphone) of a driver, and/or may send the assignment to a remote device. In some configurations, assigning the vehicle may also include initiating a payment procedure. For example, the electronic device <NUM> may present a message (on the display <NUM>) and/or send a message to a remote device (e.g., a user's smartphone) requesting acceptance of an agreement (e.g., contract, terms and conditions, etc.) to pay in exchange for the vehicle transportation. Additionally or alternatively, the electronic device <NUM> may request payment for the vehicle transportation (upon assignment and/or upon completion of the ride).

Dispatching a vehicle may include commanding and/or instructing a vehicle and/or a driver (via a remote device such as a smartphone) to fulfill the vehicle request. For example, the electronic device <NUM> may send one or more commands and/or instructions to an autonomous vehicle and/or to a vehicle driver (via a remote device) to fulfill the vehicle request. Commands and/or instructions may include driving directions, departure time, pickup time, pickup location, destination, one or more time constraints, etc..

It should be noted that one or more of the elements or components of the electronic device <NUM> may be combined and/or divided. For example, one or more of the request obtainer <NUM>, the data obtainer <NUM>, the vehicle matcher <NUM>, the image obtainer <NUM>, the computer vision analyzer <NUM>, and/or the inventory determiner <NUM> may be combined. Additionally or alternatively, one or more of the request obtainer <NUM>, the data obtainer <NUM>, the vehicle matcher <NUM>, the image obtainer <NUM>, the computer vision analyzer <NUM>, and/or the inventory determiner <NUM> may be divided into elements or components that perform a subset of the operations thereof.

<FIG> is a flow diagram illustrating one configuration of a method <NUM> for customizing amenities in shared vehicles. The method <NUM> may be performed by the electronic device <NUM>, for example. In some configurations, the method <NUM> may be utilized to match (e.g., pair) a vehicle with a vehicle request (e.g., user).

The electronic device <NUM> may obtain <NUM> a vehicle request with an auxiliary product request and/or an auxiliary service request. This may be accomplished as described above in connection with <FIG>. For example, the electronic device <NUM> may obtain an auxiliary product request and/or an auxiliary service request from a user interface and/or from a remote device.

The electronic device <NUM> obtains <NUM> auxiliary product data and/or auxiliary service data associated with at least one vehicle. This may be accomplished as described above in connection with <FIG>. For example, the electronic device <NUM> performs computer vision analysis on one or more images to determine auxiliary product data (e.g., quantity and/or type of one or more auxiliary products in one or more vehicles) and/or auxiliary service data and/or may receive the auxiliary product data and/or auxiliary service data from one or more remote devices (e.g., electronic devices, vehicles, scanners, smartphones, etc.). Additionally or alternatively, the electronic device <NUM> may utilize dedicated optical and/or mechanical sensors to detect auxiliary product data (e.g., quantity and/or type of one or more auxiliary products in one or more vehicles). Additionally or alternatively, the electronic device <NUM> may utilize one or more scanners (e.g., manual scanners, fixed scanners, etc.) to obtain the auxiliary product data.

The electronic device <NUM> may determine <NUM> whether the auxiliary product data and/or auxiliary service data for the at least one vehicle satisfies the auxiliary product request and/or the auxiliary service request. This may be accomplished as described in connection with <FIG>. For example, the electronic device <NUM> may determine whether one or more types and/or one or more quantities of auxiliary products (e.g., auxiliary product inventory) in one or more vehicles meets or exceeds the type(s) and/or quantity(ies) of the auxiliary product request. Additionally or alternatively, the electronic device <NUM> may determine whether one or more types of auxiliary services in one or more vehicles are available to satisfy the auxiliary service request. In some configurations, one or more additional or alternative criteria (e.g., time constraint, distance, conventional vehicle options, etc.) may be taken into account when attempting to match a vehicle to a vehicle request and/or to rank vehicles.

In some configurations, the electronic device <NUM> may assign and/or dispatch a vehicle based on the determination <NUM>. For example, the electronic device <NUM> may assign and/or dispatch a vehicle with one or more auxiliary products and/or auxiliary services that satisfy the auxiliary product request and/or the auxiliary service request.

<FIG> is a diagram illustrating an example of a client device <NUM>, a server device <NUM>, and a vehicle <NUM>. Some configurations of the systems and methods disclosed herein may be implemented in the client device <NUM>, the server device <NUM>, and/or the vehicle. For example, the electronic device <NUM> described in connection with <FIG> may be implemented in the client device <NUM>, the server device <NUM>, or the vehicle <NUM>. Additionally or alternatively, one or more of the functions, operations, procedures, blocks, components, elements, etc., described in connection with one or more of <FIG> and <FIG> may be distributed between the client device <NUM>, the server device <NUM>, and/or the vehicle <NUM>. The client device <NUM>, the server device <NUM>, and/or the vehicle <NUM> may communicate directly (with one or more wired and/or wireless links) and/or may communicate via a network (e.g., wired and/or wirelessly with one or more intervening network devices (e.g., routers, switches, hubs, servers, gateways, access points, etc.).

In one example, the electronic device <NUM> described in connection with <FIG> may be implemented as the client device <NUM> described in connection with <FIG>. For instance, the client device <NUM> may send a vehicle request to the server device <NUM>. The server device <NUM> may obtain one or more images, auxiliary product data, and/or auxiliary service data associated with the vehicle <NUM>, which the server device <NUM> may provide to the client device <NUM>. In an approach where one or more images are provided, the client device <NUM> may perform computer vision analysis on the image data to determine the auxiliary product data and/or the auxiliary service data, and/or whether the vehicle <NUM> would satisfy the vehicle request (e.g., auxiliary product request, auxiliary service request, etc.). In an approach where the auxiliary product data and/or the auxiliary service data are provided, the client device <NUM> may determine whether the auxiliary product data and/or the auxiliary service data would satisfy the vehicle request. The client device <NUM> may assign and/or dispatch the vehicle <NUM> (via the server device <NUM>, for example) if the auxiliary product data and/or the auxiliary service data would satisfy the vehicle request.

In another example, the electronic device <NUM> described in connection with <FIG> may be implemented as the server device <NUM> described in connection with <FIG>. For instance, the server device <NUM> may receive a vehicle request from the client device <NUM>. The server device <NUM> may obtain one or more images, auxiliary product data, and/or auxiliary service data associated with the vehicle <NUM>. In some approaches, the server device <NUM> may provide (e.g., send, transmit, etc.) the image(s), auxiliary product data, and/or auxiliary service data to the client device <NUM>. In another approach, the server device <NUM> may perform computer vision analysis on the image data to determine the auxiliary product data and/or the auxiliary service data, and/or whether the vehicle <NUM> would satisfy the vehicle request (e.g., auxiliary product request, auxiliary service request, etc.). In an approach where the auxiliary product data and/or the auxiliary service data are obtained directly from the vehicle <NUM>, the server device <NUM> and/or the client device <NUM> may determine whether the auxiliary product data and/or the auxiliary service data would satisfy the vehicle request. The server device <NUM> or the client device <NUM> may assign and/or dispatch the vehicle <NUM> if the auxiliary product data and/or the auxiliary service data would satisfy the vehicle request.

In yet another example, the electronic device <NUM> described in connection with <FIG> may be implemented as the vehicle <NUM> described in connection with <FIG>. For instance, the vehicle <NUM> may receive a vehicle request from the client device <NUM> and/or the server device <NUM>. The vehicle <NUM> may obtain one or more images, auxiliary product data, and/or auxiliary service data associated with the vehicle <NUM>. In some approaches, the vehicle <NUM> may provide (e.g., send, transmit, etc.) the image(s), auxiliary product data, and/or auxiliary service data to the server device <NUM> and/or client device <NUM> (via the server device <NUM>, for instance). In another approach, the vehicle <NUM> may perform computer vision analysis on the image data to determine the auxiliary product data and/or the auxiliary service data, and/or whether the vehicle <NUM> would satisfy the vehicle request (e.g., auxiliary product request, auxiliary service request, etc.). In an approach where the auxiliary product data and/or the auxiliary service data are determined by the vehicle <NUM>, the vehicle <NUM>, the server device <NUM>, and/or the client device <NUM> may determine whether the auxiliary product data and/or the auxiliary service data would satisfy the vehicle request. The vehicle, <NUM>, the server device <NUM>, or the client device <NUM> may assign and/or dispatch the vehicle <NUM> if the auxiliary product data and/or the auxiliary service data would satisfy the vehicle request.

<FIG> is a flow diagram illustrating one configuration of a method <NUM> for determining a product inventory based on computer vision analysis. The method <NUM> is performed by the electronic device <NUM>, for example.

The electronic device <NUM> obtains <NUM> image data from an interior of at least one vehicle. This may be accomplished as described above in connection with <FIG>. For example, the electronic device <NUM> captures the image data with one or more cameras or receives the image data from a remote device (e.g., a vehicle).

The electronic device <NUM> perform <NUM> computer vision analysis on the image data. This may be accomplished as described above in connection with <FIG>. For example, the electronic device <NUM> may perform object detection, object recognition, object tracking, and/or object classification on the image data.

The electronic device <NUM> determines <NUM> an auxiliary product inventory of the at least one vehicle based on the computer vision analysis. This may be accomplished as described in connection with <FIG>. For example, the electronic device <NUM> determines auxiliary product data based on the computer vision analysis. In some configurations, the auxiliary product inventory may indicate a quantity of one or more food items.

In some approaches, the electronic device <NUM> may determine a number of auxiliary products based on object detection and may determine one or more types of auxiliary products based on object recognition. The electronic device <NUM> may count a number of detected auxiliary products and/or may sort the detected auxiliary products according to product type. The auxiliary product inventory may indicate a quantity (e.g., number) of one or more types of auxiliary products. In some configurations, the auxiliary product inventory may be stored in memory (e.g., the memory <NUM> as part of the product data <NUM>) and/or may be sent to a remote device. For example, the auxiliary product inventory may be stored in a database (e.g., a supply and/or ingredient inventory database). Additionally or alternatively, the auxiliary product inventory may be presented on a display (e.g., display <NUM> and/or user interface <NUM>).

In some configurations, the electronic device <NUM> may assign and/or dispatch a vehicle based on the auxiliary product inventory. For example, the electronic device <NUM> may assign and/or dispatch a vehicle with one or more auxiliary products that satisfy the auxiliary product request and/or the auxiliary service request.

<FIG> is a flow diagram illustrating a more specific configuration of a method <NUM> for customizing amenities in shared vehicles. The method <NUM> may be performed by the electronic device <NUM>, for example.

The electronic device <NUM> may obtain <NUM> a vehicle request with an auxiliary product request and/or an auxiliary service request. This may be accomplished as described above in connection with one or more of <FIG>.

The electronic device <NUM> may obtain <NUM> auxiliary product data and/or auxiliary service data associated with one or more vehicles. This may be accomplished as described above in connection with one or more of <FIG>.

The electronic device <NUM> may determine <NUM> whether the auxiliary product data and/or auxiliary service data for the at least one vehicle satisfies the auxiliary product request and/or the auxiliary service request. This may be accomplished as described in connection with one or more of <FIG>. For example, the electronic device <NUM> may determine whether one or more types and/or one or more quantities of auxiliary products (e.g., auxiliary product inventory) in one or more vehicles meets or exceeds the type(s) and/or quantity(ies) of the auxiliary product request. Additionally or alternatively, the electronic device <NUM> may determine whether one or more types of auxiliary services in one or more vehicles are available to satisfy the auxiliary service request. In some configurations, one or more additional or alternative criteria (e.g., time constraint(s), distance, conventional vehicle options, etc.) may be taken into account.

If the auxiliary product data and/or auxiliary service data satisfies the auxiliary product request and/or auxiliary service request for one or more vehicles, the electronic device <NUM> may assign <NUM> (and/or dispatch) a vehicle to fulfill the vehicle request. For example, the electronic device <NUM> may assign and/or dispatch a vehicle with one or more auxiliary products and/or auxiliary services that satisfy the auxiliary product request and/or the auxiliary service request.

If the auxiliary product data and/or auxiliary service data does not satisfy the auxiliary product request and/or auxiliary service request for one or more vehicles, the electronic device <NUM> may indicate <NUM> (e.g., provide) one or more different options. This may be accomplished as described in connection with <FIG>. For example, the electronic device <NUM> may indicate one or more options that partially satisfy (e.g., match) the criteria. For instance, the one or more options may include one or more vehicles with longer wait times (than the time constraint, for example) and/or that fully or partially satisfy the auxiliary product request and/or auxiliary service request, etc. In some approaches, the options may be ranked (e.g., prioritized) based on the degree to which the criteria would be satisfied. The options may be presented on a display (e.g., user interface) and/or may be sent for presentation to a user (e.g., sent to a user's smartphone).

The electronic device <NUM> may determine <NUM> whether to modify the vehicle request. For example, the electronic device <NUM> may determine whether to modify the auxiliary product request and/or the auxiliary service request. Additionally or alternatively, the electronic device <NUM> may determine whether to modify one or more time constraints and/or other criteria of the vehicle request.

In some approaches, the electronic device <NUM> may receive a selection (based on an input received from a user, for example) of the one or more indicated options. If a selection is received, the electronic device <NUM> may modify the vehicle request in accordance with the selected option. In some approaches, the electronic device <NUM> may return to determine <NUM> whether the modified vehicle request satisfies the auxiliary product request and/or auxiliary service request for one or more vehicles. In other approaches, the electronic device <NUM> may directly assign <NUM> the vehicle corresponding to the selected option. If it is determined <NUM> not to modify the vehicle request, operation may end <NUM>.

In an additional or alternative approach, if there is no vehicle available that would satisfy the vehicle request, the electronic device <NUM> may assign (and/or dispatch) a vehicle to obtain the requested auxiliary product(s) and/or auxiliary service(s). For example, the electronic device <NUM> may assign a vehicle to go and obtain fast food in accordance with an auxiliary product request (if obtaining the auxiliary product request can be completed within one or more time constraints and/or geographical constraints, for example). In some configurations, the user may be charged an additional fee for obtaining the requested auxiliary product(s) and/or auxiliary service(s) (if the user accepts the additional charge in advance, for example).

<FIG> is a flow diagram illustrating another more specific configuration of a method <NUM> for customizing amenities in shared vehicles. The method <NUM> may be performed by the electronic device <NUM>, for example.

The electronic device <NUM> obtains <NUM> image data from an interior of at least one vehicle. This may be accomplished as described above in connection with one or more of <FIG>.

The electronic device <NUM> performs <NUM> computer vision analysis on the image data. This may be accomplished as described in connection with one or more of <FIG>.

The electronic device <NUM> determines <NUM> at least one auxiliary product inventory of the at least one vehicle based on the computer vision analysis. This may be accomplished as described in connection with one or more of <FIG> and <FIG>. For example, the electronic device <NUM> produces the auxiliary product inventory according to auxiliary product data as indicated by the computer vision analysis. In some cases and/or configurations, an auxiliary product inventory may be determined <NUM> for each vehicle.

The electronic device <NUM> may determine <NUM> whether an auxiliary product inventory (associated with each vehicle, for example) satisfies the auxiliary product request. This may be accomplished as described in connection with one or more of <FIG>. For example, the electronic device <NUM> may determine whether one or more types and/or one or more quantities of auxiliary products (e.g., auxiliary product inventory) in one or more vehicles meets or exceeds the type(s) and/or quantity(ies) of the auxiliary product request. In some configurations, one or more additional or alternative criteria (e.g., time constraint(s), distance, conventional vehicle options, etc.) may be taken into account.

If an auxiliary product inventory satisfies the auxiliary product request (for one or more vehicles, for example), the electronic device <NUM> may assign <NUM> (and/or dispatch) a vehicle to fulfill the vehicle request. This may be accomplished as described in connection with one or more of <FIG>. For example, the electronic device <NUM> may assign and/or dispatch a vehicle with one or more auxiliary products that satisfy the auxiliary product request.

If an auxiliary product inventory does not satisfy the auxiliary product request for one or more vehicles, the electronic device <NUM> may assign <NUM> a vehicle to obtain at least one product for the auxiliary product request. This may be accomplished as described in connection with one or more of <FIG> and <FIG>. In another approach, the electronic device <NUM> may present one or more different options.

<FIG> is a flow diagram illustrating yet another more specific configuration of a method <NUM> for customizing amenities in shared vehicles. The method <NUM> may be performed by the electronic device <NUM>, for example.

The electronic device <NUM> may obtain <NUM> a vehicle request with an auxiliary product request, an auxiliary service request, a pickup location, a destination, and/or a time constraint. This may be accomplished as described above in connection with one or more of <FIG> and <FIG>. The auxiliary product request may indicate a request for one or more auxiliary products. The auxiliary service request may indicate a request for one or more auxiliary services.

The electronic device <NUM> may obtain <NUM> vehicle availability data. This may be accomplished as described above in connection with <FIG>. For example, the electronic device <NUM> may request and/or receive vehicle availability data that indicates whether one or more vehicles are available or unavailable. A vehicle may be unavailable for one or more reasons. For example, the vehicle may be off-duty, may be currently serving another user, and/or may be indicated as unavailable by a driver.

The electronic device <NUM> may obtain <NUM> a quantity of one or more auxiliary products and/or an availability of one or more auxiliary services associated with one or more vehicles (e.g., of one or more available vehicles). This may be accomplished as described above in connection with one or more of <FIG>. For example, the auxiliary product data, auxiliary service data, computer vision analysis, auxiliary product inventory, etc., may indicate the quantity of the auxiliary product(s) and/or the availability of the auxiliary service(s).

The electronic device <NUM> may determine <NUM> whether all of the criteria (e.g., the auxiliary product data, auxiliary service data, time constraint for the pickup location and/or the destination, etc.) matches for at least one vehicle (e.g., at least one available vehicle). This may be accomplished as described in connection with one or more of <FIG>. For example, the electronic device <NUM> may determine whether there is a sufficient quantity of auxiliary product(s) (in the vehicle and/or that may be obtained, for instance) to satisfy the auxiliary product request and/or whether the one or more auxiliary services are available (in the vehicle and/or may be obtained) to satisfy the auxiliary service request. The electronic device <NUM> may also determine whether the ride may be completed within the time constraint based on the location of each vehicle, the pickup location, and/or the destination. For instance, the electronic device <NUM> may calculate (based on one or more driving distances, traffic, weather, etc., for example) an amount of time required to complete (with a threshold probability, for example) the ride and/or obtain any product(s) or service(s) needed to fulfill the request(s). If the amount of calculated time is less than the time constraint and if there is a sufficient quantity of auxiliary product(s) and/or available service(s) (or may be obtained within the time constraint), then the criteria may match for that vehicle.

If all of the criteria match for at least one vehicle, the electronic device <NUM> may assign <NUM> (and/or dispatch) a vehicle with the closest matching criteria to fulfill the vehicle request. This may be accomplished as described in connection with one or more of <FIG>. For example, the electronic device <NUM> may assign and/or dispatch a vehicle with one or more auxiliary products and/or auxiliary services that may satisfy the auxiliary product request and/or the auxiliary service request within the time constraint(s).

If not all of the criteria match for at least one vehicle (e.g., for any vehicle within a geographical area, for example), the electronic device <NUM> may offer <NUM> (e.g., provide) a longer wait time and/or one or more different options. This may be accomplished as described in connection with one or more of <FIG> and <FIG>. For instance, the one or more options may include one or more vehicles with longer wait times (than the time constraint, for example) and/or that fully or partially satisfy the auxiliary product request and/or auxiliary service request, etc. In some approaches, the one or more options may include one or more vehicles that are currently unavailable but are anticipated to become available at a later time.

The electronic device <NUM> may determine <NUM> whether to modify the vehicle request. For example, the electronic device <NUM> may determine whether to modify the auxiliary product request, the auxiliary service request, the time constraint(s), and/or one or more other criteria of the vehicle request. For instance, the electronic device <NUM> may receive a selection (based on an input received from a user, for example) of the one or more indicated options. If a selection is received, the electronic device <NUM> may modify the vehicle request in accordance with the selected option. In some approaches, the electronic device <NUM> may return to determine <NUM> whether all the criteria (of the modified vehicle request) match for one or more vehicles. In other approaches, the electronic device <NUM> may directly assign <NUM> the vehicle corresponding to the selected option. If it is determined <NUM> not to modify the vehicle request, operation may end <NUM>. In an additional or alternative approach, if there is no vehicle available that would satisfy the vehicle request, the electronic device <NUM> may assign (and/or dispatch) a vehicle to obtain the requested auxiliary product(s) and/or auxiliary service(s).

<FIG> is a block diagram illustrating examples of a request obtainer <NUM>, a data obtainer <NUM>, and a vehicle matcher <NUM>. The request obtainer <NUM>, data obtainer <NUM>, and/or vehicle matcher <NUM> may be examples of corresponding components described in connection with <FIG> and/or may be implemented in the electronic device <NUM> (e.g., processor <NUM>) described in connection with <FIG> in some configurations.

The request obtainer <NUM> may obtain a vehicle request <NUM>. For example, the request obtainer <NUM> may obtain the vehicle request <NUM> from a user interface and/or may receive the vehicle request <NUM> from a remote device. The vehicle request <NUM> may include a pickup location <NUM>, a destination <NUM>, an auxiliary product request <NUM>, an auxiliary service request <NUM>, and/or one or more time constraints <NUM>. The pickup location <NUM> may be obtained from a locating device (e.g., GPS, Wi-Fi-based locating device, inertial navigation device, etc.) or may be obtained from an input (e.g., a manual user input). The destination <NUM> may be obtained from a locating device (e.g., GPS, Wi-Fi-based locating device, inertial navigation device, etc.) or may be obtained from an input (e.g., a manual user input). The pickup location <NUM> and/or destination <NUM> may be obtained locally (e.g., by the electronic device <NUM>) and/or from a remote device (via the communication interface <NUM>, for example).

In some configurations, the vehicle request may indicate whether the user wants a vehicle with a driver or a driverless (e.g., autonomous) vehicle. In some configurations, the vehicle request <NUM> may specify one or more conventional vehicle items or options (e.g., some kinds of vehicle-integrated electronics, seats, climate control unit(s) (e.g., air conditioning, heating, etc.), vehicle adjustment settings (e.g., seat adjustment settings, rear-view mirror settings, steering wheel adjustments, etc.), etc.)) in addition to the auxiliary product request <NUM> and/or the auxiliary service request <NUM>.

The auxiliary product request <NUM> may indicate one or more auxiliary products requested by a user. The auxiliary service request <NUM> may indicate one or more services requested by a user.

In some configurations, the request obtainer <NUM> may obtain the vehicle request from a user interface. For example, the user interface may receive one or more inputs (e.g., keyboard inputs, touch inputs, voice inputs, gesture inputs, etc.) that indicate the vehicle request <NUM>. Additionally or alternatively, the request obtainer <NUM> may obtain the vehicle request <NUM> from a remote device. For example, the request obtainer <NUM> may receive the vehicle request <NUM> via one or more communication interface(s). The vehicle request <NUM> may be stored in memory in some configurations.

The data obtainer <NUM> may obtain auxiliary product data <NUM>, auxiliary service data <NUM>, one or more vehicle locations <NUM>, and/or vehicle availability <NUM> information associated with one or more vehicles. For example, the data obtainer <NUM> may determine an auxiliary product inventory and/or an auxiliary service availability. The auxiliary product data <NUM> is stored in memory. The auxiliary service data <NUM> may be stored in memory. Obtaining the auxiliary product data and/or the auxiliary service data is based on image data from an interior of at least one vehicle and/or based on computer vision analysis of the image data as described herein. Additionally or alternatively, dedicated optical sensing, dedicated mechanical sensing, and/or scanning may be utilized to obtain the auxiliary product data <NUM> and/or auxiliary service data <NUM>.

The data obtainer <NUM> may request and/or receive the vehicle location(s) <NUM> and/or vehicle availability <NUM> (and/or one or more vehicle aspects, for instance) from one or more remote devices (e.g., vehicles, electronic devices, servers, etc.). Additionally or alternatively, one or more of the vehicle location(s), vehicle availability, and/or vehicle aspect(s) may be obtained locally and/or may be predetermined. In some configurations, each vehicle location may be determined and/or reported by each vehicle and/or electronics in each vehicle. For example, a vehicle may include a location device (e.g., GPS, Wi-Fi-based locating device, inertial navigation device, etc.) that may be utilized to determine the vehicle location <NUM>.

In some configurations, the vehicle availability <NUM> may be determined based on whether the vehicle is currently carrying out an assigned trip. For example, if a vehicle has been assigned to carry out (and is currently carrying out, for instance) an assignment, the vehicle and/or electronics in the vehicle may indicate that the vehicle is unavailable. Otherwise, the vehicle and/or electronics in the vehicle may indicate that the vehicle is available. Additionally or alternatively, the vehicle availability <NUM> may be determined based on an input (e.g., a user input). For example, a device may receive an input (from a driver, for example) indicating whether the vehicle is available or unavailable. In some configurations, the vehicle availability <NUM> may be determined based on the location of the vehicle (e.g., vehicle location <NUM>).

The vehicle matcher <NUM> may obtain the vehicle request <NUM> (e.g., one or more vehicle requests) and may obtain auxiliary product data <NUM>, auxiliary service data <NUM>, one or more vehicle locations <NUM> and/or vehicle availability <NUM> information from the data obtainer <NUM>. The vehicle matcher <NUM> may determine whether one or more vehicles associated with the auxiliary product data <NUM>, the auxiliary service data <NUM>, the vehicle location(s) <NUM>, and/or the vehicle availability <NUM> may satisfy the vehicle request <NUM> (e.g., pickup location <NUM>, destination <NUM>, auxiliary product request <NUM>, auxiliary service request <NUM>, and/or one or more time constraints <NUM>). For example, the vehicle matcher <NUM> may determine whether one or more vehicles have auxiliary product(s) (e.g., inventory) available, auxiliary service(s) available, vehicle location, and/or vehicle availability to satisfy the vehicle request <NUM> (auxiliary product request <NUM> and/or the auxiliary service request <NUM>). In some configurations, the vehicle matcher <NUM> may perform a query on one or more databases (e.g., an auxiliary products database, an auxiliary service database, a supply and/or ingredient inventory database, etc.) based on the auxiliary product request <NUM> and/or the auxiliary service request <NUM>. For example, the vehicle matcher <NUM> may query the database(s) for one or more auxiliary products and/or one or more auxiliary services identified by the auxiliary product request <NUM> and/or the auxiliary service request <NUM>. The auxiliary product data <NUM> may indicate a quantity of one or more auxiliary products available in one or more vehicles. The auxiliary service data <NUM> may indicate a quantity of one or more auxiliary services available in one or more vehicles. If all the criteria are (or would be) satisfied, the vehicle matcher <NUM> may indicate a match and/or assign (and/or dispatch) the vehicle to satisfy the vehicle request. If not all of the criteria would be satisfied for any vehicle (within a geographical range, for example), the vehicle matcher <NUM> may indicate (e.g., offer, provide, etc.) one or more other options in some approaches.

<FIG> is a flow diagram illustrating one configuration of a method <NUM> for monitoring one or more quantities of one or more auxiliary products (e.g., auxiliary product inventory). The method <NUM> may be performed by the electronic device <NUM>, for example.

The electronic device <NUM> obtains <NUM> image data (from an interior of at least one vehicle, for example). This may be accomplished as described above in connection with one or more of <FIG>, <FIG>, and <FIG>.

The electronic device <NUM> performs <NUM> computer vision analysis on the image data. This may be accomplished as described in connection with one or more of <FIG> and <FIG>. For example, the electronic device <NUM> may perform object detection, object recognition, object tracking, and/or object classification on the image data.

The electronic device <NUM> may determine <NUM> whether addition of an auxiliary product is detected. For example, the electronic device <NUM> may determine <NUM> whether one or more additional auxiliary products (e.g., item, supply, ingredient, etc.) are detected, recognized, tracked, and/or classified from the image data. For example, if a number of auxiliary products detected in current image data (e.g., a current frame) is greater than a number of auxiliary products in previous image data (e.g., a previous frame), the electronic device <NUM> may determine <NUM> that addition of an auxiliary product has been detected. If addition of an auxiliary product is detected, the electronic device <NUM> may update <NUM> the product inventory to indicate the addition. For example, the electronic device <NUM> may increment the product inventory quantity for a number of auxiliary products that have been detected. For instance, the electronic device <NUM> may update a database of auxiliary products (e.g., a supply and/or ingredient inventory database).

If an addition of an auxiliary product is not detected or upon updating <NUM>, the electronic device <NUM> may determine <NUM> whether removal of one or more auxiliary products (e.g., item, supply, ingredient, etc.) is detected. For example, the electronic device <NUM> may determine <NUM> whether fewer auxiliary products are detected, recognized, tracked, and/or classified from the image data. For example, if a number of auxiliary products detected in current image data (e.g., a current frame) is less than a number of auxiliary products in previous image data (e.g., a previous frame), the electronic device <NUM> may determine <NUM> that removal of an auxiliary product has been detected. If removal of an auxiliary product is detected, the electronic device <NUM> may update <NUM> the auxiliary product inventory to indicate the removal. For example, the electronic device <NUM> may decrement the product inventory quantity for a number of auxiliary products that have been detected. For instance, the electronic device <NUM> may update a database of auxiliary products (e.g., a supply and/or ingredient inventory database).

If a removal of an auxiliary product is not detected or upon updating <NUM>, the electronic device <NUM> may determine <NUM> whether the quantity of an auxiliary product is greater than or equal to an auxiliary product request amount. If the quantity of the auxiliary product is greater than or equal to the auxiliary product request amount, the electronic device <NUM> may indicate <NUM> that the auxiliary product inventory of a vehicle satisfies (or would satisfy) the auxiliary product request. For example, the electronic device <NUM> (e.g., vehicle matcher <NUM>) may indicate that the auxiliary product inventory of at least one vehicle satisfies the auxiliary product request. In some approaches, the electronic device <NUM> may assign and/or dispatch the vehicle. As described herein, whether to assign and/or dispatch a vehicle may be based on a combination of criteria (e.g., auxiliary product inventory, auxiliary service availability, distance to a pickup location, whether the vehicle may meet a time constraint, vehicle availability, whether one or more other vehicle aspects match the vehicle request, etc.) in some approaches.

If the quantity of the auxiliary product is not greater than or equal to (e.g., is less than) the auxiliary product request amount, the electronic device <NUM> may indicate <NUM> that the auxiliary product inventory of a vehicle does not satisfy (or would not satisfy) the auxiliary product request. For example, the electronic device <NUM> (e.g., vehicle matcher <NUM>) may indicate that the auxiliary product inventory of at least one vehicle does not satisfy the auxiliary product request. It should be noted that a similar approach to that of the method <NUM> may be utilized to monitor availability, use, and/or consumption of one or more auxiliary services.

<FIG> illustrates certain components that may be included within an electronic device <NUM>. The electronic device <NUM> may be an example of and/or may be implemented in accordance with the electronic device <NUM> described in connection with <FIG>. The electronic device <NUM> may be (or may be included within) a camera, video camcorder, digital camera, cellular phone, smart phone, computer (e.g., desktop computer, laptop computer, etc.), tablet device, media player, television, vehicle, automobile, surveillance camera, mounted camera, connected camera, robot, aircraft, drone, unmanned aerial vehicle (UAV), healthcare equipment, gaming console, personal digital assistant (PDA), set-top box, etc. The electronic device <NUM> includes a processor <NUM>. The processor <NUM> may be a general purpose single- or multi-chip microprocessor (e.g., an ARM), a special purpose microprocessor (e.g., a digital signal processor (DSP), image signal processor (ISP), etc.), a microcontroller, a programmable gate array, etc. The processor <NUM> may be referred to as a central processing unit (CPU). Although just a single processor <NUM> is shown in the electronic device <NUM>, in an alternative configuration, a combination of processors (e.g., an ARM and DSP) could be used.

The electronic device <NUM> also includes memory <NUM>. The memory <NUM> may be any electronic component capable of storing electronic information. The memory <NUM> may be embodied as random access memory (RAM), read-only memory (ROM), magnetic disk storage media, optical storage media, flash memory devices in RAM, synchronous dynamic random-access memory (SDRAM), on-board memory included with the processor, EPROM memory, EEPROM memory, registers, and so forth, including combinations thereof.

Data 1070a and instructions 1068a may be stored in the memory <NUM>. The instructions 1068a may be executable by the processor <NUM> to implement one or more of the methods <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> described herein. Executing the instructions 1068a may involve the use of the data 1070a that is stored in the memory <NUM>. When the processor <NUM> executes the instructions <NUM>, various portions of the instructions 1068b may be loaded onto the processor <NUM>, and various pieces of data 1070b may be loaded onto the processor <NUM>.

The electronic device <NUM> may also include a transmitter <NUM> and a receiver <NUM> to allow transmission and reception of signals to and from the electronic device <NUM>. The transmitter <NUM> and receiver <NUM> may be collectively referred to as a transceiver <NUM>. One or multiple antennas 1080a-b may be electrically coupled to the transceiver <NUM>. The electronic device <NUM> may also include (not shown) multiple transmitters, multiple receivers, multiple transceivers and/or additional antennas.

The electronic device <NUM> may include a digital signal processor (DSP) <NUM>. The electronic device <NUM> may also include a communication interface <NUM>. The communication interface <NUM> may enable one or more kinds of input and/or output. For example, the communication interface <NUM> may include one or more ports and/or communication devices for linking other devices to the electronic device <NUM>. In some configurations, the communication interface <NUM> may include the transmitter <NUM> and/or the receiver <NUM> (e.g., the transceiver <NUM>). Additionally or alternatively, the communication interface <NUM> may include one or more other interfaces (e.g., touchscreen, keypad, keyboard, microphone, camera, etc.). For example, the communication interface <NUM> may enable a user to interact with the electronic device <NUM>.

The various components of the electronic device <NUM> may be coupled together by one or more buses, which may include a power bus, a control signal bus, a status signal bus, a data bus, etc. For the sake of clarity, the various buses are illustrated in <FIG> as a bus system <NUM>.

Also, "determining" can include resolving, selecting, choosing, establishing, and the like.

The term "processor" should be interpreted broadly to encompass a general purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, and so forth. Under some circumstances, a "processor" may refer to an application specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc. The term "processor" may refer to a combination of processing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

The term "memory" should be interpreted broadly to encompass any electronic component capable of storing electronic information. The term memory may refer to various types of processor-readable media such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, etc. Memory is said to be in electronic communication with a processor if the processor can read information from and/or write information to the memory. Memory that is integral to a processor is in electronic communication with the processor.

The terms "instructions" and "code" should be interpreted broadly to include any type of computer-readable statement(s). For example, the terms "instructions" and "code" may refer to one or more programs, routines, sub-routines, functions, procedures, etc. "Instructions" and "code" may comprise a single computer-readable statement or many computer-readable statements.

The functions described herein may be implemented in software or firmware being executed by hardware. The functions may be stored as one or more instructions on a computer-readable medium. The terms "computer-readable medium" or "computer-program product" refers to any tangible storage medium that can be accessed by a computer or a processor. By way of example, and not limitation, a computer-readable medium may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray® disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. It should be noted that a computer-readable medium may be tangible and non-transitory. The term "computer-program product" refers to a computing device or processor in combination with code or instructions (e.g., a "program") that may be executed, processed, or computed by the computing device or processor. As used herein, the term "code" may refer to software, instructions, code, or data that is/are executable by a computing device or processor.

Software or instructions may also be transmitted over a transmission medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio and microwave are included in the definition of transmission medium.

In other words, unless a specific order of steps or actions is required for proper operation of the method that is being described, the order and/or use of specific steps and/or actions may be modified without departing from the scope of the claims.

Claim 1:
A method performed by an electronic device for determining an inventory of auxiliary products in the interior of a vehicle, the method comprising:
storing in memory (<NUM>) a plurality of recognition models corresponding to a plurality of auxiliary products;
obtaining, by the processor of the electronic device, image data of a scene of the interior of the vehicle from one or more image sensors in the vehicle;
performing, by the processor of the electronic device, computer vision analysis on the image data, computer vision analysis comprising performing object recognition on the image data, wherein object recognition comprises:
determining recognition data from the image data;
comparing the recognition data with the recognition models to determine whether the recognition data matches one or more of the recognition models;
storing, based on the comparing, in an auxiliary product inventory of the vehicle , auxiliary product data indicating the objects represented by the recognition models that are in the vehicle.