DRIVE THROUGH SYSTEM INCLUDING VISION SYSTEM AND TRANSACTION SYSTEM INTEGRATION

A control system for a drive through includes processing circuitry. The processing circuitry is configured to determine, using image data obtained from multiple cameras, a characteristic of a vehicle in a lane of the drive through. The processing circuitry can operate a point of sale unit to provide an indication of the characteristic of the vehicle. The processing circuitry can, responsive to detection of one or more of a plurality of vehicles using the image data, perform at least one of (i) initiating an alert via the point of sale unit for staff of the drive through, (ii) initiating an action to open an additional lane, or (iii) initiating a kitchen action to prepare one or more food or beverage items.

FIELD

The present disclosure relates to vision systems. More specifically, the present disclosure relates to vision systems for drive throughs.

BACKGROUND

Drive through systems may have multiple lanes with a combination of display interfaces, microphones, speakers, and vehicle detection capabilities. When a customer arrives at the drive through system, the customer may communicate via a menu board or unit with an order taker to place their order. The customer then pulls around to pay and pick up the customer's order. Cameras may obtain image data of vehicles at the drive through system.

SUMMARY

One implementation of the present disclosure is a drive through system, according to some embodiments. The drive through system can include a transaction system, such as a point of sale unit, configured to be operated to implement a point of sale. The drive through system can also include multiple cameras configured to obtain image data of a lane of the drive through system. The drive through system can also include processing circuitry configured to determine, using the image data, a characteristic of a vehicle at the lane, and operate the point of sale unit to provide an indication of the characteristic of the vehicle. The processing circuitry may also be configured to, responsive to detection of one or more of multiple vehicles using the image data, perform at least one of (i) initiating an alert via the point of sale unit for staff of the drive through system, (ii) initiating an action to open an additional lane, or (iii) initiating a kitchen action to prepare one or more food or beverage items.

The characteristic of the vehicle may include a type of the vehicle selected from multiple predetermined types of vehicles using the image data, and a color of the vehicle. The multiple predetermined types of vehicles can include at least a car, a truck, and an SUV.

The processing circuitry may be further configured to determine, using the image data from one of multiple cameras or the multiple cameras and predetermined zones, a location of the vehicle along a path of the lane. The predetermined zones may be defined for the image data of each of the multiple cameras to account for different orientations and positions of the cameras that results in image data of the predetermined zones spanning different portions of the image data of each of the cameras. The processing circuitry can also be configured to, responsive to the vehicle arriving at an order placement location of the zone, operating the point of sale unit to provide the indication of the characteristic of the vehicle.

The indication of the characteristic of the vehicle can include at least one of a textual indication of the characteristic of the vehicle or a visual representation of the characteristic of the vehicle. The drive through system can also include at least one of a point of sale unit or a display screen (e.g., notifier, alarm) at a pickup window of the drive through system viewable by an employee of the drive through system. The processing circuitry may be configured to operate at least one of the point of sale unit or the display screen at the pickup window to provide order data associated with the vehicle and the characteristic of the vehicle responsive to the vehicle arriving at the pickup window.

The processing circuitry can be configured to track multiple vehicles through the lane the drive through system using the image data obtained from the cameras. The processing circuitry can also be configured to, responsive to a number of the plurality of vehicles, at least one of initiate an alert for staff of the drive through system, initiate an action to open an additional lane, or initiate a kitchen action to prepare a quantity of food items. The processing circuitry can also be configured to operate, based on the image data, a carry-out alert device to notify carry-out staff regarding (i) the characteristic of a vehicle tracked to a carry-out location, (ii) an identification of which carry-out location of a plurality of carry-out locations at which the vehicle is positioned, and (iii) order data for the vehicle at the carry-out location.

Another implementation of the present disclosure is a method, according to some embodiments. The method may include obtaining image data of a lane of a drive through system and determining, using the image data, a visual characteristic of a vehicle at the lane. The method can also include operating a point of sale unit to provide an indication of the visual characteristic of the vehicle. The point of sale unit may be configured to be operated to implement a point of sale. The visual characteristic may include a type of the vehicle selected from multiple predetermined types using the image data, and a color of the vehicle. The method can also include, processing, at the point of sale unit, vehicle information relating to the visual characteristic of the vehicle and order information for an order receive from an occupant of a drive through to provide compiled order information. The method may also include updating multiple display devices of the drive through system to include an indication of the compiled order information during a service journey of the vehicle at the drive through.

The multiple types of vehicles can include at least a car, a truck, and an SUV. The method can also include determining, using the image data and multiple predetermined zones, a location of the vehicle along a path of the lane. The method can also include, responsive to the vehicle arriving at an order placement location of the zone, operating multiple point of sale units and at least one display screen to provide the indication of the characteristic of the vehicle to staff of the drive through system. The multiple point of sale units may be positioned at different employee stations of the drive through system and the display screen may be positioned in a kitchen or at a pickup window of the drive through system.

The indication of the visual characteristic of the vehicle can include at least one of a textual indication of the characteristic of the vehicle or a visual representation of the visual characteristic of the vehicle. The method can further include operating an alert device to provide an indication of the compiled order information responsive to the vehicle arriving at a pickup window. The alert device may be positioned at a pickup window of the drive through system and viewable by an employee of the drive through system.

The method can also include tracking multiple vehicles through the lane the drive through system using the image data obtained from multiple cameras. The method can also include, responsive to a number of the vehicles, at least one of initiating an alert for staff of the drive through system, initiating an action to open an additional lane, or initiating a kitchen action to prepare a quantity of food items. The method can also include operating a carry-out alert device to provide an indication of the compiled order information responsive to the vehicle arriving at a carry-out location. The carry-out alert device can also be configured to provide an indication of which of multiple carry-out locations at which the vehicle is located.

Another implementation of the present disclosure is a control system for a drive through, according to some embodiments. The control system includes processing circuitry, according to some embodiments. The processing circuitry may determine, using image data obtained from multiple cameras, a characteristic of a vehicle in a lane of the drive through. The processing circuitry can operate a point of sale unit to provide an indication of the characteristic of the vehicle. The processing circuitry can, responsive to detection of one or more of a plurality of vehicles using the image data, perform at least one of (i) initiating an alert via the point of sale unit for staff of the drive through, (ii) initiating an action to open an additional lane, or (iii) initiating a kitchen action to prepare one or more food or beverage items.

In some embodiments, the processing circuitry is further configured to, responsive to detecting that a vehicle has arrived at an order fulfillment location, operate a display screen at the order fulfillment location to display an initially obtained image of the vehicle to an order fulfillment staff to facilitate accurate order fulfillment. In some embodiments, the point of sale unit is a first point of sale unit of multiple point of sale units. The processing circuitry is configured to operate the point of sale units to provide the indication of the characteristic of the vehicle, and a location of the vehicle in the drive through on display screens of the point of sale units in unison, according to some embodiments. In some embodiments, the point of sale units are each configured to receive a user input to transition a corresponding one of multiple portable communications devices to communicate on an audio channel according to the user input. In some embodiments, the processing circuitry is further configured to operate a kitchen display screen to provide order information, the characteristic of the vehicle, and a location of the vehicle to kitchen staff of the drive through.

Another implementation of the present disclosure is a point of sale system for a drive through, according to some embodiments. The point of sale system can include a point of sale unit and processing circuitry. The point of sale unit can be configured to be operated to implement a point of sale and obtain order data. The processing circuitry can be configured to determine, using image data obtained from a camera, a characteristic of a vehicle at a lane of the drive through. The processing circuitry may also be configured to operate the point of sale unit to provide an indication of the characteristic of the vehicle.

The characteristic of the vehicle include a type of the vehicle selected from multiple predetermined types of vehicles using the image data, and a color of the vehicle. The multiple predetermined types of vehicles can include at least a car, a truck, and an SUV.

The processing circuitry can further be configured to determine, using the image data and multiple predetermined zones, a location of the vehicle along a path of the lane. The processing circuitry can also be configured to, responsive to the vehicle arriving at an order placement location of the zone, operate the point of sale unit to provide the indication of the characteristic of the vehicle.

The indication of the characteristic of the vehicle can include at least one of a textual indication of the characteristic of the vehicle or a visual representation of the characteristic of the vehicle. The point of sale system can include an alert device at a pickup window of the drive through system viewable by an employee of the drive through system. The processing circuitry may be configured to operate the alert device to provide the order data associated with the vehicle and the characteristic of the vehicle responsive to the vehicle arriving at the pickup window.

The processing circuitry can be configured to track multiple vehicles through the lane the drive through system using the image data obtained from multiple cameras. The processing circuitry can also, responsive to a number of the multiple vehicles, at least one of initiate an alert for staff of the drive through system, initiate an action to open an additional lane, or initiate a kitchen action to prepare a quantity of food items.

This summary is illustrative only and is not intended to be limiting. Various aspects, inventive features, and advantages of the systems described herein are set forth in more detail below.

Various aspects of the present disclosure, such as the graphical user interfaces (GUIs) shown and described, can be implemented in connection with the drive-through systems and interfaces set forth in U.S. Provisional Application No. 63/539,920 filed Sep. 22, 2023, which is incorporated herein by reference in its entirety.

DETAILED DESCRIPTION

Overview

Referring generally to the FIGURES, a vision system for a drive through system can include cameras that track vehicles through their life cycle of the drive through system. The vision system may implement a machine learning technique in order to automatically identify a type and color of the vehicles in the drive through system. The vision system is integrated with a transaction system, e.g., a system of point of sale units, such that the point of sale units are updated to indicate the type and color of the vehicles. Integrating the vision system with the point of sale units facilitates improved speed and order fulfillment accuracy of the drive through system.

Drive Through System

Referring particularly toFIGS.1-2, a drive through system10for a store14(e.g., a restaurant, a diner, a fast food restaurant, an establishment, etc.) includes a lane system16(e.g., a starting lane, one or more ordering lanes where a transaction such as a point of sale (POS) transaction may be conducted or initiated, ordering lanes where customers may communicate with employees of the store14, etc.), a starting location32, and an ending location34. The starting location32may be a zone or area at which one or more customers (e.g., customers in vehicles, shown as vehicle30) enter the lane system16. The lane system16includes an entry lane18, a first order lane20a, a second order lane20b, a third order lane20c, a fourth order lane20d, and a pickup lane22. The pickup lane22(e.g., an exit lane) may extend along a side of the store14including one or more pickup windows, shown as first window28a, second window28b, and third window28csuch that customers (e.g., vehicles30) may access the different windows28to pick up their orders from the store14. The term “window” should be interpreted broadly to mean a customer-store interface which is not limited to those with a physical screen or barrier.

The starting location32and the ending location34may define starting and ending points for customers, with the order lanes20defining different paths between the starting location32and the ending location34. In some embodiments, each of the order lanes20defines a corresponding order zone24at which the customer may place an order for pickup at one of the windows28. In some embodiments, the first lane20aincludes a first order zone24a, the second lane20bincludes a second order zone24b, the third lane20cincludes a third order zone24c, and the fourth lane20dincludes a fourth order zone24d. Each of the order zones24includes a corresponding drive through unit108(e.g., a menu board, a kiosk, a field communications unit, an order placement unit at which a customer may place an order, etc.) which facilitates communications between the customer at the order zones24and personnel of the store14, as well a display screen or signage indicating available items for purchase, and order or POS information (e.g., a number of items, corresponding cost, total cost, tax, etc., for an ongoing order). In some embodiments, the first order lane20aincludes a first drive through unit108a, the second order lane20bincludes a second drive through unit108b, the third order lane20cincludes a third drive through unit108c, and the fourth order lane20dincludes a fourth drive through unit108d. In some embodiments, the order lanes20define one or more paths that include a series of discrete geometric forms (e.g., polygonal shapes, curvilinear shapes) mapped to different physical locations along the order lanes20. The series of geometric forms may match between multiple cameras (e.g., cameras114) that have different field of views in order to facilitate awareness, identification, and tracking of vehicles along the one or more paths between the starting location32and the ending location34.

The drive through units108are interfaces that may be components of a communications system or POS system, shown as communications system100of the drive through system10. In some embodiments, the drive through units108are integrated via one or more POS systems. The communications system100includes the drive through units108, a controller102, one or more imaging devices114(e.g., cameras) positioned about the lane system16, a cloud computing system104, one or more input devices, shown as order taking stations110, and one or more wearable communications devices112. In some embodiments, the one or more order taking stations110are disposed in each of a corresponding zone within the store14proximate the windows28. The order taking stations110may include a touch screen or user interface configured to both display image data (e.g., a graphical user interface, a menu, selectable options for an order or POS, etc.), and receive user inputs from a corresponding employee of the store14to add items to an order or POS.

The wearable communications devices112may have the form of headphones, earpieces, etc., and can include both speakers (e.g., acoustic transducers, aural output devices, etc.) and microphones (e.g., aural input devices). In some embodiments, the controller102is communicably coupled with each of the order taking stations110at the windows28, the imaging devices114, and the head wearable communications devices112. The controller102may also be communicably coupled with a cloud computing system104and can upload or provide various information regarding points of sale to the cloud computing system104for analysis. In some embodiments, the controller102is configured to receive, from the cloud computing system104, software or firmware updates for various devices of the communications system100. In some embodiments, the wearable communications devices112may be portable communications devices including but not limited to headphones, earphones, earbuds, devices configured to clip to a belt or article of clothing, ear pieces, etc., any device having at least one of a speaker or microphone and configured to be worn, held, or otherwise move with an individual to establish an end point for audio communications (e.g., to at least one of produce sound via a speaker, or receive a sound input via a microphone). It should be understood that the components of the communications system100(e.g., the drive through units108, the controller, the one or more imaging devices114, the cloud computing system104, the order taking stations110, and the wearable communications devices112) may optionally be integrated with each other via a POS (e.g., all such components or a subset of the foregoing components). Integrating the components of the communications system100via the POS may facilitate improved efficiency of the drive through, for example.

The communications system100provides selective communicability according to multiple channels between any of the personnel (e.g., personnel40aat the first window28a, personnel40bat the second window28b, and personnel30cat the third window28c) and customers at one of the order zones (e.g., the first order zone24a, the second order zone24b, the third order zone24c, and the fourth order zone24d) via the wearable communications devices112and the drive through units108. In some embodiments, each of the drive through units108is configured to communicate according to a corresponding channel (e.g., a first order channel, a second order channel, a third order channel, and a fourth order channel) and the personnel at the windows28can selectively establish communicability with the customers at the different order lanes20by selectively switching between the channels such that the wearable communications devices112are communicatively coupled on a selected one of the channels. In some embodiments, the communications system100also facilitates private communication between one or more of the personnel40within the store14or at other remote locations (e.g., a service center). For example, the personnel may include kitchen personnel40dthat may communicate with any of the personnel40a, the personnel40b, or the personnel40cat the windows28via one or more remote channels. In some embodiments, the kitchen personnel40dmay also listen on one or more of the channels on which the customers at the zones24can communicate.

Referring particularly toFIG.2, one of the order zones24is shown in greater detail, according to some embodiments. The order zones24may include a space or surface of the order lane20for the customer, shown as vehicle30to pull up and place an order. The order zone24may include at least one pressure or force sensor106. The at least one pressure or force sensor106is configured to detect the presence of the vehicle30and notify the controller102that a customer has arrived to the specific order zone24(e.g., pulled up, reversed into, merged into, parked in, etc.). The order zones24may also include one or more imaging devices114(e.g., cameras, image sensors, sensor arrays) for detecting the presence of the vehicle30. The drive through units108include a display screen118configured to display a current order, items in the order, and cost of the order or other aspects of an order property, alone or in any combination. The display screen118may be operable by one or more of the personnel40who is taking the customer's order. In some embodiments, the order zones24may optionally include at least one pressure sensor and/or at least one magnetic ground loop. For example, if the order zones24already include a pressure sensor and/or a magnetic ground loop, the controller102may communicate with and use feedback from the pressure sensor and/or the magnetic ground loop. In some embodiments, the order zones24may lack both a pressure sensor and/or a magnetic ground loop.

The drive through unit108may also include a microphone122configured to capture audio (e.g., spoken audio) from the customer and transmit data (audio data, audio signals) to the personnel40that is taking the customer's order via a corresponding channel of the communications system100for the order lane20of the order zone24. In some embodiments, the drive through unit108also includes a speaker120configured to provide audio data of the voice or spoken words of the personnel40obtained by the wearable communication device112that is communicating on the channel of the drive through unit108. In this way, the communications unit108may provide a customer end for audio communications and the wearable communications device112may provide a personnel end for audio communications between the customer and one of the personnel40(e.g., according to an audio or communications channel). In some embodiments, the at least one pressure or force sensor106, the speaker120, the microphone122, the imaging device114, and the display screen118are configured to communicate directly with the controller102of the communication system100. In some embodiments, the pressure or force sensor106, the speaker120, the microphone122, the imaging device114, and the display screen118are configured to communicate with a zone controller124that communicates with the controller102.

Referring again toFIG.1, the controller102and the imaging devices114may form a vision system200. The vision system200can be implemented on the controller102based on image data received from the imaging devices114. In some embodiments, the vision system200is configured to integrate with one or more POS units of the communications system100, such as the order taking units110. The vision system200is configured to track vehicles or customers through the drive through system10(e.g., along different lanes) and integrate with a POS system. The vision system200may trigger notifications to the personnel40. In some embodiments, the vision system200is configured to notify the personnel40regarding the presence of a vehicle at one of the drive through units108. In some embodiments, the vision system200is configured to identify which vehicle has arrived at a pickup window and notify associated personnel40regarding which order the vehicle at the pickup window should receive. In some embodiments, the vision system200is configured to prompt the personnel40to open another order taking or drive through lane. In some embodiments, the vision system200is configured to notify the kitchen personnel40dregarding an influx of vehicles30to the drive through system10to prompt the kitchen personnel40dto begin preparing food in advance.

Advantageously, the vision system200may track vehicles through the drive through system10and integrate with POS systems of the drive through system10. The vision system200may facilitate improved order taking, speed of service, and fulfillment accuracy. The vision system200integrates directly with the POS units such that graphical user interfaces of the POS units may be updated in real-time based on tracking of vehicles30through the drive through system10by the vision system200.

Vision System

Offline Setup

Referring toFIG.3, a setup system300for configuring the vision system200includes a setup device302, a user interface304, the controller102, the cloud computing system104, and a satellite306, according to some embodiments. The setup device302is configured to facilitate setting up and configuring the controller102to perform image analysis techniques and tracking of vehicles30through the drive through system10. In some embodiments, the setup device302is configured to generate a profile or configuration settings for the controller102. In some embodiments, the setup device302is configured to determine the profile or configuration settings for the controller102for a specific location or drive through system10. For example, the setup device302may implement one or more set-up functions (e.g., a guide for a user) to determine the profile or configuration settings for the controller102. The profile or the configuration settings may include paths corresponding to different lanes of traffic of the drive through system10, and one or more geometric shapes or zones along the paths. The profile or the configuration settings may be location-specific and tailored to the lanes of the specific location of the drive through system10. In some embodiments, the profile or the configuration settings include one or more preferred operating or alert settings for the store14(e.g., for kitchen personnel, for order takers, etc.). In some embodiments, the profile or the configuration settings include integration for the tracking of the vehicles30with one or more POS units. It should be understood that any of the functionality of the setup device302may be performed locally on the controller102, or remotely on the cloud computing system104.

The setup device302may obtain satellite image data of the drive through system10from the satellite306. The satellite image data may indicate different lanes of traffic, emergency lanes, parking lots, etc., of a store or restaurant. The setup device302is configured to operate the user interface304in order to display the satellite image data to a user, according to some embodiments. In some embodiments, the setup device302is configured to operate the user interface304to prompt the user to provide inputs indicative of one or more paths of the drive through system10. The paths of the drive through system10may correspond to different lanes of traffic for the store14. The user may provide to the setup device302, via the user interface304, one or more points indicating locations along a first path. The user may provide points corresponding to multiple different paths (e.g., a second path, a third path, etc.), corresponding to a number of lanes of traffic or routes. In some embodiments, the setup device302is configured to receive the points for each of the one or more paths and perform a curve fitting technique in order to generate the path based on the points. For example, the setup device302may implement a variety of curve fitting techniques based on the points, including but not limited to Bezier curves, B-spline curves, Catmull-Rom splines, interpolating Lagrange curves, etc. In some embodiments, the setup device302uses the point to generate paths or curves including control points. The control points may correspond to the points provided by the user via the user interface302that the setup device302uses to generate the curves or paths. In some embodiments, the points provided to the setup device302are global positioning system (GPS) coordinates such as latitude and longitude which are used by the setup device302to define the curves.

Once the paths or curves for each lane of traffic are determined by the setup device302, the setup device302may operate the user interface304to display the paths or curves. The setup device302may also receive updates to the curves or paths as user inputs from the user interface304. The updates to the curves or paths (e.g., the Catmull-Rom splines) may include the addition of or subtraction of control points, the adjustment (e.g., movement) of control points, changing an amount or direction of a tangent of one or more of the control points, etc.

The curves may generally define the lanes of traffic of vehicles30through the drive through system10and facilitate tracking the vehicles30through the drive through system10(e.g., from starting location32to the ending location34). In the example drive through system shown inFIG.1, six different curves may be defined by the user: (i) a first path or curve defined along the entry lane18, (ii) a second path or curve defined along the first order lane20a, (iii) a third path or curve defined along the second order lane20b, (iv) a fourth path or curve defined along the third order lane20c, (v) a fifth path or curve defined along the fourth order lane20d, and (vi) a sixth path or curve defined along the pickup lane22.

In some embodiments, the setup device302is configured to determined, based on the paths or curves defined by the user for the drive through system10, one or more suggested locations for the cameras114. If the cameras114are already installed such that the cameras114can obtain image data from a variety of locations of all of the paths or curves, the setup device302may proceed to generation and customization of different tracking zones along the paths or curves. In some embodiments, the setup device302is configured to select from a variety of different cameras and determine, based on the cameras fields of view, positions and orientations for multiple cameras such that image data of all of the paths or curves can be obtained. In some embodiments, the setup device302is configured to provide, via the cloud computing system104, the satellite image data and the curves or paths superimposed on the satellite image data, to a system administrator of the setup system300. The system administrator (e.g., a manufacturer of the vision system200) may provide recommended locations, orientations, and types of cameras for the drive through system10. The system administrator may further provide, to the setup device302via the cloud computing system104, recommended locations, orientations, and types of cameras for the drive through system10based on an on-site or remote inspection, according to some embodiments.

The setup device302may be configured to determine multiple zones (e.g., geometric shapes, areas, etc.) along the paths or curves. The setup device302may automatically determine the zones along the paths or curves and operate the user interface304to display the satellite image data, the paths or curves, the points, and the zones superimposed on the satellite image data. In some embodiments, the user interface304is configured to use selections, provided by the user interface304as user inputs, and provide different image data from corresponding cameras114to the user via the user interface304in response to the selections. In some embodiments, the setup device302is configured to display the image data to the user interface304such that the user may define the zones.

The user interface304is configured to receive user inputs to define points in the image data of each of the cameras114to define zones along the curves or paths. In some embodiments, the setup device302is configured to receive four points for each zone in the image data. The setup device302may receive four points for each of the zones, and allow, via operation of the user interface304, the user to define zones for all of the lanes of traffic that are visible in the image data of one of the cameras114. After the user has defined the zones for the image data of a first camera114, the user may select a second camera114, and proceed to define correspondingly numbered zones for the image data of the second camera114. This process may be repeated by the user via the user interface304and the setup device302until all of the zones have been defined for the image data of each of the cameras114.

The user may be prompted to define the zones and assign corresponding numbers or labels in a uniform manner across the image data from different cameras114. For example, if a zone A is defined in front of a pickup window, then the zone A should be in front of the pickup window for the image data across different cameras114. Advantageously, the zones can be used by the controller102(e.g., in the profile) in order to track the vehicle30through the drive through system10.

Referring still toFIG.3, the setup device302may receive inputs from the user interface304in order to define one or more masks for the image data obtained from the cameras114. For example, if the first camera114includes image data of surrounding areas that vehicles30will not be detected in, the user inputs may define a mask for the surrounding areas. In some embodiments, the setup device302is configured to provide display data to the user interface304such that the user interface304displays the image data. The user interface304may provide an “eraser tool” to the user, a pen tool, a paintbrush tool, etc., such that the user can provide inputs to define the masks. The setup device302may store information of the masks of the image data as a part of the setup profile.

The setup device302may also receive user inputs from the user interface304defining locations of interest along the paths, curves, or routes. The locations of interest may include an arrival location, a drive through entry location, an order lane entry location, an order location, an order lane exit location, a payment location, an order pickup location, an exit location, etc. In some embodiments, the locations are stored by the setup device302and used in order to determine the setup profile. The locations may be point locations or may be tied to corresponding zones. For example, the locations may be labels associated with corresponding zones (e.g., zone501is the order pickup zone).

The setup device302is configured to provide the setup profile to the controller102for use in operating the drive through system10, according to some embodiments. In some embodiments, the setup profile includes (i) the paths, routes, or curves for the different lanes of traffic (e.g., entry lanes, parking lot lanes or routes, order lanes, pickup lanes, emergency lanes, exit lanes, etc.), (ii) the zones along the lanes of traffic (e.g., a series of geometric shapes that follow the lanes of traffic) for the satellite image data, and the image data from each of the cameras114, (iii) one or more locations of interest, (iv) and an image analysis technique or set of instructions. In some embodiments, the image analysis technique is provided in the form of a neural network, an artificial intelligence, a machine learning technique or set of instructions, etc. The image analysis technique may be performed by the controller102locally such that the controller102can use the image data provided by the cameras114in order to detect a type and color of vehicle. In some embodiments, performing the image analysis technique locally at the controller102of the drive through system10facilitates ensuring privacy of the image data.

Referring toFIG.4, a flow diagram of a process400for setting up a vision tracking system and implementing the vision tracking system in an integrated manner with a POS unit includes steps402-418, according to some embodiments. In some embodiments, the process400is implemented by the setup system300or components thereof, and the controller102of the drive through system10. In some embodiments, the process400is implemented in order to set up a profile in a location-specific area or for a specific restaurant location.

The process400includes obtaining map data of a drive through location (step402), according to some embodiments. In some embodiments, the map data of the drive through location is satellite image data. In some embodiments, the map data of the drive through location is obtained by the setup device302from the cloud computing system104or the satellite306. The map data may otherwise be obtained from a database, a maps service, a satellite image database, a drone that is flown around the drive through location, etc.

The process400includes defining a route of the drive through location (step404), according to some embodiments. In some embodiments, step404includes operating a user interface, display screen, computer screen, tablet, touch screen, etc., in order to display the map data obtained in step402. Step404may also include allowing the user, via a user interface, to provide one or more points that define the route. The points may be geographic (e.g., GPS) coordinates and may define different locations along a lane of traffic. In some embodiments, step404is performed by the setup device302. The setup device302may implement a curve-fitting technique (e.g., Catmull-Rom) based on the points defined by the user in order to determine the define the route. In some embodiments, step404includes defining multiple routes by allowing the user to input, via a user device, multiple points for each of multiple routes. For example, the different routes may correspond to different lanes of traffic (e.g., multiple order lanes, emergency lanes, parking lot locations or lanes of traffic, pickup lanes, entry lanes, exit lanes, etc.).

The process400includes identifying locations for imaging devices along the route (step406), according to some embodiments. In some embodiments, step406is optional. For example, if cameras are already installed at the drive through location, then step406may be unnecessary. In some embodiments, step406is performed automatically by the setup device302based on an identification of the routes, possible mounting locations, as well as camera models, fields of view, etc. In some embodiments, step406is performed by a manufacturer or installation service of a vision system for the drive through location. For example, the map data and the route(s) obtained in steps402-404may be provided to the manufacturer or installation service. The manufacturer or installation service may provide suggestions regarding the locations, orientations, and models of the imaging devices (e.g., cameras) in order to obtain image data of the entirety of the route from multiple locations.

The process400includes installing imaging devices at the identified locations along the route (step408), according to some embodiments. In some embodiments, step408includes installing imaging devices (e.g., cameras) at the locations identified in step406about the drive though location. Step408may include installing posts or poles and imaging devices at the top of the posts or poles for locations where additional imaging devices are required. In some embodiments, step408is optional if the drive through location already has imaging devices installed on the premises. Step408may be performed by the manufacturer of the vision system for the drive through location, an installation service, etc.

The process400includes defining zones along the route in image data obtained from the imaging devices (step410), according to some embodiments. In some embodiments, step410is performed by the setup device302based on inputs from a user (e.g., via the user interface304). For example, step410may include defining multiple zones (e.g., geometric shapes, areas, etc.) along the routes determined in step404for each of multiple imaging devices. In some embodiments, step410is performed by first prompting the user to define multiple zones or locations along the routes for the map data of the drive through location. In some embodiments, the setup device302is configured to automatically map the zones defined by the user (or automatically by defining zones along the routes) in the map data to the image data of the imaging devices. In some embodiments, the setup device302is configured to use one or more of a location, orientation, height, field of view, etc., of the imaging devices in order to automatically identify preliminary boundaries (e.g., edges, corners, etc.) for the zones in the image data of the imaging devices. The zones along the route may be definable by the user for the image data of the imaging devices and/or may be adjustable if an initial definition of the zones is provided automatically. The setup device302may prompt the user to define the zones in a manner that is consistent across the image data obtained from the imaging devices. For example, if a zone A is defined at a first order location (e.g., an order placement location in front of the drive through unit108) in image data obtained from a first imaging device, and the image data obtained from a second imaging device includes the first order location from a different perspective, the user may be prompted to define a zone at the first order location in the image data obtained from the second imaging device as zone A. In this way, the zones may be consistently defined in a uniform manner between the image data obtained from multiple imaging devices in order to account for different perspectives or views of a same location along the routes. Step410may be performed for any number of routes that are defined in step404.

The process400includes defining locations of interest along the route (step412), according to some embodiments. In some embodiments, step412is performed by the setup device302by prompting the user to input, via the user interface304, one or more locations of interest. The locations of interest may include an entry location, an exit location, a location with a best view of an incoming vehicle for use in image analysis, an entry to a first order lane, an entry to a second order lane, an entry to a third order lane, an entry to a fourth order lane, a first, second, third, or fourth order location, an exit to any order lane, a first window location, a second window location, a pickup order location, etc. In some embodiments, step412is performed by labelling one or more of the zones defined in step410along the route as any of the locations. In this way, identification of a vehicle in one of the zones may result in identification that the vehicle is at a location of interest (e.g., that a vehicle has pulled up to a first order location). The locations of interest may be used in order to integrate the vision system with a POS unit. The locations of interest may also be used in order to identify performance parameters of the drive through location (e.g., service time, total life cycle time of a vehicle in the drive through location, etc.).

The process400includes masking the image data obtained from the imaging devices (step414), according to some embodiments. In some embodiments, step414is performed by the setup device302and the user interface304. For example, the setup device302may operate the user interface304to provide image data of the imaging device and allowing the user to define one or more zones or areas of the image data within which vehicles are unlikely to be detected. The user interface304may provide a pen tool, an eraser tool, a paintbrush tool, etc., for the user to define the zones or areas of the image data as the masks. The masks defined in step414generally define regions of the image data obtained from each of the imaging devices that should not be used or considered by the vision system when tracking vehicles through the drive through location. For example, if the image data from the imaging device includes surrounding areas (e.g., sidewalks, background image data, surroundings, etc.) where a vehicle is unlikely to be detected, the mask may remove the corresponding regions of the image data from consideration in image analysis and tracking. In some embodiments, the masks indicate areas of the image data that should be considered by the image analysis and tracking techniques (e.g., an area of interest). The masks can facilitate speed and accuracy of the image analysis and tracking techniques performed on the image data obtained from the imaging device. In some embodiments, the masks define areas or regions that have extraneous image data.

The process400includes determining a profile for the drive through location (step416), according to some embodiments. In some embodiments, the profile is a configuration file or dataset for a controller (e.g., the controller102) of the drive through location. The profile may include the map data obtained in step402, the route(s) defined in step402, the locations and orientations of each of the imaging devices, the zones defined in step410, the locations of interest defined in step412, the masks defined in step414, and an image analysis technique. The profile may be location-specific for the particular drive through location and accounts for the unique camera placement, lanes of traffic shapes, distances, curvatures, the parking lot size, overall store arrangement, order locations, pickup locations, etc., of the specific drive through location for which process400is performed. In some embodiments, the profile is a locally-executable file for the controller of the drive through location. The profile may configure the controller of the drive through location to perform one or more image analysis and vehicle tracking techniques as desired. In some embodiments, the profile utilizes one or more of a neural network, a machine learning technique, an artificial intelligence tool, etc., that is provided by the manufacturer of the vision system described herein that can be implemented locally on the controller of the drive through location. If updates to the profile are desired, a drive through location administrator may re-initiate any of steps402-416in order to account for updates to lane configuration, construction at the store, closing of order lanes, etc. Step416may be performed by the setup device302by providing the profile to the controller102.

The process400includes operating a POS unit of the drive through location using a vision system that implements the profile (step418), according to some embodiments. In some embodiments, step418is implemented by the controller102of the drive through location. For example, the controller102may implement the profile and identify incoming customers' vehicles and track the customers vehicles through the various lanes of traffic (e.g., along the routes). The controller102may track the vehicles as the vehicles travel along the routes and pass from one zone to the next along the routes. In response to the vehicle approaching to an order location, the controller102may operate a POS unit (e.g., associated with an order taker) to notify the order taker that a customer has arrived at the order location. The POS unit may be operated to prompt the order taker to initiate an order for the customer at the order location, and may include providing information regarding the customer at the order location (e.g., color of vehicle, type of vehicle, etc.). Step418may also include operating a display screen or an alert device (e.g., an audiovisual system, an alarm, an LED display, a speaker configured to provide an audible notification) to drive through personnel fulfilling orders to customers that pull up to an order pickup location. For example, step418may include operating the display screen to notify personnel regarding the POS data, as well as the type of the vehicle and the color of the vehicle as the vehicle is identified at the order pickup location. Advantageously, notifying the personnel who fulfill orders to the customers regarding the POS data, as well as identifying characteristics of the vehicle associated with the POS data improves order fulfillment accuracy and reduces a likelihood that the wrong order is provided to the customers (e.g., reducing order mix-ups).

Referring toFIG.5, a diagram illustrates map data500(e.g., satellite data) of a particular restaurant514. The map data500includes a route502(e.g., a drive through route, a lane of travel, etc.) that extends around the restaurant514from an entry point to an order location, to a payment or pickup location, to an end point. In some embodiments, the route502is user-defined by presenting the map data500to the user (e.g., the owner of the restaurant514) and allowing the user to define points for the route502. In some embodiments, the map data500includes camera locations504that can be automatically suggested and populated on the map data500by the setup device302. The camera locations504may be provided as suggested locations by an administrator or manufacturer of a vision system. The camera locations504are provided such that cameras or imaging devices, when positioned at the camera locations504, are configured to obtain image data of an entirety of the route502.

Referring toFIG.6, a graphical user interface (GUI)600illustrates information that may be displayed on the user interface304during setup, according to some embodiments. In some embodiments, the GUI600includes an editing window602a(e.g., a first window). The editing window602apresents image data610from a selected one of the cameras114. For example, as shown inFIG.6, the editing window602aincludes image data610associated with “Camera 2.” The user may define a boundary634of a mask608in the editing window602a. In some embodiments, the user may define the boundary634of the mask608using a mouse cursor, a pen tool, a paint brush, a stylus on a tablet, etc. The mask608defines portions of the image data (e.g., ranges or areas of pixel) of the associated camera114that should be disregarded by an image analysis technique of the controller102. The mask608generally causes the controller102to disregard potions of the image data610that are surrounding or background image data of lanes of travel. In some embodiments, the mask608defines areas of interest and disinterest in the image data610obtained from the corresponding camera114. For example, the mask608may focus the controller102to analyze portions of the image data610corresponding to lanes of travel of the restaurant14.

The GUI600also includes a toolbar622including a camera selector624(e.g., buttons, a drop-down menu, selectable options, etc.) and a lane selector626, according to some embodiments. In some embodiments, the lane selector626allows the user to select between multiple lanes to edit a series of zones604or zones606corresponding to the selected lane. In the example shown inFIG.6, the restaurant14includes two lanes that are viewable by the “Camera 2.” The user may select between a first lane and a second lane in order to edit zones for the first lane or the second lane. In the example shown inFIG.6, the first lane includes a series of zones604, shown as first zone604a, second zone604b, and third zone604c. The second lane includes a similar series of zones706, shown as first zone704a, second zone704b, and third zone606c. In some embodiments, when the second lane is selected at the lane selector626, the user may control points or corners of each of the zones606(e.g., adding points, moving points, removing points, etc.) in order to define the zones606. Once the user has appropriately defined the corners of the zones606of the second lane, the user may select the first lane via the lane selector626and repeat the process for the first lane. The user may define correspondingly labeled zones (e.g., first zone, second zone, third zone, etc.) for lanes of each of the cameras114. For example, the zone604bof the first lane is proximate a window in the image data610shown in the editing window602a(e.g., the “Camera 2” image data). The user may similarly define the second zone604bas in front of the window for image data obtained from other cameras114that also include the window such that the labeling of the zones604and606is consistent across different camera114.

Referring still toFIG.6, the GUI600includes a first preview pane602b, a second preview pane602c, and a third preview pane602d. The second preview pane602cmay correspond go “Camera 2.” The first preview pane602bcorresponds to “Camera 1” and the third preview pane602ccorresponds to “Camera 3.” In some embodiments, in response to receiving a selection of different cameras114via the camera selector624, the GUI600provides the image data610of the selected camera114on the editing window602a. The user may then define the mask608and boundary634of the mask for each of the cameras114, as well as the zones604or606for the lanes, similarly as described above with reference to “Camera 2.”

In some embodiments, the GUI600includes a first side pane628(e.g., an information display area) corresponding to the mask608. The first side pane628displays information corresponding to points or pixels of the boundary of the mask634. The GUI600includes a second side pane630(e.g., an information display area) corresponding to the first lane and the zones604of the first lane. In some embodiments, the second side pane630illustrates which pixels or points of the corresponding camera114define the zones604(of the first lane). In some embodiments, the GUI600includes a third side pane632that illustrates which pixels or points of the corresponding camera114define the zones606(of the second lane). In some embodiments, the information displayed in the first side pane628, the second side pane630, and the third side pane632are unique for each of the cameras114. The information displayed in the first side pane628, the second pane630, and the third side pane632may be a part of the setup profile that is provided from the setup device302to the controller102for each camera114at the restaurant14.

Referring toFIG.7, a diagram700illustrates the image data610of the “Camera 2” shown inFIG.6in an on-line implementation of the setup profile, according to some embodiments. As shown inFIG.7, the image data610of the “Camera 2” includes masked out image data at mask608(e.g., clipped portions of the image data610, disregarded portions of the image data610, etc.). In some embodiments, the masked out image data, shown blocked by mask608, are disregarded by the controller102during implementation of image analysis techniques in order to improve the performance of the controller102.

Referring toFIG.8, various lanes of travel may be defined by the user or store owner about the store14. The lanes of travel can include, but are not limited to, a first order lane802a, a second order lane802b, a pickup lane802c, an emergency or exit lane802d(e.g., a fire lane), an entry lane802e, and a return lane802f. In some embodiments, the various lanes of travel include or are defined by paths804that extend through the lanes802. For example, the first order lane802aincludes a first order lane path804a, the second order lane802bincludes a second order lane path804b, the pickup lane802cincludes a pickup path804c, the emergency or exit lane802dincludes an exit path804d, the entry lane802eincludes an entry path804e, and the return lane802fincludes a return path804f. Each of the lanes802may include corresponding zones810defined along the paths804having unique identifiers. In some embodiments, the paths804and corresponding zones810are defined by the user during creation of the setup profile as described in greater detail above with reference toFIGS.3-7. The paths804and zones810may be consistently defined between different perspectives of cameras114such that the cameras114provide a comprehensive and unified tracking of vehicles30along the zones810and paths804.

The exit lane802dmay be an escape or emergency lane to provide a point of egress from the pickup lane802c. The exit lane802dmay be defined from a point of the pickup lane802cto a road or exit location surrounding the store14. In some embodiments, the return lane802fincludes the return path804fthat wraps around from the pickup lane802cto one or more parking spots808at a front of the store14. The parking spots808may each include corresponding zones for parking to await a carry out order. In some embodiments, the parking spots808are positioned at a pickup location806at a front of the store14.

It should be understood that the vision system200that is setup for the store14as shown inFIG.8may include cameras114configured to obtain image data of all of the paths804. The cameras114may be configured to provide the image data of each of multiple zones810along the paths804to the controller102during on-line implementation of the setup profile.

Online Implementation

Referring toFIG.9, the cloud computing system104may provide the setup profile to the controller102once the setup profile is fully generated. In some embodiments, the controller102and the vision system200are integrated into a POS system900of the drive through system10. The controller102is configured to use the setup profile in order to control operation of one or more POS units902. The controller102may also control operation of one or more alert devices904of the store14to provide feedback to employees of the store14. The controller102receives image data from the cameras114and uses the setup profile with the image data to track vehicles30through the drive through system10. In some embodiments, the controller102is configured to control operation of the POS units902based on the tracking and location of different vehicles30through the drive through system10.

The POS units902may be positioned at any windows (e.g., windows28) of a restaurant or store. For example, both pickup windows at which orders are fulfilled and payment windows at which customers pay for their items may include corresponding POS units902, in addition to other display screens. In another example, pickup windows (e.g., an order fulfillment window) may include both a display screen (e.g., a kitchen display screen, a tablet, a fulfillment display screen, etc.) and a POS unit902. The display screens and the POS units902may be operated by the controller102based on the image data from the cameras114in order to provide characteristics of the vehicles30(e.g., type and color) according to one or more modalities (e.g., textual information, visual information such as icon that represent the characteristics of the vehicles30). The display screens and the POS units902may be operated based on tracking the vehicles30in a virtual space based on the zones that are defined when setting up the profile for the image data of each of the cameras114. As described in greater detail above, the zones may be defined differently for the image data of each camera114in order to account for different perspectives, orientations, positions, fields of view, etc., of the cameras114which results in the image data of the zones spanning different areas of the image data (e.g., different pixel regions). The definition of the zones accounts for the different locations, sizes, etc., of the image data of the zones in the image data of the cameras114such that the controller102can consistently track vehicles30through the drive through system10from camera to camera. The definition of the zones facilitates seamless tracking of the vehicles30through the drive through system10despite different perspectives and positioning of the camera114.

The controller102may obtain the image data from each of the cameras114in real-time. In some embodiments, when a vehicle30first arrives at the drive through system10, the controller102may use an initial image of the vehicle30to determine (1) a color of the vehicle30, and (2) a type of the vehicle30. In some embodiments, the controller102is configured to implement an image analysis technique using the initial image of the vehicle30to determine (1) the color of the vehicle30, and (2) the type of the vehicle30. In some embodiments, the controller102includes different predetermined colors such as red, blue, orange, black, white, green, etc. In some embodiments, the controller102is configured to determine, using the image analysis technique and the initial image of the vehicle30, which of the predetermined colors to which the color of the vehicle30corresponds. In some embodiments, the controller102includes different predetermined types of vehicles such as car, truck, SUV, Jeep, etc. In some embodiments, the controller102is configured to determine, using the image analysis technique and the initial image of the vehicle30, which of the predetermined types of vehicle to which the type of the vehicle30corresponds.

In some embodiments, the controller102is configured to use the paths and zones along the paths in the setup file to track location of the vehicle30throughout a lifecycle of the vehicle30. In some embodiments, the controller102is configured to track which of the zones the vehicle30travels along or at which the vehicle30is currently located. For example, the controller102may identify that the vehicle30is traveling along the first order lane path804aby detecting the vehicle30at zones along the first order lane path804a. In some embodiments, the controller102is configured to use the image analysis techniques and one or more locations of interest (e.g., tagged or identified zones) to determine if the vehicle30has arrived at a specific location. The controller102may use the locations of interest (e.g., the tagged or identified zones) and the detection of the vehicles30at the locations of interest to operate corresponding POS units902, or the alert devices904. For example, in response to detecting that the vehicle30has arrived at a first order location, the controller102may operate corresponding POS units902to notify staff (e.g., order taking staff) that the vehicle30is awaiting service at a particular drive through unit108. The controller102may operate the POS units902to notify the order taking staff regarding the color and the type of the vehicle30that is at the particular drive through unit108. In this way, the vision system200(e.g., the image data of the cameras114, the image analysis and tracking techniques performed by the controller102, etc.) can be integrated with the POS units902of the store14. The POS units902may be display screens, the order taking stations110, etc.

In some embodiments, the POS units902may be operated by the order taking staff. POS data resulting from the POS units902may be provided to the controller102. The controller102may operate alert devices904responsive to at least one of the POS data obtained from the POS units902, or based on results of the image analysis techniques performed using the image data.

In some embodiments, the controller102is configured to operate a pickup alert device908or a carry-out alert device910. In some embodiments, the pickup alert device908is a display screen positioned proximate a window at which the vehicle30pulls up or arrives at in order to receive their order. The controller102may track the vehicle30to the window, and in response to the vehicle30arriving at the window, operate the pickup alert device908to display the color of the vehicle30and the type of the vehicle30that is at the window. In some embodiments, the controller102is also configured to operate the pickup alert device908to display POS data such as an order number, items in the order, a name of the customer in the vehicle30, etc. In this way, the POS data of the customer in the vehicle30may be tied to the detected color and type of the vehicle30, which may be displayed to staff as the staff fulfill the order to the customer in the vehicle30. Advantageously, displaying POS data (e.g., order information) in combination with characteristics of the vehicle30to receive the order (e.g., the type and color of the vehicle30) facilitates accurate order fulfilment and reduces a likelihood of the incorrect order or items being handed to customers.

Referring still toFIG.9, the controller102is also configured to operate the carry-out alert device910to notify carry out staff regarding a customer awaiting carry out. In some embodiments, for example, during busy times of the day, the staff may instruct customers in the vehicles30to pull around to the pickup location806. The controller102may receive an input from the POS units902that a specific customer has been asked to pull to the pickup location806. The controller102is configured to track the vehicle30of the specific customer as the vehicle30travels to the pickup location806, and identify a zone at which the vehicle30parks (e.g., which of the parking spots808that the vehicle30parks at). The controller102may operate the carry-out alert device910to notify the carry out staff regarding the POS data (e.g., the order data, a customer's first name, etc.), the characteristics of the customer's vehicle30(e.g., the color and type of the car), as well as which of the parking spots808that the vehicle30has parked at. In some embodiments, the carry-out alert device910is a display screen (e.g., a touch screen, a user interface, etc.). The carry-out alert device910may provide the POS data, the characteristics of the customer's vehicle30, and which of the parking spots808the vehicle30is located by presenting textual information, a graphical representation of the vehicle30and the corresponding parking spot808, or any combination thereof.

Referring still toFIG.9, the controller102may also operate the drive through unit(s)108and one or more audio communications devices912. In some embodiments, the audio communications devices are wearable headphones or wearable communications devices configured to be worn by staff or personnel of the store14in order to establish communications with the drive through units108(e.g., audio communications). The audio communications devices912may each include one or more speakers and one or more microphones for providing and obtaining audio (e.g., the microphones are configured to obtain audio input, and the speakers are configured to provide audio output). The drive through units108may similarly include speakers and microphones for providing a second end of audio communications. In some embodiments, the controller102is configured to implement one or more audio channels to facilitate bi-directional audio communication between the audio communications devices912and the drive through units108. In some embodiments, the audio communications device912are operated based on the vehicle30detected at an order location by the controller102to prompt the order taker to switch to a channel to take the customer's order.

The controller102may also be configured to count a number of customers (e.g., a number of vehicles30) that are in line awaiting order. In some embodiments, the number of customers that are in line at the drive through system10awaiting their turn to place an order is referred to as a “stack size.” In some embodiments, during busy times of day, the controller102may identify that a number of vehicles30have wrapped around the store14, into a parking lot, onto the street, etc. The controller102may use the image data provided by the cameras114of surrounding areas of the store14(e.g., the parking lot, an adjacent road, an entrance to the parking lot, etc.), and determine the stack size of the drive through system10. In some embodiments, the controller102is configured to record a time of day and corresponding stack size. The time of day and corresponding stack size may be used by the controller102in order to track busy times of day and initiate preparatory actions for preparing food and/or beverage items before the busy times of day.

In some embodiments, the controller102is configured to operate one or more kitchen alert devices906responsive to the stack size. In response to detecting a stack size above a threshold using the image data provided by the cameras114, the controller102may initiate cooking operations by prompting kitchen staff to prepare food and/or beverage items. The controller102may initiate the cooking operations or preparatory actions (e.g., a kitchen action) by operating the one or more kitchen alert devices906. In some embodiments, the controller102uses a predetermined set of instructions based on the stack size to determine which preparatory actions to initiate. For example, if the controller102includes historical data that 90% of customers typically order a specific type of food and/or beverage item that requires a significant amount of time to prepare, the controller102may prompt the kitchen staff, based on the stack size, to initiate preparation of a corresponding amount of the food and/or beverage item. In this way, the controller102may initiate kitchen options using the image data provided by the cameras114. In some embodiments, the controller102is configured to operate the POS unit(s)902or a manager display screen914to display the stack size of the drive through system10.

Referring toFIG.10, the POS system900is shown in greater detail. The controller102is shown to include includes processing circuitry1002, a processor1004, and memory1006. Processing circuitry1002can be communicably connected to a communications interface such that processing circuitry1002and the various components thereof can send and receive data via the communications interface. Processor1004can be implemented as a general purpose processor, an application specific integrated circuit (ASIC), one or more field programmable gate arrays (FPGAs), a group of processing components, or other suitable electronic processing components.

Memory1006(e.g., memory, memory unit, storage device, etc.) can include one or more devices (e.g., RAM, ROM, Flash memory, hard disk storage, etc.) for storing data and/or computer code for completing or facilitating the various processes, layers and modules described in the present application. Memory1006can be or include volatile memory or non-volatile memory. Memory1006can include database components, object code components, script components, or any other type of information structure for supporting the various activities and information structures described in the present application. According to some embodiments, memory1006is communicably connected to processor1004via processing circuitry1002and includes computer code for executing (e.g., by processing circuitry1002and/or processor1004) one or more processes described herein.

In some embodiments, controller102is implemented within a single computer (e.g., one server, one housing, etc.). In various other embodiments controller102can be distributed across multiple servers or computers (e.g., that can exist in distributed locations). For example, controller102can be made up of multiple controllers or microcontrollers that individually or collectively are configured to carry out operations as described herein.

The controller102is configured to obtain the setup profile from the cloud computing system104, and store the setup profile in the memory1006, shown as setup profile1020. The controller102may use the setup profile in order to implement the techniques described herein, or to configure the controller102to implement the techniques as described herein. In some embodiments, the controller102is also configured to provide POS data and performance metrics to the cloud computing system104. The controller102may also receive POS data or performance metrics of other drive through systems10from the cloud computing system104.

The controller102includes a database1008, an order taking manager1010, a communications manager1012, and a vision system manager1030, according to some embodiments. The database1008may be configured to store POS data, menu data, performance data, historical metrics of busy times, inventory or stock, personnel identification, etc. In some embodiments, the order taking manager1010is configured to retrieve menu data from the database1008for use in constructing a GUI of the POS units902, or to update the GUI of the POS units902(e.g., when an item is out of stock, when a price of the item has changed, etc.). In some embodiments, the order taking manager1010is configured to implement any of the techniques as described in U.S. application Ser. No. 17/325,871, now U.S. Pat. No. 11,244,681, filed May 20, 2021, the entire disclosure of which is incorporated by reference herein.

Referring still toFIG.10, the communications manager1012is configured to route traffic between the drive through units108and the wearable communications device112. In particular, the communications manager1012may define various channels such that the wearable communications devices112are configured to switch between the various channels. Each of the drive through units108may communicate on an assigned channel that the wearable communications device112can switch between. In particular, the communications manager1012is configured to route audio traffic from the drive through units108(e.g., drive through audio) to the wearable communications devices112and vice versa (e.g., the communications provided by the wearable communications devices112are provided to the drive through units108). In some embodiments, the switching of the wearable communications devices112is achieved by selection or input on the display screen of the POS units902.

The vision system manager1030is configured to integrate the vision system200(e.g., the cameras114) with the POS units902. The vision system manager1030may be configured to implement an image analysis technique1014using the image data obtained from the cameras114. In some embodiments, the image analysis technique1014is implemented using a neural network, a machine learning technique, a deep learning technique, an artificial intelligence, etc. The image analysis1014is configured to detect the presence of a vehicle in the image data provided by the cameras114as well as a color and type of the vehicle.

The vision system manager1030also includes a vehicle tracker1028. The vehicle tracker1028is configured to use the setup profile1020(e.g., the zones of the setup profile1020) and track the vehicles detected by the image analysis1014through the drive through system10. In particular, the vehicle tracker1028may use the detection of the color and type of the vehicle30provided by the image analysis1014in combination with a map including the zones. The vehicle tracker1028may output notifications of the vehicle30traveling into zones of interest. The vehicle tracker1028is configured to track position, speed, change in position, etc., of the vehicles30in the drive through system10.

The GUI manager1016is configured to use the tracking of the vehicles30provided by the vehicle tracker1028and the color and type of the vehicles30provided by the image analysis1014to generate or adjust the GUI that is provided to the POS units902. In particular, when a vehicle arrives at one of the drive through units108as identified by the vehicle tracker1028, the GUI manager1016may update a corresponding icon of the GUI that is displayed on the POS units902to visually indicate the color and type of the vehicle30. The GUI manager1016may also adjust the GUI of the POS units902such that the POS units902display textual information of the color and type of the vehicle30that is present at the drive through units108. The order takers (e.g., users of the POS units902) may select corresponding drive through units108via the POS units902in order to switch their wearable communications device112to the drive through unit108at which the vehicle30is detected.

In some embodiments, the order takers may initiate an order taking process for the customer of the vehicle30(e.g., by adding items to the order). Once the order taking process is complete, the POS unit902may provide the POS data (e.g., order information, order number, etc.) which is provided to the database1008and the vehicle tracker1028. The vehicle tracker1028may record an identifier of the POS data and tie the POS data with the vehicle30. In this way, as the image analysis1014continues to use newly obtained image data to identify the vehicle30in the image data, and the vehicle tracker1028tracks the vehicle30through the drive through system, the POS data (e.g., order data, customer name, items in order, total cost, etc.) may be linked with the vehicle30. Responsive to the vehicle tracker1028indicating that the vehicle30has arrived at a pay window or a pickup window (e.g., the first window28a, the second window28b, etc.), the GUI manager1016may operate POS units902to indicate that the vehicle30(with the linked POS data) has arrived at the pay window or the pickup window. Advantageously, the POS units902are integrated with the vision system200such that the POS units902can be updated responsive to tracking of vehicles30through the drive through system10. Similarly, the vehicle tracker1028may track vehicles through the drive through system10while linking the POS data of the vehicle30with the tracking. The GUIs that are generated by the GUI manager1016based on results of the vision system manager1030using the image data may be any of the GUIs described in greater detail below with reference toFIGS.12-15.

In some embodiments, the GUI manager1016is also configured to generate a manager GUI and operate the manager display914to provide the manager GUI. The manager GUI may be any of the GUIs described in greater detail with reference toFIGS.16-18. In some embodiments, the manager display914is a separate display screen that is presented to all personnel of the store14on a separate display screen. In some embodiments, the manager display914is a store manager or administrator's display screen that is accessible remotely (e.g., via the cloud computing system104). The manager GUI generally displays an overall map of the drive through system10, icons of detected vehicles along the map of the drive through system10that visually indicate colors and types of the vehicles, and POS data for each of the vehicles30(e.g., an indication).

In some embodiments, the POS data includes order data that may be selected and viewed by selecting the icons of the manager GUI. In some embodiments, the manager GUI is generated by the GUI manager1016using results of the vehicle tracker1028, the image analysis1014, the POS data, and the setup profile1020. The manager GUI may include a visualization (e.g., textual, an icon, etc.) of a stack size of vehicles30in the drive through system10. The stack size indicates a number of vehicles30that are beyond a certain point (e.g., upstream of the drive through units108, upstream of an entrance of the drive through system10, etc.) or between other vehicles30in the drive through system10. The manager GUI may also display various performance metrics or characteristics of the drive through system10. The manager GUI may also display vehicles30at a carry-out (take-out, pick-up) location (e.g., that have been ordered by staff to pull to a parking spot) and vehicles awaiting a mobile pickup order.

The vision system manager1030includes a performance manager1026that is configured to use results of the vehicle tracker1028to determine one or more performance characteristics of the drive through system10. In some embodiments, the performance manager1026is configured to identify, based on the results of the vehicle tracker1028, when a vehicle arrives at an entry point of the drive through system10. The performance manager1026may record a time at which each vehicle arrives at the entry point. The time at which the vehicle30arrives at the entry point may be linked to the particular vehicle30. The performance manager1026may also record a time at which the vehicles30each arrive at one of the drive through units108. The performance manager1026may also record a time at which the customers in the vehicles30begin their order (e.g., once the order taker switches to communicating via the drive through units108). The performance manager1026can also record a time at which the customers in the vehicles30complete their order. The performance manager1026may also record a time at which the customers in the vehicles30arrive at a first window (e.g., a payment window) and a second window (e.g., a pickup window). The performance manager1026may also record a time at which the customers in the vehicles30have their orders fulfilled. In some embodiments, the performance manager1026is configured to use any of the recorded times to estimate various quantities of elapsed time. For example, the performance manager1026may estimate a total amount of time that it takes each vehicle30to enter the drive through system10, place their order, pick up their order, and exit the drive through system10. The performance manager1026may determine an average amount of time it takes a vehicle to arrive, order, receive their order, and exit the drive through system10. In some embodiments, the performance manager1026is configured to track any of a number of vehicles that are commanded to pull over to a pickup location, a number of vehicles30that leave the drive through system10without ordering (e.g., due to the line being too long), an average amount of time that a customer waits at the drive through units108to begin placing an order, an average amount of time it takes a customer to place their order once initiated, number of vehicles30in the drive through system10, a number of customers served so far for the day, etc.

The performance manager1026may be configured to provide any of the performance characteristics or metrics described herein to the cloud computing system104. The performance manager1026may receive performance metrics or rankings of the store14from the cloud computing system104. For example, the ranking may indicate which place the store14is in relative to other stores in the area or in a chain. The ranking of the store14may be determined based on any of the performance metrics or characteristics described herein (e.g., average time for a customer to enter the drive through system10, place their order, receive their order, and leave the drive through system10).

In some embodiments, the performance manager1026is configured to provide any of the performance characteristics or metrics of the drive through system10to the GUI manager1016for display on the POS units902or the manager display914. In some embodiments, the performance manager1026is configured to store any of the performance metrics or characteristics of the drive through system10in the database1008. The performance manager1026may also provide ranking of the drive through system10to the GUI manager1016for display on the POS units902or the manager display914.

The vision system manager1030may include a prediction engine1024, an action manager1022, and an alert manager1018, according to some embodiments. In some embodiments, the prediction engine1024is configured to use historical (e.g., seasonal historical data) of demand at the drive through10to predict actions that should be taken by kitchen staff or personnel of the drive through system10. For example, the prediction engine1024may predict busy times, and provide the results of the predicted busy times of the store14to the action manager1022. The action manager1022is configured to use the results of the prediction engine1024in order to identify actions that should be taken to prepare for busy times of the store14, and prompt the alert manager1018. The alert manager1018is configured to provide alerts to the alert devices904to notify staff of the store14regarding the identified actions. Identified actions may include preparation of certain food and/or beverage items, an amount of food and/or beverage items to be prepared, sending out line busters with handheld POS units, opening new lanes, etc.

The action manager1022may use outputs from the prediction engine1024and may also use outputs from the performance manager1026. In some embodiments, outputs of the performance manager1026include the stack size, and indicate real-time or current data of the drive through system10as opposed to predicted data. In some embodiments, the action manager1022may defer to using results of the performance manager1026which indicate real-time performance metrics as opposed to using the outputs of the prediction engine1024. For example, if the prediction engine1024predicts, based on historical data, that there is a likelihood that ten customers are present in the drive through system10at a current time, but the image data indicates that twenty-five customers are present in the drive through system10(e.g., as indicated by the outputs of the performance manager1026), the action manager1022may use the results of the image data instead of the outputs of the prediction engine1024. In some embodiments, the action manager1022is configured to use a machine learning technique in order to determine one or more cooking or kitchen operations that should be performed responsive to number of customers in the drive through system10. For example, the action manager1022may use a known percentage of customers that will order a specific type of food and/or beverage item, and initiate preparation of a corresponding amount of the specific type of food and/or beverage item based on the number of customers in the drive through system10.

The action manager1022may also determine if it is appropriate to open an additional lane based on the current number of customers. In some embodiments, the action manager1022is configured to observe the average amount of time for a customer to enter the drive through system10, place their order, receive their order, and exit. In response to the average amount of time exceeding a threshold, the action manager1022may determine that the additional lane should be opened, and may either initiate the opening of the additional lane automatically, or may prompt a store manager to open the additional lane. Similarly, the action manager1022may initiate or prompt opening the additional lane in response to the total number of customers in the drive through system10exceeding a threshold or in response to the stack size exceeding a threshold.

The alert manager1018is configured to receive actions or determinations of the action manager1022, and operate the alert devices904to prompt staff to perform the action(s) determined by the action manager1022. For example, the alert devices904may be kitchen alert devices such that the alert manager1018initiates preparation of food and/or beverage items in accordance with the determinations of the action manager1022. Similarly, the alert devices904may be alert devices for order takers or staff personnel to open new lanes. In some embodiments, the GUI manager1016is configured to operate the manager GUI to prompt the manager to initiate opening the additional lane. The manager of the store may be presented with a request to open the additional lane, and provide via the manager display914confirmation to open the additional lane. Responsive to receiving the confirmation from the manager, the alert manager1018may operate the alert devices904to notify order taking personnel to open the additional lane.

Referring toFIG.11, a flow diagram of a process1100of operating a drive through system includes steps1102-1108, according to some embodiments. The process1100may be implemented by the controller102using local techniques without uploading image data to a cloud computing system. In some embodiments, the process1100illustrates implementation and use of the profile in process400. In some embodiments, the process1100as described herein is performed as step418of process400(e.g., implementing the profile).

The process1100includes providing a drive through system including a vision system having a camera and a controller (step1102), according to some embodiments. The vision system may be installed at a drive through of a store or restaurant. The vision system can include the camera configured to obtain image data of different lanes, pickup windows, order locations, exit lanes, emergency lanes, parking lots, etc. In some embodiments, the vision system includes multiple cameras that are positioned along different lanes of travel. The cameras may be positioned at locations, heights, and orientations as suggested or determined in step406of process400.

The process1100includes detecting a vehicle using the image data (step1104), according to some embodiments. In some embodiments, step1104includes obtaining the image data at the controller from the camera. Detecting the vehicle may include identifying a color (e.g., black, red, blue, purple, green, brown, white, etc.) and a type of the vehicle (e.g., compact car, sedan, SUV, truck, etc.). In some embodiments, step1104includes using an image analysis technique (e.g., a machine learning technique, an artificial intelligence, a neural network, etc.) in order to predict the color and type of the vehicle in the image data. Step1104can also be used to detect a location of the vehicle. In some embodiments, step1104includes detecting which of multiple zones along various routes of the drive through system at which the detected vehicle is currently located. In some embodiments, step1104is performed simultaneously with steps1106and1108in real-time in order to track the vehicle's location. Step1104may also include detecting a speed or motion of the vehicle. Step1104may also include detecting if the vehicle is stopped or parked. In some embodiments step1104is performed by the controller (e.g., controller102).

The process1100includes operating a POS unit based on the detection of the vehicle (step1106), according to some embodiments. In some embodiments, step1106includes operating the POS unit of an order taker in order to notify the order taker regarding the detection of the vehicle. For example, the POS unit may be operated to notify the order taker that the vehicle has arrived at an order location. In some embodiments, operating the POS unit includes prompting the order taker to initiate an order process. In some embodiments, operating the POS unit includes providing a notification of the color and type of the vehicle. In some embodiments, operating the POS unit includes providing a graphical representation of the vehicle that visually illustrates both the color and type of the vehicle. Step1106may include notifying an order taker to switch to an audio channel to take the customer's order. Step1106may be performed by the controller102.

The process1100includes operating a display screen based on the detection of the vehicle (step1108), according to some embodiments. The display screen may be a display screen positioned within a kitchen of the store or restaurant, a display screen for a store owner or manager, a display screen at a station (e.g., an order take out station), etc. The alert may include providing a graphical representation of all vehicles in the drive through system. The alert may additionally or alternatively include recommendations to open a new lane (e.g., a store manager operation), perform cooking operations (e.g., a kitchen operation), carry out an order to a customer that is parked in a pickup zone (e.g., a take out operation), etc. The alert may additionally or alternatively include providing notifications regarding the color and type of the detected vehicle at a particular location. In some embodiments, the alert includes POS or order data provided simultaneously with at least one of a snapshot of the detected vehicle or the color and type of the detected vehicle. For example, the display screen may be positioned at an order pickup window facing staff that has the responsibility of handing orders (e.g., food and/or beverage items) to customers at the order pickup window. Step1108may include operating the display screen to provide both (1) order data (e.g., POS data, order number, order identification, first name of customer, etc.) and (2) vehicle data (e.g., a snapshot of the vehicle, the color and type of the vehicle, etc.). Providing the order data and the vehicle data simultaneously can facilitate improved order fulfillment accuracy and reduce a likelihood that one customer's order is handed to another customer.

POS GUIs

Referring toFIGS.12-15, the POS unit(s)902may be configured to display various graphical user interfaces (GUIs) provided by the controller102. The controller102may operate the POS units902to display the GUIs using the image data provided by the cameras114. Advantageously, the controller102is configured to integrate the vision system200(e.g., the image data and the analysis performed by the controller102based on the image data) with the POS units902.

Referring toFIG.12, a first GUI1200includes a lane pane1202(e.g., a lane window), a windows pane1204(e.g., a windows lane), a carry out pane1206, and an order take out pane1208, according to some embodiments. In some embodiments, the lane pane1202is configured to display data regarding each of multiple lanes. In particular, the lane pane1202includes an entry, box, window, icon, widget, or pane corresponding to each drive through unit108of different lanes. The windows pane1204may provide data regarding customers that are present at multiple pickup windows of the drive through system10. The carry out pane1206may provide data regarding carry out orders (e.g., customers parked in parking spots808). The order take out pane1208may provide data regarding various take out orders.

The lane pane1202may be configured to display data corresponding to each of the drive through units108and the corresponding vehicles30that are present at each of the drive through units108(either currently placing an order, or awaiting to place an order). In particular, the lane pane1202may display both the color and the type of the car that is present at the drive through units108as determined by the controller102using the image data. For example, as shown inFIG.12, a blue SUV is currently present at the drive through unit108of lane3, and a black car is present at the drive through unit108of lane1. The lane pane1202may be updated in real-time by the controller102based on determinations by the controller102using the image data. For example, when the controller102determines, using the image data, that a blue SUV has arrived at the drive through unit108of lane3, the controller102may adjust the GUI1200of the POS units902in order to notify the order takers that a blue SUV has arrived at the drive through unit108of lane3. In some embodiments, the lane pane1202includes multiple selectable icons that display the color and type of the vehicles30that are present at the drive through units108of each lane. In some embodiments, GUI1200is configured to display vehicle information relating to the characteristic of the vehicle and order information for an order received from an occupant of a drive through to provide compiled order information. The vehicle information and order information may be processed at a single one of the POS units902or by multiple POS units902. Further, controller102is configured to update a display of the compiled order information during a service journey of the vehicle at the drive through. The service journey corresponds to the total course from entry to exit at the drive through.

Referring toFIGS.13-14, the GUI1200may be configured to open a new window or widget1218responsive to selection of one of the icons of the windows pane1202. In some embodiments, the widget1218includes a check in button1210(e.g., icon) and a new order button1212. For example, the order taker may select the check in button1210in order to initiate checking in a customer that has placed an online or mobile order and has arrived at the drive through system10for pickup. The order taker may select the new order button1212in order to initiate a POS and add items to the customer's order. In some embodiments, the widget1218includes vehicle information1214, and an option1216to change the vehicle information1214. In some embodiments, the vehicle information1214is populated by the controller102using the detected characteristics of the vehicle30(e.g., the color and type of the vehicle). Once the POS has been completed and the order has been placed, the POS data (e.g., the order data) may be tied to the corresponding vehicle information1214throughout the rest of the vehicle30's lifecycle through the drive through system10.

The GUI1200may similarly be provided to fulfillment staff via the pickup alert device908(e.g., a display screen, a touch screen, etc.). However, the GUI1200as provided to the fulfillment staff may exclude the options for adding items to the customer's order. The GUI1200may present both the type and color of the vehicle30that is currently present at the corresponding window (e.g., the red truck at window1) along with order information that is tagged to the characteristics of the vehicle30(e.g., the type and color).

Referring toFIG.15, the windows pane1204may include multiple selectable icons corresponding to multiple pickup windows of the store14. The selectable icons (e.g., widgets, boxes, buttons, etc.) may include vehicle information, similar to the lane pane1202indicating both the color and type of vehicle30that is currently present at the windows of the store14. The vehicle information of the windows pane1204may be populated or updated by the controller102based on the determinations obtained by the controller102using the image data. When the vehicle30arrives at the window (e.g., “red truck” shown at “Window1” inFIG.15), the staff at the corresponding window (e.g., Window1) may select the corresponding icon. Responsive to selection of the corresponding icon, the GUI1200displays an order pane1220. The order pane1220can include vehicle information1222illustrating the color and type of the vehicle30that is present at the window. The order pane1220can also include a check in button1224and an add order button1226, similar to as shown inFIGS.13-14. The order pane1220can also include order information1228and a serve button1230. In this way, the GUI1200is configured to display the order information1228in combination with the vehicle information1222. Advantageously, providing both the order information1228in combination with the vehicle information1222facilitates reduced likelihood of incorrect order fulfillment.

Referring again toFIGS.12-15, the lane pane1202and the windows pane1204may provide textual information of the vehicle data (e.g., the color and type of vehicle) as well as graphical information of the vehicle data. For example,FIG.12illustrates an icon or graphical illustration of a black car, a blue SUV, and a red truck.

Tracking GUI

Referring toFIGS.16-18, the controller102may operate a display screen (e.g., the manager display914) to include a GUI that illustrates live positions of all vehicles at the drive through system10. The GUI may be an overall or universal GUI that is determined by the controller102based on the image data of the cameras114.

Referring particularly toFIG.16, the controller102may generate and operate the manager display914to provide a GUI1600. The GUI1600may include a graphical representation of the store14, shown as store icon1602. The GUI1600may also include graphical representations of vehicles30in the drive through system10, shown as vehicle icons1618. In some embodiments, the vehicle icons1618are selected in order to visually indicate both the color and type of the vehicle as detected by the controller102. The vehicle icons1618are located and adjusted in their location in real-time corresponding to live detection of the vehicles30in the drive through system10.

The GUI1600may include menu icons1604, corresponding to the drive through units108. In particular, the GUI1600may include menu icon1604aand menu icon1604bcorresponding to two different drive through units108. The GUI1600illustrates which vehicle is currently located at the drive through units108. The GUI1600may also include a pay window icon1606and a pickup window icon1608. The GUI1600may visually indicate which of the vehicles30are at the pay window1606and the pickup window1608. The GUI1600can also include a finish line1612that visually indicates which customers are leaving the drive through system10after a successful order. In some embodiments, the GUI1600also visually illustrates vehicles30that have been requested to pull to a pickup location, shown as pickup spots1614. The GUI1600may place vehicle icons1618corresponding to vehicles30that are currently detected at the pickup spots1614. In some embodiments, the GUI1600also visually illustrates vehicles30that are awaiting mobile order pickup, shown as mobile order pickup spots1616.

Referring still toFIG.16, the GUI1600may also include stack size indicators, shown as stack size icon1620. The stack size icon1620visually illustrates a number of customers that are present between two points, or beyond a certain point. As shown inFIG.16, the stack size icon1620illustrates that seven vehicles are present between the pay window and the pickup window, visually illustrated by pay window icon1606and pickup window icon1608.

Referring toFIG.17, the controller102may generate and operate the manager display914to provide a GUI1700. The GUI1700may be similar to the GUI1600. The GUI1700may include vehicle icons1718, menu icons1704, a finish line1712, a pay window icon1606, and a pickup window icon1708. Similarly to GUI1600, the GUI1700can include pickup spots1714and mobile order pickup sports1716. The GUI1700also includes a stack size1720that indicates a number of customers that are beyond a certain point (e.g., vehicles30that are in line for the drive through system10upstream of the point at which the stack size1720is displayed).

Referring toFIG.18, the GUI1700is shown in a simplified view, illustrating only one lane of traffic of the drive through system10. The GUI1700may also present a drive through performance tab1740illustrating various performance characteristics of the drive through system10. In particular, the drive through performance dab1740may include a car count1742(e.g., a number of vehicles that are currently in the drive through system10), a total car count1744(e.g., a total number of vehicles or customers that have been tracked through the drive through system10for the day so far), an average menu time1746(e.g., an average amount of time that a customer is at the drive through unit108), and a number of lost customers1748(e.g., a number of customers that leave the drive through system10due to long lines). The GUI1700may also include a performance dashboard1750(e.g., a window, a pane, an area, a report, etc.) that includes a performance icon1752indicating overall performance of the drive through system10. In particular, the performance dashboard1750may indicate an average amount of time1754for a customer to enter, order, pay, receive their order, and exit the drive through system10. In some embodiments, the performance dashboard1750includes a leaderboard1756that indicates a ranking of the drive through system10of the store14that is illustrated by the GUI1700, relative to other drive through locations. The ranking of the leaderboard1756may be determined based on the average amount of time1754.

Point of Sale and Kitchen GUIs

Referring toFIGS.19-31, the controller102may operate any of the POS units (e.g., POS units902) to display the GUIs1800shown inFIGS.19-31at least partially using the image data obtained from the cameras114. In particular, the GUIs can display a screen for a director of a drive-through system to view, for example, a total number of mobile orders, and identify the source of particular mobile orders, e.g., from different services. For each mobile order, a user, a processor, a unique order identifier, a time of ordering, and a total amount can be displayed. The GUIs can also display, simultaneously with the order information, a lane drop-down menu and a window drop-down menu, allowing for selection of a particular lane and/or window of interest to a user to be selected to display additional information. As shown inFIGS.19-31, the POS units can be operated to provide GUIs in real-time that are updated based on identified visual characteristics of the vehicles30in the drive through system10and updated as the vehicles30are tracked through the drive through system10. For example, as shown inFIG.21a blue truck arrives at lane3, an order is placed, and the GUI is updated when the blue truck pulls around to the second window as shown inFIG.31. Accordingly, the GUI can display, for example, a detected vehicle, a check-in button indicating the customer in the detected vehicle has checked in, and a corresponding order. In addition, mobile order information can also be displayed.

Referring particularly toFIG.19, the GUI1800includes a lanes pane1802, a windows pane1804, a mobile orders pane1806, and one or more delivery service panes, shown as delivery service pane1808and delivery service pane1810(e.g., third party delivery services). When a vehicle arrives at the drive through unit108at the second lane (e.g., a white SUV), the GUI1800is updated in real-time as shown inFIG.20. In some embodiments, all of the POS units902that display the GUI1800are updated in real-time responsive to detection of vehicles in the drive through10. As shown inFIG.21, another vehicle, a blue truck, has arrived at the third lane. An icon of the vehicle may indicate in real-time whether another order taker has switched to communicating (e.g., via audio communications) with the vehicle and is taking the customer's order. As shown inFIG.22, when the user of the POS unit902on which the GUI1800is displayed selects the blue truck icon at the third lane, the GUI1800presents various options1812. The options1812include a button1818to change a characteristic of the detected vehicle (e.g., if the vehicle that is detected by the vision system200has a different characteristic than what is identified by the vision system200). For example, if the vehicle detected is not a blue truck and is instead a green SUV, the user may select the button1818to correct the characteristics detected by the vision system200.

Referring still toFIG.22, the options1812also include a check-in button1814, and a new order button1816. The user may select the check-in button1814and select an appropriate mobile order from the mobile order pane1806or one of the delivery panes1808and1810if the customer in the blue truck is picking up an order that has already been placed (e.g., by a mobile application or for delivery). If the customer in the blue truck is a new customer that has not yet placed an order, the user may select the new order button1816.

Responsive to selecting the new order button1816, the user (e.g., the order taker) is presented with a menu, shown inFIG.23. The order taker may select items from the menu in order to add to the customer's order. As shown inFIG.23, the order menu may include a selectable switch1824(e.g., a toggle button) to select between breakfast, lunch, and dinner menus. The order menu may also include a ribbon of item options1822including burgers, drinks, sides, meals, combos, etc. Selection of the different items options1822causes the menu to display different menu items that can be added to the customer's order. As shown inFIG.23, the combos menu option is selected, and the menu displays combo items1826and limited time offers1828. The order taker may select these items and add them to the customer's order as instructed by the customer. The order taker may also select between a carry out option and a dine in option for the new order.

Referring toFIG.24, responsive to selecting items to add to the customer's order, the menu displays a current order1820including items that has been added to the order, and a cost associated with each item. The current order1820may mirror or reflect an order that is displayed to the customer via the drive through unit108in real-time. Once the customer's order is completed, the order taker may select a pay or complete order button.

Referring toFIG.25, once the customer's order has been completed, the associated icon for the blue truck includes an indicator bar (e.g., a green indicator bar) to show that the customer has completed an order. The indicator may be provided on all the POS units902that display the GUI1800so that other order takers are informed that the customer in the blue truck has already placed their order and does not require further assistance at that lane. At this point, the order taker may, for example, select the icon for the green car as shown inFIGS.26and27or the white SUV as shown inFIG.25to start the next order.

Referring toFIGS.28and29, the white SUV which was previously in lane2(shown inFIG.5) has now pulled around to the third window, shown in the windows pane1804in order to place an order at the window in order to pay. As shown inFIG.31, the blue truck has pulled up to the second window in order to pay and pickup their order. The flow of traffic from the drive through units108to the pickup windows is updated in real-time across all the POS units902based on tracking of the vehicles by the vision system200. When a vehicle (e.g., the white SUV) pulls up to the third window, the windows pane1804indicates the presence of the white SUV by providing a textual representation (e.g., “White SUV”) and a graphical representation (e.g., an icon showing a white SUV). Responsive to selection of the pane or icon corresponding to the white SUV, the GUI1800presents the options1812to the user. The GUI1800also presents an order pane1820which indicates an order number, a total amount of time the vehicle has been in the drive through10, and a required payment amount.

Responsive to the blue truck arriving at the second window, shown inFIG.31, the GUI1800updates in real-time to show the user which window the blue truck is currently at. The user may select the blue truck icon to view the order information of the customer in the blue truck, similarly toFIG.30for the white SUV. In some embodiments, the GUI1800shown inFIGS.25-31illustrate a director screen for a manager or director of the restaurant to oversee drive through activity and details. In some embodiments, the GUI1800shown inFIGS.19-24illustrate a screen that is presented to an order taker or cashier of the drive through10.

Referring toFIG.32, a kitchen GUI1900illustrates various orders that may be presented to kitchen staff (e.g., an employee of the drive through10preparing orders). The operation of the kitchen GUI1900is coordinated with order placement at the POS units902that implement the GUI1800described in detail above with reference toFIGS.19-31. The controller102may configured to operate the kitchen display screen (e.g., a display device, the kitchen alert devices906, a tablet, etc.) to display the kitchen GUI shown inFIG.32. As shown inFIG.32, the kitchen display screen is operated in real-time to include newly placed orders and associated visual characteristics of the vehicles30for the orders. The kitchen GUI1900can be updated in real-time based on operations at the POS units (e.g., updated to include items that are added to the order). The kitchen screen GUIs indicate both the visual characteristics of the vehicles30(e.g., “blue truck”) as well as order information that the customers in the vehicles30have placed. When an order is fulfilled or the items have been prepared, the controller102may operate the kitchen display screen(s) to remove corresponding widgets, panes, or windows (e.g., as shown inFIGS.37and38where the grey car order window is removed).

As shown inFIG.32, the kitchen GUI1900includes various widgets1902-1908that are populated in real-time responsive to order placement. Each of the widgets1902-1908corresponds to a different order and lists the various items in the order. For example, the widget1902illustrates a burger meal and a chicken sandwich, as well as a lane that the vehicle is in (e.g., the white SUV), an identification of the vehicle (e.g., “white SUV”), an order number, whether the vehicle is in the drive through or not, and whether the order has been paid for or not. Likewise, the widget1904is populated for the blue truck responsive to order placement at the POS unit902, the widget1906is populated for the green car, and the widget1908is populated for the grey car. In some embodiments, the widgets1902-1908are updated in real-time based on feedback from the vision system200. For example, when the white SUV arrives at the third window (FIG.28), the widget1902may be updated to read “WINDOW 2” instead of “LANE 2.”

Referring toFIGS.12-15,16-18, and19-32, the controller102may be configured to operate the POS units902, the kitchen alert devices906(e.g., kitchen display screens), and other display screens (e.g., the manager display914) to provide various GUIs in a coordinated manner based on the image data and tracking performed based on the image data. As shown inFIGS.16-18, the manager display914may be operated provide a tracking GUI that illustrates visual indications of the locations of vehicles30in the drive through system10. As the vehicles30travel through the drive through system10(that is, as they transit through the drive through on their service journey), the tracking GUI may be updated in real-time to reflect current location and amount elapsed time since the vehicles30have entered the drive through system10(e.g., indicated by the status ring with increasing shaded region surrounding the icons of the vehicles30). When the vehicles30arrive at the drive through or order units108, the controller102may update the GUIs of the POS units902(shown inFIGS.12-15and19-31) to indicate that the vehicles30have arrived at the drive through units108as well as visual characteristics of the vehicles30. As the customers in the vehicles30place their orders and the order takers enter order information via the POS units902, the kitchen GUI1900(shown inFIG.32) may be operated by the controller102to indicate the items added to the order and corresponding visual characteristics of the vehicle30of the customer (e.g., type and color of vehicle). When the vehicles30arrive at pickup windows, the POS GUI (GUI1800) may be updated to notify order takers or order fulfillment personnel regarding the vehicle30at the windows as well as the visual characteristics of the vehicle30at the window.

It should be understood that any of the GUIs described herein with reference toFIGS.19-32may be implemented by the controller102and provided on a variety of display screens throughout the drive through10in a coordinated manner. The GUIs described herein with reference toFIGS.19-32may be the result of detecting and tracking vehicles30through the drive through system10in order to provide real-time feedback or prompts to kitchen staff, order taking staff, fulfillment staff, restaurant managers, analysis systems, etc.

Configuration of Exemplary Embodiments

It should be noted that the term “exemplary” and variations thereof, as used herein to describe various embodiments, are intended to indicate that such embodiments are possible examples, representations, and/or illustrations of possible embodiments (and such terms are not intended to connote that such embodiments are necessarily extraordinary or superlative examples).

The term “coupled,” as used herein, means the joining of two members directly or indirectly to one another. Such joining may be achieved with the two members coupled directly to each other, with the two members coupled to each other using a separate intervening member and any additional intermediate members coupled with one another, or with the two members coupled to each other using an intervening member that is integrally formed as a single unitary body with one of the two members. Such members may be coupled mechanically and/or electrically.

The hardware and data processing components used to implement the various processes, operations, illustrative logics, logical blocks, modules and circuits described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose single- or multi-chip processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, or, any conventional processor, controller, microcontroller, or state machine. A processor also may be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

In some embodiments, particular processes and methods may be performed by circuitry that is specific to a given function. The memory (e.g., memory, memory unit, storage device, etc.) may include one or more devices (e.g., RAM, ROM, Flash memory, hard disk storage, etc.) for storing data and/or computer code for completing or facilitating the various processes, layers and modules described in the present disclosure. The memory may be or include volatile memory or non-volatile memory, and may include database components, object code components, script components, or any other type of information structure for supporting the various activities and information structures described in the present disclosure. According to an exemplary embodiment, the memory is communicably connected to the processor via a processing circuit and includes computer code for executing (e.g., by the processing circuit and/or the processor) the one or more processes described herein. References to “a” or “the” processor should be understood to encompass a plurality of processors individually or collectively configured to carry out operations as described herein.

Although the figures and description may illustrate a specific order of method steps, the order of such steps may differ from what is depicted and described, unless specified differently above. Also, two or more steps may be performed concurrently or with partial concurrence, unless specified differently above. All such variations are within the scope of the disclosure. Likewise, software implementations of the described methods could be accomplished with standard programming techniques with rule-based logic and other logic to accomplish the various connection steps, processing steps, comparison steps, and decision steps.

It is important to note that the arrangement of the system as shown in the various exemplary embodiments is illustrative only. All such modifications are intended to be included within the scope of the present disclosure. Other substitutions, modifications, changes, and omissions may be made in the design and arrangement of the exemplary embodiments without departing from the scope of the present disclosure.