Patent Description:
There are a wide variety of different types of mobile work machine such as agricultural vehicles and construction vehicles. Some vehicles include harvesters, such as forage harvesters, sugar cane harvesters, combine harvesters, and other harvesters, that harvest grain or other crop. Such harvesters often unload into carts which may be pulled by tractors or semi-trailers as the harvesters are moving. Some construction vehicles include vehicles that remove asphalt or other similar materials. Such machines can include cold planers, asphalt mills, asphalt grinders, etc. Such construction vehicles often unload material into a receiving vehicle, such as a dump truck or other vehicle with a receiving vessel.

As one example, while harvesting in a field using a forage harvester, an operator attempts to control the forage harvester to maintain harvesting efficiency, during many different types of conditions. The soil conditions, crop conditions, and other things can all change. This may result in the operator changing control settings. This means that the operator needs to devote a relatively large amount of attention to controlling the forage harvester.

At the same time, a semi-truck or tractor-pulled cart is often in position relative to the forage harvester (e.g., behind the forage harvester or alongside the forage harvester) so that the forage harvester can fill the truck or cart while moving through the field. In some current systems, this requires the operator of the forage harvester to control the position of the unloading spout and flap so that the truck or cart is filled evenly, but not overfilled. Even a momentary misalignment between the spout and the truck or cart may result in hundreds of pounds of harvested material being dumped on the ground, or elsewhere, rather than in the truck or cart.

The receiving vehicle often has more freedom to move relative to the harvester than the harvester has to slow down or speed up due to crop unloading. Thus, the operators of the receiving vehicle currently attempt to adjust to the harvester so that the receiving vehicles are filled evenly, but not overfilled. However, the operator of the harvester may unexpectedly stop the harvester (such as when the harvester head becomes clogged and needs to be cleared or for other reasons), so the operator of the receiving vehicle may not react quickly enough, and the receiving vehicle may thus be out of position relative to the harvester.

Other harvesters such as combine harvesters and sugar cane harvesters, can have similar difficulties. Also, construction vehicles can be difficult to operate while attempting to maintain alignment with a receiving vehicle.

Also, some current agricultural equipment is highly customizable by an operator. Therefore, even if the equipment is being operated properly, there may be room for improvement in the customization by the operator.

Some prior art documents describe combinations of forage harvesters and transport vehicle attempting to automatize crop transfer with giving an alert to an operator if automatic control is not possible such that the operator can continue with manual control (<CIT>), while <CIT> describes an operator display indicating the direction in which the relative position of spout and transport vehicle should be moved in order to avoid crop losses and <CIT> indicates the remaining space of a transport vehicle for filling with crop.

<CIT> describes a harvesting machine with a controller and a user interface showing a set of zones within a boundary shape of a storage compartment on a crop transport vehicle on a display. The controller can present a list of target unloading areas and if useful, the controller may instruct the operator to recalibrate the alignment of the conveyor outlet and the storage compartment.

<CIT> describes a loader positioning system with a sensor and visual indicators for centering a product being loaded from a loading vehicle into a receiving vehicle.

<CIT> shows a combine harvester with a display showing recommendations to the operator for changing the settings of the harvester.

The present discussion proceeds with respect to an agricultural harvester, but it will be appreciated that the present discussion is also applicable to construction machines or other material loading vehicles as well, such as those discussed elsewhere herein.

As discussed above, many harvesters are highly customizable by the operator. Some machines may therefore be operated in a sub-optimum way. Though the customizable settings are set by the operator to a level that meets the operators requirements, modifications to the settings may improve the performance of the machine in terms of efficiency, speed, or in other ways.

Therefore, one portion of the present description proceeds with respect to a system that stores tutorial content (such as tutorial videos), and dynamically detects tutorial selection criteria during operation of the harvester and selects tutorial content to recommend to the operator of the harvester based upon the detected tutorial selection criteria. The tutorial selection criteria may be any of a wide variety of different criteria that can be sensed by the system on the harvester and that may indicate that the customizable settings available to the operator can be improved. The detected tutorial selection criteria can be used to dynamically identify a tutorial (such as tutorial video), that may benefit the harvester. The identified tutorial can be suggested to the operator in a variety of ways. By dynamically it is meant, in one example, that the tutorial is identified during operation of the harvester based on a sensed variable that is sensed in real time or in near real time.

On some harvesters, automatic cart filling control systems have been developed to automate portions of the filling process. One such automatic fill control system uses a stereo camera on the spout of the harvester to capture an image of the receiving vehicle. An image processing system determines dimensions of the receiving vehicle and the distribution of the crop deposited inside the receiving vehicle. The system also detects crop height within the receiving vehicle, in order to automatically aim the spout toward empty spots and control the flap position to achieve a more even fill, while reducing spillage. Such systems can fill the receiving vehicle according to a fill strategy (such as front-to-back, back-to-front, etc.) that is set by the operator or that is set in other ways.

However, these systems may not always be operated in the best way. For example, if the operator controls the position of the spout to be higher or lower than is normal, then the field of view of the camera mounted on the spout may not capture the entire receiving vehicle. This can mean that the camera mounted on the spout may not capture the entire receiving vehicle. This can lead the automatic cart filling control system to be less accurate. Therefore, as one example, system described herein can detect that the camera field of view is not properly capturing the receiving vehicle and can suggest a tutorial to the operator which describes proper spout/camera position. This is just one example.

In addition, some current harvesters are provided with a machine synchronization control system. The harvester may be a combine harvester so that the spout is not moved relative to the frame during normal unloading operations. Instead, the relative position of the receiving vehicle and the combine harvester is changed in order to fill the receiving vehicle as desired. Thus, in a front-to-back fill strategy, for instance, the relative position of the receiving vehicle, relative to the combine harvester, is changed so that the spout is first filling the receiving vehicle at the front end, and then gradually fills the receiving vehicle moving rearward. In such an example, the combine harvester and receiving vehicle may have machine synchronization systems which communicate with one another. When the relative position of the two vehicles is to change, the machine synchronization system on the combine harvester can send a message to the machine synchronization system on the towing vehicle to nudge the towing vehicle slightly forward or rearward relative to the combine harvester, as desired. By way of example, the machine synchronization system on the combine harvester may receive a signal from the fill control system on the combine harvester indicating that the position in the receiving vehicle that is currently being filled is approaching its desired fill level. In that case, the machine synchronization system on the combine harvester can send a "nudge" signal to the machine synchronization system on the towing vehicle. The "nudge", once received by the machine synchronization system on the towing vehicle, causes the towing vehicle to momentarily speed up or slow down, thus nudging the position of the receiving vehicle forward or rearward, respectively, relative to the combine harvester.

However, these types of systems may not always react quickly enough when the harvester makes a sudden stop or change in direction. Also, this type of machine synchronization system is normally implemented on a subset of towing vehicles or other receiving vehicles that are used for harvesting operations. Older vehicles, for instance, may not be fitted with such a system. Therefore, the system described herein can detect an alert condition on the harvester (such as a sudden speed or direction change) and communicate the alert condition to a mobile app running on a mobile device on the receiving vehicle, so the mobile app can generate an alert output to the operator of the receiving vehicle.

Similarly, it may be that the receiving vehicle (instead of the harvester) may suddenly stop or change directions. In one example, the mobile app on the receiving vehicle senses these and other alert conditions on the receiving vehicle and sends an alert message to the control system on the harvester where an alert message is generated for the operator of the harvesters.

<FIG> is a pictorial illustration showing one example of a self-propelled forage harvester <NUM> (a material loading vehicle) filling a tractor-pulled grain cart (or receiving vehicle) <NUM>. Cart <NUM> thus defines an interior that forms a receiving vessel <NUM> for receiving harvested material through a receiving area <NUM>. In the example shown in <FIG>, a towing vehicle (e.g., a tractor) <NUM>, that is pulling grain cart <NUM>, is positioned directly behind forage harvester <NUM> and has a mobile device <NUM> which may be a smart phone, tablet computer, etc. either mounted in the operator compartment of tractor <NUM>, or carried by the operator of tractor <NUM>. Also, in the example illustrated in <FIG>, forage harvester <NUM> has a camera <NUM> mounted on the spout <NUM> through which the harvested material <NUM> is traveling. The spout <NUM> can be pivotally or rotationally mounted to a frame <NUM> of harvester <NUM>. Camera <NUM> can be a stereo-camera or a mono-camera that captures an image (e.g., a still image or video) of the receiving area <NUM> of cart <NUM>. In the example shown in <FIG>, the receiving area <NUM> is defined by an upper edge of the walls of cart <NUM>.

When harvester <NUM> has an automatic fill control system that includes image processing, as discussed above, the automatic fill control system can gauge the height of harvested material in cart <NUM>, and the location of that material. The system thus automatically controls the position of spout <NUM> and flap <NUM> to direct the trajectory of material <NUM> into the receiving area <NUM> of cart <NUM> to obtain an even fill throughout the entire length and width of cart <NUM>, while not overfilling cart <NUM>. By automatically, it is meant, for example, that the operation is performed without further human involvement except, perhaps, to initiate or authorize the operation.

For example, when executing a back-to-front automatic fill strategy the automatic fill control system may attempt to move the spout and flap so the material begins landing at a first landing point in the back of vessel <NUM>. Then, once a desired fill level is reached in the back of vessel <NUM>, the automatic fill control system moves the spout and flap so the material begins landing just forward of the first landing point in vessel <NUM>.

<FIG> is a pictorial illustration showing another example of a self-propelled forage harvester <NUM>, this time loading a semi-trailer (or receiving vessel on a receiving vehicle) <NUM> in a configuration in which a semi-tractor (that also has a mobile device <NUM>) is pulling semi-trailer <NUM> alongside forage harvester <NUM>. Therefore, the spout <NUM> and flap <NUM> are positioned to unload the harvested material <NUM> to fill trailer <NUM> according to a pre-defined side-by-side fill strategy. Again, <FIG> shows that camera <NUM> can capture an image (which can include a still image or video) of semi-trailer <NUM>. In the example illustrated in <FIG>, the field of view of camera <NUM> is directed toward the receiving area <NUM> of trailer <NUM> so that image processing can be performed to identify a landing point for the harvested material in trailer <NUM>.

In other examples, where machine <NUM> is a combine harvester, it may be that the spout <NUM> is not moved relative to the frame during normal unloading operations. Instead, the relative position of the receiving vehicle <NUM>, <NUM> and the combine harvester is changed in order to fill the receiving vessel <NUM> as desired. Thus, if a front-to-back fill strategy is to be employed, then the relative position of the receiving vessel, relative to the combine harvester, is changed so that the spout is first filling the receiving vessel at the front end, and then gradually fills the receiving vessel moving rearward. In such an example, the towing vehicle may not have any type of machine synchronization systems, as discussed above. Thus, it can be difficult for the operators of the harvester and the towing vehicle to communicate with one another. As discussed above, the operator of the towing vehicle may not be able to react to sudden changes in the speed or direction by the harvester. Sometimes the operators use horns or radios to try to communicate with one another but this can be ambiguous and confusing, especially when more than one harvester is operating in a field.

Referring again to the examples discussed above with respect to <FIG> and <FIG>, the present discussion proceeds with respect to an example in which a mobile device <NUM> (such as a smartphone, a tablet computer, etc.) is accessible by the operator of the receiving vehicle (e.g., the driver of the towing vehicle or semi-tractor). The mobile device <NUM> may be mounted within the operator compartment of the receiving vehicle, carried by the operator, or otherwise accessible by the operator. An alert detection system on harvester <NUM> detects alert conditions and sends the mobile device an alert message. The mobile device <NUM> then surfaces an indicator to the operator of the receiving vehicle or towing vehicle indicative of the alert message.

<FIG> is a pictorial illustration showing one example of an operator interface display <NUM> that can be displayed on a display mechanism <NUM>, for the operator in an operator compartment of forage harvester <NUM>. The display (or portion of it) can also be sent to the mobile device <NUM> for use by the operator of the receiving vehicle or towing vehicle (tractor <NUM> or the semi-tractor). The operator interface display <NUM> in <FIG> shows a view of images (or video) captured by camera <NUM>. The image(s) show material <NUM> entering trailer <NUM>. An image processing system on harvester <NUM> illustratively identifies the perimeter of the opening <NUM> in trailer <NUM> and also processes the image of the material <NUM> in trailer <NUM> to determine the fill height relative to opening <NUM>. The perimeter defining opening <NUM> can be visually enhanced by overlaying a visual overlay over the opening <NUM> so that the operator can easily identify the opening <NUM>, as it is being recognized by the image processing system.

It should also be noted that, in one example, forage harvester <NUM> may have an automatic fill control system (or active fill control system) which fills trailer <NUM> according to a fill strategy (such as a back-to-front fill strategy, front-to-back fill strategy, etc.). In that case, a current location indicator (such as indicator <NUM>) may be displayed to show the current location where material <NUM> is being loaded into trailer <NUM> through spout <NUM> and the direction that spout <NUM> is, or should be, moving relative to trailer <NUM> as the filling operation continues. It can be seen in <FIG>, for instance, that indicator <NUM> is an arrow pointing in the front-to-back direction. The location of arrow <NUM> on the representation of trailer <NUM> indicates the current fill position, while the direction of the arrow <NUM> indicates the direction that spout <NUM> will be moved relative to trailer <NUM> in executing the selected front-to-back fill strategy. Therefore, in one example, the streaming video and indicator <NUM> can be sent to mobile device <NUM> so the operator of the receiving vehicle can easily see the fill level in the trailer <NUM>, and the direction that trailer <NUM> needs to move relative to the harvester in order to execute an efficient filling operation. While the indicator <NUM> in <FIG> points in the direction that spout <NUM> is to move relative to trailer <NUM>, the indicator can also be reversed when shown on mobile device <NUM> to show the direction that the trailer <NUM> (and hence receiving vehicle <NUM>) is to move relative to harvester <NUM>. These are just some examples of how the operator interface display <NUM> can be generated, and other examples are also contemplated herein.

It may be that the operator of harvester <NUM> has a setting that is undesirable. For example, <FIG> is another example of an operator interface display <NUM> which can be generated for the operator of harvester <NUM> and/or sent to mobile device <NUM>. Some items are similar to those shown in <FIG> and they are similarly numbered. The display <NUM> shows that the position of spout <NUM> is too high so that the field of view of the camera <NUM> does not capture the entire opening <NUM> of the trailer <NUM>. This can cause the automatic fill control system to be less accurate. This undesirable position of the spout <NUM> can be detected and used to select a tutorial for recommendation to the operator of harvester <NUM>. The tutorial may, for instance, be a tutorial video that explains how to position the spout <NUM> and/or the camera <NUM>.

<FIG> is a block diagram showing one example of a material loading system (an agricultural system) <NUM> that includes mobile material loading machine which comprises agricultural harvester <NUM> in more detail. Agricultural harvester <NUM>, in the example shown in <FIG>, includes one or more processors or servers <NUM>, communication system <NUM>, data store <NUM> (which can include tutorials <NUM> and other items <NUM>), sensors <NUM>, fill control system <NUM>, remote application interaction system <NUM>, operator interface mechanisms <NUM>, tutorial condition detection system <NUM>, controllable subsystems <NUM>, alert condition detection system <NUM>, and other harvester functionality <NUM>. Sensors <NUM> can include automatic fill control sensors <NUM> that are used by fill control system <NUM>. Sensors <NUM> can include camera <NUM> (which may be a mono-camera, stereo-camera or another type of camera) and other sensors <NUM>. The other sensors can include such things as Doppler sensors, RADAR sensors, other image sensors or any of a wide variety of other types of sensors. Sensors <NUM> can also include spout position sensor <NUM> and flap position sensor <NUM>. Spout position sensor <NUM> illustratively senses the position of spout <NUM> relative to the frame of harvester <NUM>. Sensor <NUM> can do this by sensing the position of an actuator that drives movement of spout <NUM> relative to the frame of harvester <NUM>, or sensor <NUM> can be a rotary position sensor, a linear sensor, a potentiometer, a Hall Effect sensor, or any other of a wide variety of sensors that can sense the position of spout <NUM> relative to the frame of harvester <NUM>. Similarly, flap position sensor <NUM> can be a sensor that senses the position of the flap <NUM>. Thus, sensor <NUM> can be a rotary position sensor, a linear sensor, a potentiometer, a Hall Effect sensor, a sensor that senses a position of an actuator that drives movement of flap <NUM>, or any of a wide variety of other sensors.

Sensors <NUM> can also include machine synchronization sensors <NUM>. Sensors <NUM> can include relative position sensors <NUM> that sense the relative position of the harvester <NUM>, relative to the receiving vehicle. Such sensors can include RADAR sensors, Doppler sensors, image or other optical sensors, or a wide variety of other relative position sensors. The relative position sensors <NUM> can also include position sensors (such as a GPS receiver, or another GNSS sensor) that senses the position of harvester <NUM>. This can be used, in conjunction with another position sensor signal from a position sensor on the receiving vehicle, to determine the position of the two vehicles relative to one another. The machine synchronization sensors <NUM> can include other sensors <NUM>, and sensors <NUM> can include a wide variety of other sensors <NUM> as well.

Fill control system <NUM> illustratively controls operations of various parts of harvester <NUM> (and possibly the towing vehicle <NUM>) to fill the receiving vehicle <NUM>, <NUM>, as desired. Fill control system <NUM> can include automatic fill control system <NUM> (which, itself, can include fill strategy selector <NUM>, fill strategy implementation processor <NUM>, and other items <NUM>), manual fill control system <NUM> (which, itself can include manual position adjustment detector <NUM> and other items <NUM>), and/or machine synchronization fill control system <NUM>. Fill control system <NUM> can also include fill control signal generator <NUM> and other items <NUM>.

Remote application interaction system <NUM> can include connection controller <NUM>, communication controller <NUM>, incoming alert detection system <NUM>, and other items <NUM>. Operator interface mechanisms <NUM> can include interactive display mechanism <NUM> and a variety of other operator interface mechanisms <NUM>. Controllable subsystems <NUM> can include propulsion subsystem <NUM>, steering subsystem <NUM>, one or more spout actuators <NUM>, one or more flap actuators <NUM>, and other items <NUM>. <FIG> also shows that operator <NUM> can interact through operator interface mechanism <NUM> to control and manipulate agricultural harvester <NUM>. Further, <FIG> shows that harvester <NUM> is connected over network <NUM> to receiving vehicle <NUM>, <NUM>, towing vehicle <NUM>, tutorial playing service <NUM>, and/or it can be connected to other systems <NUM>. Before describing the overall operation of agricultural harvester <NUM> in more detail, a brief description of some of the items in agricultural harvester <NUM>, and their operation, will first be provided.

Communication system <NUM> can facilitate communication among the items of harvester <NUM> (such as over a controller area network (CAN) bus), communication with receiving vehicles <NUM>, <NUM>, towing vehicle(s) <NUM>, tutorial playing service <NUM>, and with other systems <NUM> over network <NUM>. Network <NUM> can be a wide area network, a local area network, a near field communication network, a Bluetooth communication network, a cellular communication network, a Wi-Fi network, or any of a variety of other networks or combinations of networks. Therefore, communication system <NUM> can use a controller area network (CAN) bus or other controller to facilitate communication of the items on harvester <NUM> with other items. Communication system <NUM> can also be different kinds of communication systems, depending on the particular network or networks <NUM> over which communication is to be made.

Operator interface mechanisms <NUM> can be a wide variety of different types of mechanisms. Interactive display mechanism <NUM> can be a display mechanism, such as that shown in <FIG> and <FIG>, or mechanism <NUM> can be a display mechanism on a mobile device <NUM>, such as a tablet computer, a smartphone, etc., that is carried by the operator <NUM> and/or mounted in the operator compartment of harvester <NUM>. Thus, interactive display mechanism <NUM> can be a touch sensitive display screen, a display screen that receives inputs through a point and click device, or other kinds of display mechanisms.

Other operator interface mechanisms <NUM> can include a steering wheel, levers, buttons, pedals, a microphone and speaker (where speech recognition and speech synthesis are provided), joysticks, or other mechanical, audio, visual, or haptic mechanisms that can be used to provide outputs to operator <NUM> or to receive inputs from operator <NUM>.

Controllable subsystems <NUM> can be controlled by various different items on harvester <NUM>. Propulsion subsystem <NUM> can be an engine that drives ground-engaging elements (such as wheels or tracks) through a transmission, hydraulic motors that are used to drive ground-engaging elements, electric motors, direct drive motors, or other propulsion systems that are used to drive ground-engaging elements to propel harvester <NUM> in the forward and rearward directions. Propulsion subsystem <NUM> can illustratively be controlled with a throttle to increase or decrease the speed of travel of harvester <NUM>.

Steering subsystem <NUM> can be used to control the heading of harvester <NUM>. One or more spout actuators <NUM> are illustratively configured to drive rotation or movement of spout <NUM> relative to the frame of harvester <NUM>. Actuators <NUM> can be hydraulic actuators, electric actuators, pneumatic actuators, or any of a wide variety of other actuators. Similarly, one or more flap actuators <NUM> are used to drive the position of flap <NUM> relative to spout <NUM>. The flap actuators <NUM> can also be hydraulic actuators, electric actuators, pneumatic actuators, or any of a wide variety of other actuators.

Fill control system <NUM> can use automatic fill control system <NUM> to perform automated fill control to automatically execute a fill strategy in filling one of the receiving vehicles <NUM>, <NUM>. Therefore, fill strategy selector <NUM> can detect a user input selecting a fill strategy, or another input selecting a fill strategy, and access data store <NUM> for a stored fill algorithm that can be executed to perform the selected fill strategy. For instance, where the selected fill strategy is a back-to-front strategy, the algorithm will direct filling of the receiving vehicle beginning at the back of the receiving vehicle and moving to the front of the receiving vehicle. Other fill strategies can be selected as well. Fill strategy implementation processor <NUM> receives inputs from the automatic fill control sensors <NUM>, spout position sensor <NUM>, and flap position sensor <NUM>, and generates an output to fill control signal generator <NUM> based upon the inputs from the sensors to execute the desired automatic fill control strategy. Fill control signal generator <NUM> can generate control signals to control any of the controllable subsystems <NUM> (or other items) to execute the fill strategy being implemented by fill strategy implementation processor <NUM>.

As discussed above, it may be that operator <NUM> has some of the settings set so that, while they are acceptable to the operator, there is room for improvement. In such a scenario, tutorial condition detection system <NUM> (described in greater detail below with respect to <FIG>) detects the suboptimal settings or customizations made by operator <NUM> and uses those to identify a tutorial which may either be a tutorial video <NUM> stored in data store <NUM>, or tutorial content that can be played by tutorial playing service <NUM>. Tutorial condition detection system <NUM> then generates control signals to control operator interface mechanisms <NUM> to display or otherwise present a recommendation of the identified tutorial to operator <NUM>. In one example, tutorial condition detection system <NUM> controls the interactive display mechanism <NUM> to display the tutorial recommendation. In another example, tutorial condition detection system <NUM> can use communication system <NUM> to communicate the tutorial recommendation to mobile device <NUM> which, in tum, generates a tutorial recommendation display for operator <NUM>.

Also, in some examples, it may be that agricultural harvester <NUM> changes speed or direction quickly, or changes another operational characteristic in a way that the operator of the receiving vehicle may have difficulty reacting to. In such an example, alert condition detection system <NUM> detects those conditions and generates an alert output. The alert output can be provided to communication system <NUM> for communication to a mobile app running on the mobile device <NUM> in the receiving vehicles <NUM>, <NUM> or towing vehicle <NUM>. The mobile device can then generate an alert display or another alert output (such as an alert sound, a haptic output, etc.) for the operator of the receiving vehicle or towing vehicle.

Remote application interaction system <NUM> can receive inputs from, among other things, alert condition detection system <NUM> indicative of an alert condition. Remote application interaction system <NUM> can then generate an output indicative of the alert condition that is communicated to a mobile application on mobile device <NUM>. Examples of mobile device <NUM> are described below. Suffice it to say, for now, that the mobile application on mobile device <NUM> can receive the output from remote application interaction system <NUM> and generate a display or a different output on an operator interface mechanism on mobile device <NUM> to communicate the alert condition to the operator of towing vehicle <NUM> (or a semi-tractor towing trailer <NUM>).

More specifically, remote connection controller <NUM> establishes a connection with the mobile device <NUM>. This can be done in a number of different ways. Communication controller <NUM> generates control signals to control communication system <NUM> to communicate the alert condition to the mobile device <NUM>. The mobile device <NUM> on the receiving vehicle may also generate an alert message based on an alert condition detected on the receiving vehicle and send the alert message to communication system <NUM> on harvester. The alert message is sent to incoming alert detection system <NUM> that outputs the alert message to operator interface mechanism <NUM> for operator <NUM>. The mobile device <NUM> on the receiving vehicle can also send the alert message to the mobile device <NUM> on harvester <NUM> as well.

<FIG> is a block diagram showing one example of a tutorial condition detection system <NUM>. System <NUM> can include one or more detectors <NUM> which can detect various alert conditions on harvester <NUM>. Detectors <NUM> can be any of the sensors <NUM>, or processing logic that processes the signals from those sensors or from different sensors. In the example shown in <FIG>, detectors <NUM> include camera view detector <NUM> that receives the signal from camera <NUM> and performs image processing to analyze the image to determine whether the field of view of camera <NUM> captures the entire opening of the receiving vehicle at which it is pointed. The camera view detector <NUM> can detect other characteristics of the image as well. Detectors <NUM> can include fill settings detector <NUM> that detects the settings to fill control system <NUM>, such as offset values that indicate a buffer around the border of the receiving vehicle where the filling operation is to take place, the fill strategy that is selected, the flap and spout position settings, or other settings that have to do with the filling operation. Detectors <NUM> can also include machine settings detector <NUM> that detects other machine settings, such as machine speed, machine heading, header height, or any of a wide variety of other machine settings. Detectors <NUM> can include other detectors <NUM> as well.

Tutorial condition detection system <NUM> can also include evaluation system <NUM>, tutorial identification system <NUM>, recency processing system <NUM>, output generator <NUM>, user experience (UEX) generation system <NUM>, video playing system <NUM>, and other items <NUM>. Evaluation system <NUM> can evaluate the outputs of detectors <NUM> to determine whether a tutorial selection criteria is present. For instance, if the field of view <NUM> is not capturing the entire opening of the receiving vehicle, as detected by camera view detector <NUM>, evaluation system <NUM> may determine that this is indeed a tutorial criteria that may be used to select a tutorial for presentation or suggestion to operator <NUM>. If the fill settings detector <NUM> detects that the offset values set by operator <NUM> are resulting in an inefficient filling operation or in other suboptimal operation, evaluation system <NUM> may determine that this represents a tutorial criteria as well. Evaluation system <NUM> may also evaluate the machine settings that are detected by machine settings detector <NUM> or other items detected by other detectors <NUM> to determine whether any tutorial suggestion criteria are present.

If evaluation system <NUM> determines that tutorial criteria are present, system <NUM> generates an output to tutorial identification system <NUM> that identifies one or more tutorials (e.g., tutorial videos <NUM> or tutorial information hosted by tutorial playing service <NUM>) for suggestion to operator <NUM>. Tutorial identification system <NUM> may be an artificial neural network, or another classifier, or another model that receives the tutorial criteria generated by evaluation system <NUM> as an input and identifies a tutorial as an output.

Recency processing system <NUM> may determine whether the tutorial identified by tutorial identification system <NUM> has recently been accessed or played on agricultural harvester <NUM>, on mobile device <NUM>, or for operator <NUM>. For instance, it may be that the operator <NUM> provides identification information to harvester <NUM> before beginning a harvesting operation. Recency processing system <NUM> can use the identity of operator <NUM> to determine whether the tutorial identified by tutorial identification system <NUM> has recently been suggested to, played by, or otherwise accessed by operator <NUM>. If the tutorial has been played by operator <NUM>, for instance, within a predetermined threshold time period, then recency processing system <NUM> may delete that tutorial from the tutorials that will be suggested or recommended to operator <NUM>.

Output generator <NUM> generates an output identifying the tutorials that are identified by tutorial identification system <NUM> that have not been recently suggested to or played by operator <NUM>, or the operators mobile device <NUM>, or harvester <NUM>. The output may be a path name identifying a path to the location where the tutorial is stored, as well as a description describing the tutorials, UEX generation system <NUM> then generates a user experience on one or more of operator interface mechanisms <NUM> or on other systems that are accessible by operator <NUM> to suggest the tutorials and to allow operator <NUM> to select those tutorials for being played. UEX generation system <NUM> may, for instance, display an actuator, such as a thumbnail, a link, or an icon that can be actuated by operator <NUM> to play the tutorial. Also, in one example, UEX generation system <NUM> can generate an output that allows the user to delay playing of the tutorial until a later time, that is more convenient for operator <NUM>.

Once operator <NUM> selects a tutorial for playing, video playing system <NUM> plays the tutorial video for the operator. For instance, video playing system <NUM> may access tutorial videos <NUM> from data store <NUM> and play the videos on an interactive display mechanism <NUM> or send them to mobile device <NUM> where they can be played. In another example, video playing system <NUM> can access the tutorial videos from tutorial playing service <NUM> and have the videos played through service <NUM>.

<FIG> is a block diagram showing one example of alert condition detection system <NUM>. Alert condition detection system <NUM> includes one or more detectors <NUM> that detect various alert conditions on harvester <NUM>. Similar alert condition detectors can be on the receiving vehicle as well, but they are described herein as being on harvester <NUM> for the sake of example. Detectors <NUM> can include CAN message detector <NUM> that detects alert conditions based upon messages on a CAN bus or similar bus on harvester <NUM>. Detectors <NUM> can include a speed/speed change detector <NUM> that detects the speed of harvester <NUM> or a change in the speed of harvester <NUM>. Detectors <NUM> can include direction/direction change detector <NUM> that detects the heading of harvester <NUM> or a change in the heading of harvester <NUM>. Detectors <NUM> can include one or more obstacle detectors <NUM> that can detect obstacles that harvester <NUM> is approaching or that have been encountered by harvester <NUM>. Obstacle detectors <NUM> can thus be one or more optical detectors, or mechanical detectors, or RADAR, or LIDAR, or other detectors that detect the presence of an obstacle. Detectors <NUM> can include clog detectors <NUM> that detect whether harvester <NUM> is clogged. Detectors <NUM> can include a wide variety of other detectors <NUM> that can detect alert conditions in other ways.

Speed/speed change detector <NUM> can detect the speed of rotation of an axel, or a transmission, or the output of an engine, or other items. Direction/direction change detector <NUM> can include accelerometers, a GNSS receiver, a cellular triangulation sensor, a sensor that senses a steering angle, or another sensor. Clog detector <NUM> can be a torque detector that detects increased torque on drive shafts, or an optical detector that detects clogging of harvester <NUM> or other detectors.

Alert condition detection system <NUM> also includes evaluation system <NUM>, alert output system <NUM>, and other items <NUM>. Evaluation system <NUM> can evaluate one or more outputs of detectors <NUM> to determine whether an alert condition exists. For instance, if the detected CAN message detected by detector <NUM> represents an alert condition, evaluation system <NUM> generates an output indicating that the alert condition exists. If speed/speed change detector <NUM> detects a rapid acceleration or deceleration of harvester <NUM>, evaluation system <NUM> may determine that this represents an alert condition. If any of the other detectors <NUM> generate outputs that represent an alert condition, the outputs are evaluated by evaluation system <NUM> indicative of a detected alert condition. Alert output system <NUM> can generate an output that identifies the alert, or the type of alert, that describes the alert, and that includes other information about the alert.

<FIG> is a block diagram of one example of a receiving/towing vehicle <NUM>. It will be appreciated that receiving/towing vehicle <NUM> can be tractor <NUM>, the semi-tractor pulling trailer <NUM>, or another vehicle that can be operated by an operator local to that vehicle. Receiving/towing vehicle <NUM> illustratively includes one or more operator interface mechanisms <NUM>, mobile device <NUM> (which may be carried by an operator of vehicle <NUM>, mounted within an operator compartment of vehicle <NUM>, etc.), and other receiving/towing vehicle functionality <NUM>. In the example shown in <FIG>, mobile device <NUM> illustratively includes one or more processors <NUM>, data store or other memory <NUM>, user interface mechanisms <NUM>, one or more sensors <NUM>, communication system <NUM>, application running system <NUM>, and other mobile device functionality <NUM>. Application running system <NUM> can run a mobile application <NUM> and it can include other application running functionality <NUM>.

Operator interface mechanisms <NUM> can include a steering wheel, pedals, joysticks, other visual, audio, haptic, or other interface mechanisms. User interface mechanisms <NUM> can illustratively include a display screen, a keypad, buttons, icons, a touch sensitive display screen, audio output mechanisms, a haptic output mechanism, or other interface mechanisms. Sensors <NUM> on mobile device <NUM> can include position sensors (such as a GPS receiver), accelerometers, inertial measurement units, or other sensors. The sensors can sense variables (such as change in direction, change in speed, loss of traction, etc.) indicative of an alert condition on the receiving vehicle. Communication system <NUM> can include a cellular communication system, a near field communication system, a Bluetooth communication system, WIFI, local or wide area network communication systems, or other communication systems or combinations of systems.

Application <NUM> can be downloaded by mobile device <NUM>, or it can be installed on mobile device <NUM> in other ways. In the example shown in <FIG>, mobile application <NUM> illustratively includes alert processing system <NUM>, tutorial recommendation processing system <NUM>, and other mobile app functionality <NUM>. Alert processing system <NUM> can include alert identifier <NUM>, alert UI control system <NUM>, and other items <NUM>. Tutorial recommendation processing system <NUM> can include tutorial recommendation identifier <NUM>, tutorial UI control system <NUM>, and other items <NUM>. Alert processing system <NUM> processes alert messages that are received from harvester <NUM>. Alert identifier <NUM> parses the alert message to identify the alert condition that is represented by the alert message. The identity of the alert condition may then be used by alert UI control system <NUM> to control user interface mechanisms <NUM> to generate an alert output for the operator of receiving/towing vehicle <NUM>. For instance, alert UI control system <NUM> can generate a flashing message on a display screen of mobile device <NUM>, an audible alert tone on a speaker of mobile device <NUM>, a haptic output from mobile device <NUM>, or another output. The output may describe the alert condition, and contain instructions for the operator of receiving/towing vehicle <NUM>.

In an example in which a tutorial is to be displayed or played on mobile device <NUM>, tutorial recommendation processing system <NUM> receives the tutorial recommendation output by tutorial condition detection system <NUM>. Tutorial recommendation identifier <NUM> identifies the one or more tutorials that are to be recommended, and tutorial UI control system <NUM> controls user interface mechanisms <NUM> to display the tutorial recommendation, along with actuators that can be actuated by the operator to play or otherwise access the tutorial information. For instance, control system <NUM> can generate an output showing a recommended tutorial, along with an actuator, such as a thumbnail, link, or icon that can be actuated by the operator to begin playing the tutorial, to dismiss the recommendation, to delay playing of the tutorial to a later time, or to take other actions.

<FIG> is a flow diagram illustrating one example of the operation of tutorial condition detection system <NUM>. It is first assumed that harvester <NUM> is loading material into a receiving vehicle. In the present discussion, it will be assumed that the receiving/towing vehicle is vehicle <NUM> shown in <FIG>. Having the harvester <NUM> loading material into receiving vehicle <NUM> is indicated by block <NUM> in the flow diagram of <FIG>.

Detectors <NUM> then detect one or more tutorial-related variables that can be used to determine whether a tutorial should be recommended to the operator. Detecting the tutorial-related variables is indicated by block <NUM>. Tutorial-related values can be based upon a camera view <NUM>, as detected by camera view detector <NUM>. The tutorial-related variables can be harvester settings <NUM> as detected by machine settings detector <NUM>, or fill settings <NUM>, as detected by fill settings detector <NUM>. The tutorial-related variables can be a wide variety of other variables <NUM> as well.

Evaluation system <NUM> then evaluates the tutorial-related variables to determine whether a tutorial is to be recommended and, if so, tutorial identification system <NUM> identifies the one or more tutorials to recommend, as indicated by block <NUM>. Evaluating the tutorial-related criteria and identifying the tutorials to recommend can be performed by a single model, or multiple different models or mechanisms. The evaluation system <NUM> and tutorial identification system <NUM> can be one or more artificial neural networks <NUM>, other classifiers <NUM>, such as rules-based classifiers, or other classifiers or models <NUM>.

If one or more tutorials are to be recommended, as indicated by block <NUM>, then recency processing system <NUM> can determine whether any of the set of one or more tutorials have recently been viewed or otherwise accessed by the operator, as indicated by block <NUM>. The recency processing system <NUM> can identify whether the tutorials have been viewed within a particular time window or time threshold, as indicated by block <NUM>. Recency processing system <NUM> can determine whether the tutorials have been viewed by the present operator <NUM>, or by any operator of this particular harvester <NUM>, or on a given mobile device, as indicated by block <NUM>. Recency processing system <NUM> can determine whether any of the set of tutorials have recently been viewed or accessed in other ways as well, as indicated by block <NUM>.

Recency processing system <NUM> can then filter any of the recently viewed tutorials from the set of tutorials that are to be presented or recommended to the operator, to identify a filtered set of tutorials. Filtering the recently viewed tutorials to obtain the filtered set of tutorials is indicated by block <NUM> in the flow diagram of <FIG>. Output generator <NUM> then generates an interactive tutorial output indicative of the filtered set of tutorials, as indicated by block <NUM>. The output can include a description or thumbnail of each of the tutorials <NUM>, a link or icon <NUM> to the tutorials, and other information <NUM>.

UEX generation system <NUM> then conducts a user experience (UEX) outputting the interactive tutorial output, as indicated by block <NUM>. The UEX can be conducted on an in-cab interactive display mechanism <NUM>, as indicated by block <NUM>. The UEX can be sent to a mobile device <NUM>, as indicated by block <NUM>. The UEX generation system <NUM> can detect operator interaction selecting a tutorial, as indicated by block <NUM>, and video playing system <NUM> can play the tutorial, as indicated by block <NUM>. The tutorial can be played from memory <NUM> on harvester <NUM> or from memory on the mobile device <NUM>. The tutorial can also be played from a tutorial playing service <NUM>. UEX generation system <NUM> can also detect an operator input to delay viewing of the tutorial, as indicated by block <NUM>, or to dismiss the tutorial recommendation output, as indicated by block <NUM>. UEX generation system <NUM> can generate the user experience output and detect user interactions in other ways as well, as indicated by block <NUM>.

Until the operation of the harvester is complete, as determined at block <NUM>, processing may revert back to block <NUM> where the tutorial-related variables are detected.

<FIG> shows one example, of a display mechanism <NUM> that can display a user interface display <NUM> recommending tutorials. The display <NUM> can include a plurality of thumbnails <NUM>-<NUM> that display information identifying the recommended tutorials, and that may act as a user actuatable link to the underlying tutorials. The display <NUM> can include more detailed information, or a wide variety of other pictorial or other information.

<NUM> is a flow diagram illustrating one example of the operation of alert condition detection system <NUM>. Again, it is first assumed that harvester <NUM> is loading material onto a receiving vehicle <NUM>, as indicated by block <NUM>. In the example discussed herein, the receiving vehicle has a mobile device <NUM> disposed thereon, as indicated by block <NUM>. The mobile device <NUM> may be held by the operator of the receiving vehicle, or mounted in an operator compartment of the receiving vehicle, or configured in other ways, as indicated by block <NUM>.

Detectors <NUM> then detect one or more alert-related variables, as indicated by block <NUM>. For instance, CAN message detector <NUM> may detect a CAN message <NUM>. Clog detector <NUM> may detect a clog <NUM>, speed/speed change detector <NUM> may detect a speed or a change in speed of harvester <NUM>, as indicated by block <NUM>, direction/direction change detector <NUM> may detect the direction or a change in direction of harvester <NUM>, as indicated by block <NUM>. Obstacle detector <NUM> may detect an obstacle <NUM> that has been or is about to be encountered by harvester <NUM>, or other detectors <NUM> can detect other alert-related variables, as indicated by block <NUM>.

Evaluation system <NUM> then evaluates the alert-related variables to determine whether an alert condition exists, as indicated by block <NUM>. The evaluation system <NUM> can be an artificial neural network, a rules-based classifier, or another classifier, or mechanism for evaluating the alert-related variables to determine whether an alert condition exists.

If an alert condition exists, as indicated by block <NUM>. Then alert output system <NUM> generates an alert output indicative of the alert condition, as indicated by block <NUM>. The alert output can include a description <NUM> of the alert, a suggested action <NUM> for the operator of the receiving vehicle to take based on the alert condition, and the alert output can include a wide variety of other information <NUM>.

Alert output system <NUM> can then output the alert message to communication system <NUM> which communicates the alert output to a mobile app <NUM> on mobile device <NUM> on the receiving vehicle as indicated by block <NUM>.

The mobile device <NUM> on the receiving vehicle <NUM> then receives the alert output at alert processing system <NUM>. Receiving the alert output on the mobile device <NUM> is indicated by block <NUM>. Alert identifier <NUM> identifies the alert and alert UI control system <NUM> controls the user interface mechanisms <NUM> on the mobile device to generate an output based on the alert condition, as indicated by block <NUM>. The output can be to flash the screen <NUM> of the mobile device, to display a message <NUM>, to generate an audible output <NUM> or a haptic output <NUM>, or to generate any of a wide variety of other outputs <NUM>.

Until operation is complete, as indicated by block <NUM>, operation returns to block <NUM> where one or more of the alert-related variables are again detected.

<FIG> shows one example of a user interface display <NUM> that is generated based on an alert condition on a mobile device. In the example shown in <FIG>, the mobile device is a tablet computer <NUM> with a user interface display screen <NUM>. Screen <NUM> can be a touch screen or a pen enabled interface that receives inputs from a touch input or from a pen or stylus. Computer <NUM> can also use an on-screen virtual keyboard. Computer <NUM> might also be attached to a keyboard or other user input device through a suitable attachment mechanism, such as a wireless link or USB port, for instance. Computer <NUM> can also receive and process voice or speech inputs as well.

<FIG> shows that the display <NUM> can display an alert in blinking letters. The alert can be an alphanumeric alert in large letters that blink and can also instruct the operator of the receiving vehicle <NUM> what to do. In the example shown in <FIG>, the alert message instructs the operator of the receiving vehicle <NUM> to stop. Of course, <FIG> shows only one example of an alert message, and a wide variety of other alert messages can be generated as well.

It will thus be appreciated that the present system can detect an alert condition which may result in the receiving vehicle and the harvester being out of position relative to one another to establish adequate material transfer from the harvester to the receiving vehicle. The condition can be detected and transmitted to the receiving vehicle where a display or other message can be generated for the operator of the receiving vehicle. Similarly, the alert condition can be detected on the receiving vehicle and sent for display to the operator of the harvester. The present description also describes a system that can detect tutorial-related criteria which may signify that a particular tutorial should be suggested to the operator of the harvester. The tutorial-related criteria are evaluated and any tutorials are identified. An output is generated for the operator to select recommended tutorials for viewing. Thus, the tutorials are dynamically selected based upon the tutorial-related criteria that are detected during operation of the harvester.

The present discussion has mentioned processors and servers. In one example, the processors and servers include computer processors with associated memory and timing circuitry, not separately shown. The processors and servers are functional parts of the systems or devices to which they belong and are activated by, and facilitate the functionality of the other components or items in those systems.

The displays can take a wide variety of different forms and can have a wide variety of different user actuatable input mechanisms disposed thereon. For instance, the user actuatable input mechanisms can be text boxes, check boxes, icons, links, drop-down menus, search boxes, etc. The mechanisms can also be actuated in a wide variety of different ways. The mechanisms can be actuated using hardware buttons, switches, a joystick or keyboard, thumb switches or thumb pads, etc. The mechanisms can also be actuated using a virtual keyboard or other virtual actuators. In addition, where the screen on which the actuators are displayed is a touch sensitive screen, they can be actuated using touch gestures.

<FIG> is a block diagram of harvester <NUM> shown in <FIG>, as well as receiving vehicle <NUM> and towing vehicle <NUM> except that they communicate with elements in a remote server architecture <NUM>. In an example, remote server architecture <NUM> can provide computation, software, data access, and storage services that do not require end-user knowledge of the physical location or configuration of the system that delivers the services. In various examples, remote servers can deliver the services over a wide area network, such as the internet, using appropriate protocols. For instance, remote servers can deliver applications over a wide area network and they can be accessed through a web browser or any other computing component. Software or components shown in <FIG> as well as the corresponding data, can be stored on servers at a remote location. The computing resources in a remote server environment can be consolidated at a remote data center location or they can be dispersed. Remote server infrastructures can deliver services through shared data centers, even though the servers appear as a single point of access for the user. Thus, the components and functions described herein can be provided from a remote server at a remote location using a remote server architecture. Alternatively, the components and functions can be provided from a conventional server, or they can be installed on client devices directly, or in other ways.

In the example shown in <FIG>, some items are similar to those shown in <FIG> and they are similarly numbered. <FIG> specifically shows that data stores <NUM>, <NUM>, tutorial playing service <NUM>, other systems <NUM>, and other parts of the harvester <NUM> and/or mobile device <NUM> shown in <FIG> can be located at a remote server location <NUM>. Therefore, harvester <NUM> and mobile device <NUM> can access those systems through remote server location <NUM>.

<FIG> also depicts another example of a remote server architecture. <FIG> shows that it is also contemplated that some elements of <FIG> are disposed at remote server location <NUM> while others are not. By way of example, data stores <NUM>, <NUM> or portions of them, or other systems <NUM> can be disposed at a location separate from location <NUM>, and accessed through the remote server at location <NUM>. Regardless of where the items are located, the items can be accessed directly by harvester <NUM> and/or mobile device <NUM> through a network (such as a wide area network or a local area network), the items can be hosted at a remote site by a service, or they can be provided as a service, or accessed by a connection service that resides in a remote location. Also, data stored by harvester <NUM> and/or mobile device <NUM> can be stored in substantially any location and intermittently accessed by, or forwarded to, interested parties. For instance, physical carriers can be used instead of, or in addition to, electromagnetic wave carriers. In such an example, where cell coverage is poor or nonexistent, another mobile machine (such as a fuel truck) can have an automated information collection system. As the harvester <NUM> and/or mobile device <NUM> comes close to the fuel truck for fueling, the system automatically collects the information from the harvester <NUM> and/or mobile device <NUM> using any type of ad-hoc wireless connection. The collected information can then be forwarded to the main network as the fuel truck reaches a location where there is cellular coverage (or other wireless coverage). For instance, the fuel truck may enter a covered location when traveling to fuel other machines or when at a main fuel storage location. All of these architectures are contemplated herein. Further, the information can be stored on the harvester <NUM> and/or mobile device <NUM> until the harvester <NUM> and/or mobile device <NUM> enters a covered location. The harvester <NUM> and/or mobile device <NUM>, itself, can then send the information to the main network.

Claim 1:
A material loading vehicle (<NUM>), comprising:
a material conveyance subsystem configured to convey material (<NUM>) from the material loading vehicle (<NUM>) to a receiving vehicle (<NUM>) through a spout (<NUM>),
an automatic fill control system (<NUM>) configured to automatically position the spout (<NUM>) or to change the relative position of the receiving vehicle (<NUM>) and the material loading vehicle (<NUM>);
a tutorial condition detection system (<NUM>) configured to detect, during operation of the material loading vehicle, tutorial-related criteria and to identify tutorial content based on the tutorial-related criteria; the tutorial condition detection system comprising:
a detector (<NUM>) configured to detect the tutorial-related criteria and to generate a detector signal indicative of the detected tutorial-related criteria, wherein the detector (<NUM>) comprises a fill settings detector (<NUM>) configured to detect a setting of the automatic fill control system (<NUM>) and generate, as the detector signal, a setting signal indicative of the detected setting and/or wherein the material loading vehicle (<NUM>) comprises a camera (<NUM>) that has a field of view and that is configured to capture an image of the receiving vehicle (<NUM>) used by the automatic fill control system (<NUM>) and wherein the detector (<NUM>) comprises a camera view detector (<NUM>) configured to detect an orientation of the field of view of the camera (<NUM>) and generate, as the detector signal, a camera view output signal indicative of the detected orientation of the field of view of the camera (<NUM>);
an evaluation system (<NUM>) configured to evaluate the detector signal to determine whether tutorial content is to be recommended based on the detector signal;
a tutorial identification system (<NUM>) configured to, if the evaluation system (<NUM>) determines that tutorial content is to be recommended, identify the tutorial content to be recommended based on the evaluation of the detector signal; and
an output generator (<NUM>) configured to generate an output to recommend the identified tutorial content to an operator through an operator interface mechanism;
characterized in that the tutorial condition detection system (<NUM>) comprises a recency processing system (<NUM>) configured to determine whether the identified tutorial content has been recommended within a recency time threshold and, if so, omit recommending the tutorial content.