System and method providing visual aids for workpiece manipulator positioning and movement preview path

A virtual guidance apparatus and method assist an operator to maneuver a loader of a work vehicle for material handling of a workpiece. The virtual guidance apparatus includes a guidance control unit, one or more camera devices mounted on the associated work vehicle and operable to obtain an image of a loader of the associated work vehicle, and a display unit operatively coupled with the guidance control unit and displaying on a screen of the display unit that is viewable from an operator's seat of the associated work vehicle visual aids for guiding the operator in operating the loader to position a boom relative to level ground, to position a tool carrier on an end of the boom, to show a preview of a path of the tool carrier and of tools attached with the tool carrier, and to assist in positioning the workpiece load to be manipulated.

FIELD OF THE DISCLOSURE

The embodiments herein are directed to methods and systems assisting an operator to visualize and align workpiece manipulator systems operatively coupled with a work vehicle for efficient and safe vehicle and manipulator operation, material handling, and load manipulation. Although the example embodiments will be described as methods and systems for use in providing one or more visualization and alignment previews and one or more preferred orientations of a boom mechanism of a front end loader on a tractor, it is to be appreciated that the embodiments are not limited to those particular uses or applications, and instead may be used anywhere there is a need to provide an operator with a visualization of equipment positioning for helping the operator to view current positions of workpiece manipulator systems and of portions of a vehicle configured to couple with the workpiece manipulator systems as well as to visualize preferred or recommended preview paths for safe and efficient movement of the workpiece manipulator systems.

BACKGROUND

Many machines offer load manipulation that is outside of the strength and reach capabilities of the human body due for example to load size and the natural capacities or incapacities of human workers. Many machines use hydraulic boom mechanisms for example to lift or otherwise manipulate these heavy or large loads. In addition, some machines use such boom mechanisms with a tool carrier coupled to an attachment for handling various materials. However, these boom mechanisms can sometimes block the load from the view of the operator and may also adversely affect the operator's spatial perception of the tool carriers, attachments, and/or material being handled.

Therefore, when spatial perception of the tool carriers, attachments and/or material being manipulated is not easily attainable, many operations are performed by trial and error. This adds to the stress level of the operator, adversely impacts operation time wherein the time to successfully perform the operation may be more than doubled, and can create safety concerns related to unstable material loads and attachments that are not fully coupled to the tool carrier.

Many machines of the type described above include tool carriers supported by multiple parallel linkages of a boom mechanism, wherein the motion of the tool carrier follows a circular path relative to the work vehicle and is quasi-rectilinear as the work vehicle moves relative to the ground. Essentially, points of the tool carrier follow parallel circular lines during movement of the boom mechanism carrying the tool carrier relative to the work vehicle. After the tool carrier is disposed into a desired orientation relative to the work vehicle and absent steering changes in the work vehicle, all points of both the tool carrier as well as the work vehicle follow a quasi-rectilinear path relative to the ground wherein ground unevenness may contribute to imperfect rectilinear motion. Although an operator may rely on the quasi-rectilinear path of the tool carrier carrying a load relative to a target site, the enhanced reach capabilities of these systems oftentimes undermines visibility as spatial perception is difficult for an operator to realize from a single point of view, usually positioned on a seat of the work vehicle disposed on a side of the vehicle opposite from the loader mechanism. The lack of visual feedback can be unnerving at times for the operator adding to stress and also adversely impacting the operation time and safety.

It is therefore desirable to provide systems and methods for assisting an operator to maneuver the work vehicle and the boom mechanism with a tool carrier or an implement or attachment receiver integrated into the boom mechanism (collectively referred to herein as a “loader”) relative to the ground under the work vehicle, to one or more attachments and to material to be loaded or manipulated such as to approach, pick up, and deposit the material at locations such as a storage rack, platform or the like.

It is therefore also desirable to provide systems and methods for assisting an operator to maneuver a loader of a work vehicle for material handling by providing visual aids on a screen of a display unit such as for example an overhead display that is viewable from an operator's seat of the work vehicle displaying all or portions of the boom mechanism, tool carrier, one or more attachments and material to be loaded or manipulated to assist in positioning of the work vehicle, the boom mechanism, the tool carrier and/or the one or more attachments relative to the material load. Further, this overhead display may show a preview of a path of the tool carrier relative to the material to be loaded or manipulated.

SUMMARY

The embodiments herein are directed to methods and systems providing alignment aids for assisting an operator to visualize the position of portions of a loader of a work vehicle and attachments detachably coupled to the loader for efficient and safe vehicle operation, material handling, and load manipulation. It is to be understood that material handling and manipulation includes changing material position such as picking up or laying down loads such as bales, pallets or the like, and boom attachment coupling such as when attaching an implement such as bucket or pallet fork to a carrier or receiver on the end of the boom of a loader. “Material handling” as used herein includes and is not limited to any and all operations were material or loads of any type are manipulated, moved, transported, and the like. Although the example embodiments will be described as methods and systems for providing visualization of preferred orientations of a boom mechanism attached to a work vehicle, and providing previews of one or more paths for movement of the work vehicle approaching a load or tool or moving carrying a load, it is to be appreciated that the embodiments are not limited to those particular uses or applications, and instead may be used anywhere there is a need to provide an operator with a visualization of material handling attachments for helping the operator to view current positions of the loader and of the attachments coupled with a carrier of the loader as well as to visualize preferred or recommended paths for coupling these attachments and carrying a load safely and efficiently.

The embodiments herein are directed to methods and systems assisting an operator to visualize and align portions of a loader of a work vehicle and of material handling attachments detachably coupled to the loader relative to the work vehicle itself, relative to the ground under the work vehicle, and to help visualize the position of one or more attachments on the loader and of material to be loaded or manipulated such as to approach a load and/or tool, pick up the tool and/or the load using the tool, and deposit or otherwise place the load or any other material at a desired location such as at or on the ground at a selected location, at or on a storage rack, platform or the like.

The embodiments herein are further directed to methods and systems assisting an operator to visualize and align material handling attachments detachably coupled to a work vehicle for manipulating material loads relative to a storage rack, platform, deposit place or the like, for efficient and safe handling. Although the example embodiments will be described as methods and systems for use in providing a visualization of one or more visual aids such as preview paths and preferred orientations of a boom mechanism attached to a work vehicle and of attachments coupled with the carrier on a boom mechanism of the vehicle, it is to be appreciated that the embodiments are not limited to those particular uses or applications, and instead may be used anywhere there is a need to provide an operator with a visualization of the current position of a loader, and preferred or recommended visual aids such as preview paths for safely and efficiently loading, unloading, or otherwise handling the material load.

The embodiments herein provide for new and improved virtual guidance systems and methods assisting an operator manipulate a loader of a work vehicle to a desired position A virtual guidance system includes a guidance control unit, a camera device operatively coupled with the guidance control unit, and a display unit operatively coupled with the guidance control unit. The guidance control unit includes a processor and a memory device configured to store visual aid data and logic executable by the processor. The camera device is mounted relative to the work vehicle and is operable to obtain an image of the loader of the work vehicle. The display unit is operable to display a viewable image of the loader based on the image obtained by the camera device and to also display a first virtual alignment target image based on the visual aid data. In accordance with an example embodiment, alignment of a selected portion of the viewable image of the loader with the first virtual alignment target image by the operator moving the loader relative to the associated work vehicle results in the loader being manipulated to the desired position.

The embodiments herein provide for new and improved systems and methods for assisting an operator of a work vehicle to maneuver a loader of a work vehicle, wherein the loader includes a boom having one or more articulations and a tool carrier selectively attachable with a working tool, implement or the like having one or more degrees of freedom of movement relative to the boom, for material handling of a workpiece by providing visual aids on a screen of a display unit such as for example an overhead display that is viewable from an operator's seat of the work vehicle for guiding the operation in operating the loader. The visual aids may be augmented in embodiments herein to provide one or more audible aids emitted from a sound generating device near to the operator for assisting by providing audible confirmation to the operator of the work vehicle that the loader handling the material has been maneuvered into one or more orientations and/or positions relative to the workpiece. The visual aids may also or alternatively be augmented in embodiments herein to provide one or more visual aids using color changes provided on the screen of the display unit that is viewable from the operator's seat of the work vehicle for assisting by providing a colorized confirmation to the operator of the work vehicle that the loader handling the material has been maneuvered into one or more orientations and/or positions relative to the workpiece.

The embodiments herein further provide for new and improved systems and methods for assisting an operator of a work vehicle to maneuver a loader of a work vehicle for material handling of a workpiece by providing visual aids on a screen of a display unit such as for example an overhead display that is viewable from an operator's seat of the work vehicle for guiding the operation in operating the loader to position a boom of the loader of the work vehicle relative to the body of the work vehicle such as at a preferred retracted position or the like. Visual aids are provided to the operator to assist in positioning a boom and/or a tool carrier on a free end of the boom with respect to the work vehicle, wherein in an example embodiment, one or more physical markers such as for example crosshair markers, manufacturer's logos, or the like are provided on or at the boom and/or on or at the tool carrier on the end of the boom. Helping the operator to position a boom of the loader of the work vehicle relative to the vehicle body or the like at a desired inclination or raised position off the ground is useful for example when traveling with the vehicle with or without a load such as when travelling to and from a worksite, on a roadway or path, or into or out from a storage facility such as a barn.

Visual aids are also provided to the operator to assist in positioning the boom including the boom and tool carrier on the boom with respect to the ground, wherein in an example embodiment, one or more physical markers such as for example crosshair markers, manufacturer's logos, or the like are provided on or at the boom and/or on or at the tool carrier on the end of the boom. These visual aids are provided to the operator to assist in positioning the boom and/or tool carrier or both with respect to the ground and also in positioning or otherwise aligning the work vehicle with respect to a preferred path for use in approaching a load, wherein in the example embodiment, one or more preview paths are presented having one or more markers displayed over the one or more preview paths and/or overlying the one or more preview paths. This may be useful when maneuvering the vehicle prior to picking up a load.

Visual aids are also provided to the operator to assist in positioning a loader including a boom and a tool carrier on the end of the boom for efficient and safe material handling and load manipulation when picking up a load.

Visual aids are also provided to the operator to assist in positioning a loader including a boom and a tool carrier on the end of the boom for efficient and safe material handling and load manipulation when placing the load onto an associated storage rack, platform or the like.

The visual aids may be augmented in embodiments herein to provide one or more colorized visual aids on a screen of a display unit near to the operator and/or audible aids emitted from a sound generating device near to the operator for assisting the operator of the work vehicle to maneuver the loader into desired positions for material handling of the workpiece such as a load, an attachment, tool or implement to be coupled with a carrier at an end of a boom of the loader or the like.

The embodiments herein still further provide for new and improved systems and methods for assisting an operator of a work vehicle to maneuver a loader of a work vehicle for material handling of a workpiece by providing visual aids on a screen of a display unit such as for example an overhead display that is viewable from an operator's seat of the work vehicle for guiding the operation in operating the loader to position a tool carrier on an end of a boom of the loader relative to the boom.

The embodiments herein yet still further provide for new and improved systems and methods for assisting an operator of a work vehicle to maneuver a loader of a work vehicle for material handling of a workpiece by providing visual aids on a screen of a display unit such as for example an overhead display that is viewable from an operator's seat of the work vehicle for guiding the operation in operating the loader to show a preview of a path of the tool carrier on the boom. In an embodiment the visual aids include one or more preview paths representative of an actual path of one or more tools or implements attached with the tool carrier on the boom. In an embodiment the preview path is provided in the form of guidelines extending between a physical feature of the implement or tool such as a physical marker on the implement or tool and a physical feature of the target or load such as a physical marker on the target or load. In a further embodiment, the preview of the path is representative of a direct virtual path of the tool carrier or of an implement attached with the tool carrier. In an embodiment the preview path is provided in the form of guidelines extending between a coupling feature of the implement or tool and a coupling feature of the target or load. In a still further embodiment, the preview of the path is representative of an indirect virtual path of an implement attached with the tool carrier that is not directly physically visible to the operator, wherein the operator may operate the work vehicle based on the displayed virtual path preview for correctly moving the work vehicle relative to the load. In an embodiment the preview path is provided in the form of guidelines extending between a geometrical feature of the implement or tool and a physical feature of the target or load such as a physical marker on the target or load. In a further embodiment the preview path is provided in the form of guidelines extending between a virtual feature of the implement or tool and a geometrical feature of the target or load such as a physical feature of the target or load offset from a physical coupling feature of the target or load.

In accordance with an aspect of the embodiments, a virtual guidance system includes a guidance control unit, a camera device operatively coupled with the guidance control unit, and a display unit operatively coupled with the guidance control unit. The guidance control unit includes a processor and a memory device configured to store visual aid data and logic executable by the processor. The camera device is mounted relative to the work vehicle and is operable to obtain an image of the loader of the work vehicle1. The display unit is operable to display a viewable image of the loader based on the image obtained by the camera device and to also display a first virtual alignment target image based on the visual aid data. In accordance with an example embodiment, alignment of a selected portion of the viewable image of the loader with the first virtual alignment target image by the operator moving the loader relative to the associated work vehicle results in the loader being manipulated to the desired position.

In accordance with a further aspect of the embodiments, the camera device of the virtual guidance apparatus is operable to generate loader image data representative of an obtained image of the loader of the work vehicle, and the display unit is operative to receive the loader image data and the visual aid data and to display on a screen of the display unit that is viewable by the operator of the associated work vehicle the virtual alignment target image superimposed on the viewable image of the loader displayed on the screen. In the example embodiment, the virtual alignment target image is representative of a selectable target location on the screen for alignment of a visual marker portion of the viewable image of the loader. In this way, movement of the loader relative to the associated work vehicle to establish an alignment of the visual marker portion of the viewable image of the loader displayed on the screen with the virtual alignment target image displayed at the selectable target location on the screen corresponds to a desired relative physical position between one or more of the loader and the associated work vehicle and/or the loader and ground supporting the associated work vehicle.

In accordance with a further aspect of the embodiments, the virtual alignment target image displayed by the display unit of the virtual guidance apparatus comprises a virtual carrier alignment target image representative of a selectable carrier target location on the screen for alignment of a visual carrier marker portion of the viewable image. In this way, movement of the loader relative to the associated work vehicle to establish an alignment of the visual carrier marker portion of the viewable image of the loader displayed on the screen with the virtual carrier alignment target image displayed at the carrier target location on the screen corresponds to a desired relative physical position between one or more of a tool carrier on an end of the boom of the loader and the associated work vehicle and/or the tool carrier of the loader and ground supporting the associated work vehicle.

In accordance with a still further aspect of the embodiments, the display unit of the virtual guidance apparatus is operable to display on the screen a virtual directional heading image superimposed on the viewable image of the loader based on the image obtained by the camera device, the virtual directional heading image being representative of a preview path on the screen of a movement path to be followed by a tool carrier of the loader for forward movement of the associated work vehicle.

In accordance with yet a further aspect of the embodiments, a virtual guidance apparatus assists an operator manipulate a loader of an associated work vehicle to a desired position. A guidance control unit of the apparatus includes a processor and a memory device operatively coupled with the processor and configured to store visual aid data, and logic executable by the processor. A camera device of the apparatus is operatively coupled with the guidance control unit and is mounted relative to the associated work vehicle and further is operable to obtain an image of the loader of the associated work vehicle. A display unit of the apparatus is operatively coupled with the guidance control unit and is operable to display a virtual directional heading image superimposed on the viewable image of the loader based on the image obtained by the camera device. In the example embodiment, the virtual directional heading image is representative of a preview path on the screen of a movement path to be followed by a tool carrier of the loader for forward movement of the associated work vehicle.

In accordance with yet a still further aspect of the embodiments, a virtual guidance apparatus assists an operator to automatically manipulate a loader of an associated work vehicle to a desired position. A guidance control unit of the apparatus includes a processor and a memory device operatively coupled with the processor and configured to store visual aid data, and logic executable by the processor. A camera device of the apparatus is operatively coupled with the guidance control unit and is mounted relative to the associated work vehicle and further is operable to obtain an image of the loader of the associated work vehicle. The camera is leveraged to work together with the logic executed by the processor to function as an implement position sensor by providing the images captured by the camera to the guidance control unit of the apparatus for image processing to determine the current location of the bucket or other implement relative to the body of the tractor and/or relative to the ground under the tractor. In that way, the camera together with the logic executed by the processor of the guidance control unit of the apparatus may operate as a return to position (RTP) system utilized to automatically return the loader, boom arm, and implements such as a bucket to a pre-stored position in response to operator input. The camera together with the logic executed by the processor effectively provides for directly or indirectly monitoring the position of the bucket relative to the body of the tractor in lieu of the need for physical implement position sensors such as linear transducers or the like to monitor the stroke position of cylinders. The camera provides the boom stroke position and other data related to the inclination of the implement on the boom in an essentially continual manner.

In accordance with yet a still further aspect of the embodiments, a virtual guidance method is provided for assisting an operator of an associated work vehicle to maneuver portions of the associated work vehicle for material handling of an associated workpiece. The virtual guidance method includes obtaining by a camera device mounted relative to the associated work vehicle an image of a loader of the associated work vehicle. The virtual guidance method further includes displaying on a screen of a display unit that is viewable from an operator's seat of the associated work vehicle and that is operatively coupled with a guidance control unit comprising a processor and a memory operatively coupled with the processor the image of the loader of the associated work vehicle obtained by the camera device, and a first virtual alignment target superimposed on the image of the loader obtained by the camera. The first virtual alignment target is representative of a first predetermined target location on the screen for alignment of a first visual marker portion of the image of the loader. In this way, movement of the loader relative to the associated work vehicle to establish a coincidence between the first visual marker portion of the image of the loader displayed on the screen and the first predetermined target location on the screen corresponds to a first predetermined relative physical position between the loader and the associated work vehicle and/or the loader and ground supporting the associated work vehicle.

In an embodiment, the virtual guidance apparatus includes a plurality of cameras, each being provided on a segment of an articulated boom. In particular and in accordance with an example embodiment, a camera is provided for each pivot joint and/or each degree of freedom of movement of an articulated boom. The example embodiments described herein include for ease of description and understanding only a single camera for a loader having a single pivot or degree of freedom of movement.

In some embodiments the input comprises a touchscreen portion of the display unit or a pointer device operatively coupled with the guidance control unit, and the memory of the guidance control unit stores data corresponding to the boom position training signal received by the input during the training.

Other embodiments, features and advantages of the example embodiments for assisting an operator of a work vehicle to maneuver a loader of the work vehicle for material handling of a workpiece by providing visual aids for guiding the operator in operating the loader to position a boom relative to level ground, to position a tool carrier relative to the boom, to show a preview of a path of the tool carrier, and to assist in positioning the workpiece load to be manipulated will become apparent from the following description of the embodiments, taken together with the accompanying drawings, which illustrate, by way of example, the principles of the example embodiments.

DETAILED DESCRIPTION

In the following description reference is made to the accompanying figures which form a part thereof, and in which is shown, by way of illustration, one or more example embodiments of the disclosed systems and methods providing visual aids for loader positioning and preview path for material handling. Various modifications of the example embodiments may be contemplated by on of skill in the art.

Furthermore, in detailing the disclosure, terms of direction, such as “forward,” “rear,” “front,” “back,” “lateral,” “horizontal,” and “vertical” may be used. Such terms are defined, at least in part, with respect to the direction in which the work vehicle or implement travels during use. The term “forward” and the abbreviated term “fore” (and any derivatives and variations) refer to a direction corresponding to the direction of travel of the work vehicle, while the term “aft” (and derivatives and variations) refer to an opposing direction. The term “fore-aft axis” may also reference an axis extending in fore and aft directions. By comparison, the term “lateral axis” may refer to an axis that is perpendicular to the fore-aft axis and extends in a horizontal plane; that is, a plane containing both the fore-aft and lateral axes. The term “vertical,” as appearing herein, refers to an axis or a direction orthogonal to the horizontal plane containing the fore-aft and lateral axes.

Referring now to the drawings, wherein the showings are only for the purpose of illustrating the example embodiments only and not for purposes of limiting the same,FIG.1illustrates a work vehicle1that may be used with the virtual guidance apparatus100according to example embodiments of the present disclosure. In the embodiment illustrated, the work vehicle1is a tractor2but it is to be appreciated that the embodiments herein are not limited to use with tractors or use with any particular mobile material handling equipment. The virtual guidance apparatus100of the example embodiments herein may execute or otherwise perform a virtual guidance method providing visual aids for loader positioning and preview path for material handling according to examples of the present disclosure. To provide a useful, albeit non-limiting example, the virtual guidance apparatus100of the example embodiments is described below in conjunction with a particular type of work vehicle1, shown in the form of a tractor2, equipped with particular types of work implements such as a pallet fork implement27, a bucket implement25, a grapple implement26, etc. included in an front end loader (FEL) attachment. The following notwithstanding, it is emphasized that embodiments of the virtual guidance apparatus100may be deployed onboard various other types of work vehicles having one or more movable implements attached thereto. The virtual guidance apparatus100of the example embodiment shown includes an optical system120including for example a camera121. The virtual guidance apparatus100may adjust physically and/or electronically one or more operational characteristics of the one or more imaging devices such as the optical system120including for example the camera121, or other optical device(s) in response to movement of various different types of work implements without limitation. The virtual guidance apparatus100may be distributed as an integral part of a work vehicle, as an integral part of a loader10attachment or other module that may be removably attached to and detached from a work vehicle on an as-needed basis, or as a discrete assembly or multi-component kit that may be installed on an existing work vehicle via retrofit attachment.

In some examples, the camera121can be mounted on the cab facing forwards toward the bucket, attachment, or other work tool. For example, the camera can be mounted on the front surface of the cab5or the top surface (e.g., roof) of the cab5. The camera121can transmit images (or video signals) wirelessly to a display or though wiring that extends along an arm, boom, or other frame structure of the work vehicle to the display. In other examples, the camera can be mounted on an arm of the work vehicle, including the lower surface, upper surface, or sides of the arm structure used to support the bucket, attachment, or other work tool or device used during operation of the work vehicle. Alternatively, the camera can be mounted on a boom of the work vehicle, including the lower surface, upper surface, or sides of the boom that is commonly used to support the arm and is attached to the main body of the work vehicle.

FIG.1is a side perspective view of a tractor2including a front end loader10attachment and a virtual guidance apparatus100, as illustrated in accordance with an example embodiment of the present disclosure. The virtual guidance apparatus100is only partially shown inFIG.1and an example embodiment of which will be described more fully below in conjunction withFIG.2. First, however, a general description of the tractor2is provided to establish an example context in which the virtual guidance apparatus100may be better understood. In addition to the loader10attachment and the optical system120, the tractor2includes a vehicle body3having a chassis4, and a cab5. The tractor2is generally bilaterally symmetrical about its longitudinal axis, which is parallel to the X-axis identified inFIG.1by coordinate legend6.

In the illustrated embodiment, the work vehicle1includes a loader10provided for material handling of an associated load. The loader10of the work vehicle1illustrated includes an articulated boom20having a tool carrier22on a free end23of the boom20. The tool carrier22portion of the loader10may be for example a receiver mechanism or other similar device or apparatus provided on the free end23of the boom20for enabling coupling of an attachment such as an implement or a tool for example with the free end23of the boom20. In this regard, the tool carrier22is selectively mutually connectable with one or more various attachments, tools or implements24such as for example a bucket implement25, a grapple implement26, a pallet fork implement27having a set of pallet tines28, a gripper (not shown), a bail hugger (not shown), a bale spear (not shown), and/or various working devices as may be available and/or desired.

The articulated boom20of the loader10movably mounts the tool carrier22carrying a selected one of the one or more various attachments, tools or implements24to a forward portion of the vehicle body3and, more specifically, mounts the tool carrier22to chassis4. In the illustrated example, the pallet fork implement27is shown attached with the tool carrier22but it is to be appreciated that the pallet fork implement27may be replaced by any different type of work implement, such as any of the bucket implement25, the grapple implement26, the gripper, the bail hugger, the bale spear, all as mentioned above, or any other work implement as necessary and/or desired, in alternative embodiments of the tractor2. The articulated boom20assembly may assume any form capable of moving the work implement24relative the vehicle body4in response to operator commands. In the example embodiment shown inFIG.1, the boom assembly20includes a system of linkages, hydraulic cylinders, plumbing lines (not shown), and other components suitable for this purpose. More specifically, the boom assembly20includes an aft bracket set30affixed to the vehicle body4, a forward bracket set32to which the tool carrier22carrying the pallet fork implement27is pivotally attached, and an intermediate or mid bracket set34between the aft and forward bracket sets30,32. Twin lift arms36,38pivotally attach the aft bracket set30with the mid bracket set34, which is, in turn, attached with the forward bracket set32by twin implement arms46,48. Lift cylinders37,39(only one of which can be seen) are further coupled between the aft bracket set30and the mid bracket set34, while bucket cylinders47,49are coupled between the mid bracket set34and the forward bracket set33.

In the example embodiment the camera121is mounted on the cab facing forwards toward the bucket, attachment, or other work tool so that all of the lift arms36,38, the implement arms46,48, the aft, mid, and forward bracket sets30,34,32, and any implements attached with the boom are within the field of view of the imaging system120. Additionally, embodiments the virtual guidance apparatus100may adjust one or more operational characteristics of one or more imaging devices such as an optical system120including for example a camera121, or other optical device(s) in response to movement of various different types of work implements without limitation for this purpose.

The loader10attachment further includes other features, such as hydraulic lines and control valves, which are not shown inFIG.1for simplicity. When the loader10is mounted to the vehicle body4, the hydraulic lines of the loader10are fluidly connected to a pressurized hydraulic fluid supply on the tractor2in a manner permitting an operator seated within the cab5to control the cylinders37,39,47, and49. Beginning from the ground position shown inFIGS.6A,6B,7, and8, an operator may command the boom assembly20of the loader10to lift the pallet fork implement27carried on the tool carrier22as shown by controlling the lift cylinders37,39to extend. As the lift cylinders37,39retract, the boom20comprising the pallet fork implement27carried on the tool carrier22is lowered from the full height position shown inFIG.3to the lowered position shown inFIGS.4,5A,5B,6A, and6B. In the process of moving from the full height position to the lowered position, the boom20travels through the intermediate or mast level position shown inFIG.1. Similarly, as the bucket cylinders47,49extend in response to operator commands, the boom assembly20tilts the pallet fork implement27carried on the tool carrier22from the upright position shown inFIGS.4,5A, and5Bthrough an intermediate position to the forward-facing lift position shown inFIGS.6A,6B, and7. Conversely, the operator may control the boom assembly20to stroke cylinders37,38,47,48in a manner opposite to that just described to return the pallet fork implement27from the grounded, forward-facing lift position shown inFIGS.6A,6B, and7to the raised full height position shown inFIG.3.

It is to be appreciated that the motion of the cylinders37,39,47, and49may be controlled directly by operator manipulation of various levers, pedals and/or other human interface device(s) operatively coupled with hydraulic control components of the system, and further that the motion of the loader10may be indirectly controlled by the operator rendering motion commands to via the virtual guidance apparatus100which in turn may operate the various hydraulic control components of the system for moving the cylinders37,39,47, and49thereby effecting the desired movement of the loader10. It is still further to be appreciated that the motion of the cylinders37,39,47, and49may be controlled by the virtual guidance apparatus100automatically and/or semi-automatically in response to generalized operator commands such as for example return to position (RTP) commands wherein for example the virtual guidance apparatus100may function in an RTP mode to automatically return the boom20overall, portions of the boom20such as for example the lift arms36,38and/or the implement arms46,48, the various attachments, tools or implements24carried on the tool carrier22, to one or more pre-stored position(s) in response to operator input.

In the example embodiment shown in the Figures, the implements24are moved by the boom20along a vertical plane containing or parallel to the longitudinal axis of the body4of the tractor2(corresponding to an X-Y plane in coordinate legend6). The optical system120thus need only have a field of view (vertical in the example embodiment) rotatable about a single rotational axis to remain trained on the implements24throughout the range of motion of the loader10relative to the body4of the tractor2. For the camera121shown this rotational axis is parallel to the lateral axis of the tractor2; a “lateral axis” of the tractor2(or other work vehicle) defined as an axis extending within a horizontal plane (an X-Z plane in coordinate legend6) and perpendicular to the longitudinal axis of the tractor2. Stated differently, a lateral axis of tractor2is parallel to the Z-axis in coordinate legend6, and camera121may be selectively rotatable in a further example embodiment about a rotational axis likewise parallel to the Z-axis, although in the example embodiment described herein the camera121remains fixed relative to the work vehicle1and in particular relative to the cab5of the tractor2and, as such remains fixed relative the a rotational axis likewise parallel to the Z-axis. In other embodiments wherein the virtual guidance apparatus100is deployed onboard a work vehicle including a backhoe or other implement that is rotatable about a horizontal axis (corresponding to the Y-axis in coordinate legend6), the camera121or other optical device or optical devices included within the virtual guidance apparatus100may be selectively rotatable about multiple axes including an axis parallel to the horizontal axis.

The camera121or other optical device or devices included with the virtual guidance apparatus100may also be imparted with additional degrees of freedom in further embodiments. For example, the camera121may be moved to translate along any axis in three dimensional space, as appropriate to better visually capture the pallet fork implement27or other implement24throughout its full range of motion relative to the body4of the tractor2. For example, and as may be appreciated, the camera121may be mounted on a telescopic post122that extends vertically upward when the pallet fork implement27or other implement24is moved into the full height position to provide a better vantage point for observation of markers on the pallet fork implement27or other implement24as will be described below to minimize the degree to which the boom assembly20visually obstructs the camera's view of the pallet fork implement27or other implement24. In addition, the virtual guidance apparatus100may electronically adjust one or more operational characteristics of the one or more imaging devices such as the optical system120including for example the camera121, or other optical device(s) in response to movement of various different types of work implements without limitation to minimize the degree to which the boom assembly20visually obstructs the camera's view of the pallet fork implement27or other implement24.

The articulated boom20may be maneuvered such as by actuating various hydraulic cylinders37,39,47,49to extend or retract and to lower or raise an articulated boom20relative to the work vehicle so that the tool carrier22may be positioned for coupling the boom20with the one or more various implements24. The tool carrier22disposed on the end of the boom20collectively forms a loader10in accordance with the example embodiments. In an example embodiment herein, the one or more various implements24may be initially considered as a workpiece prior to being coupled with the tool carrier22of the loader10wherein an operator may desire for the one or more various implements24and the tool carrier22to be mutually latched joined or otherwise mechanically coupled prior to performing a material handling task or the like. As an example, an operator may wish to couple the pallet fork implement27with the tool carrier22prior to proceeding to approach, lift, and place a palletized load at a desired location. The visual aids described herein are developed by the system and provided to the operator of the associated work vehicle1for assisting in properly locating the tool carrier22on the free end23of the boom20at a desired height relative to the ground, and also for providing further visual aids for assisting in properly tilting the tool carrier22to a desired orientation or inclination for enabling coupling of the carrier22with the selected attachment24, and further for positioning the work vehicle1relative to the selected implement24for moving the vehicle1supporting the tool carrier22into a coupling relationship with the selected implement24for mutually connecting the tool carrier22with the selected implement24.

In addition, the visual aids described herein are developed by the system and provided to the operator of the associated work vehicle1for properly locating the tool carrier22on the free end23of the boom20at a desired height relative to the ground for training the system for RTP operation, and also for providing further visual aids for assisting in properly tilting the tool carrier22to a desired orientation or inclination for enabling coupling of the carrier22with the selected attachment24for training the system for the RTP operation, and further for positioning the work vehicle1relative to the selected implement24for moving the vehicle1supporting the tool carrier22into a coupling relationship with the selected implement24for mutually connecting the tool carrier22with the selected implement24for training the system for the RTP operation.

After the tool carrier22disposed on the end of the boom20is moved to the desired position and coupled with the one or more various implements24, further visual aids described herein are developed by the system and provided to the operator of the associated work vehicle1for assisting in properly positioning the loader10with the implement24coupled therewith, for engaging with an associated load or other workpiece (not shown) such as for example to pick up the load using the bucket implement25, the grapple implement26, the pallet fork implement27or the like. In an embodiment a preview path is provided in the form of guidelines extending between a physical feature of the implement or tool such as a physical marker on the implement or tool and a physical feature of the target or load such as a physical marker on the target or load. In a further embodiment, the preview of the path is representative of a direct virtual path of the tool carrier or of an implement attached with the tool carrier. In an embodiment the preview path is provided in the form of guidelines extending between a coupling feature of the implement or tool and a coupling feature of the target or load. In a still further embodiment, the preview of the path is representative of an indirect virtual path of an implement attached with the tool carrier that is not directly physically visible to the operator, wherein the operator may operate the work vehicle based on the displayed virtual path preview for correctly moving the work vehicle relative to the load. In an embodiment the preview path is provided in the form of guidelines extending between a geometrical feature of the implement or tool and a physical feature of the target or load such as a physical marker on the target or load. In a further embodiment the preview path is provided in the form of guidelines extending between a virtual feature of the implement or tool and a geometrical feature of the target or load such as a physical feature of the target or load offset from a physical coupling feature of the target or load.

After the load is engaged and raised using the loader10with the implement24coupled therewith, further visual aids described herein are developed by the system and provided to the operator of the associated work vehicle1for assisting in properly placing the load at a desired location such as on associated storage rack, platform or the like. In an embodiment a preview path is provided in the form of guidelines extending between a physical feature of the load as a physical marker on the load and a physical feature of the target location such as a physical marker on the target location. In a further embodiment, the preview of the path is representative of a direct virtual path of the load attached carried by the tool. In an embodiment the preview path is provided in the form of guidelines extending between a coupling feature of the load and a coupling feature of the target location. In a still further embodiment, the preview of the path is representative of an indirect virtual path of the carried load that is not directly physically visible to the operator, wherein the operator may operate the work vehicle based on the displayed virtual path preview for correctly moving the work vehicle carrying the load relative to the desired unloading location. In an embodiment the preview path is provided in the form of guidelines extending between a geometrical feature of the load and a physical feature of the target location such as a physical marker on the target location. In a further embodiment the preview path is provided in the form of guidelines extending between a virtual feature of the load and a geometrical feature of the target location such as a physical feature of the target location offset from a physical coupling feature of the target location.

In accordance with an example embodiment, the virtual guidance apparatus100includes a guidance control unit110, an optical system120such as a camera device121mounted on the associated work vehicle1and operatively coupled with the guidance control unit110, and a display unit130operatively coupled with the guidance control unit110. As appearing herein, the term “camera device” refers to an optical device, a camera system, electrical light, or other device that generates and/or detects visible light or other electromagnetic radiation. The guidance control unit110includes a processor204(FIG.2) and a memory device210(FIG.2) operatively coupled with the processor in a manner to be described in greater detail below. The memory device210stores visual aid logic211(FIG.2) executable by the processor204to provide virtual guidance for assisting an operator of the work vehicle to maneuver the loader10to a desired position relative to the work vehicle and relative to the ground, and for providing virtual guidance assistance for material handling of a workpiece such as and including during approaching and attaching the carrier22with a selected implement24, and during picking up and dropping off and/or placement of a load in accordance with the example embodiments. The memory device210also stores visual aid data213(FIG.2) that may be generated with operator assistance or input and then used for generating one or more virtual alignment target images on a display screen based on the visual aid data.

The display unit130can be positioned in the cab5or elsewhere on the work vehicle. The display unit130can be positioned so as to be viewable by the operator while the operator manipulates the work vehicle's controls to perform a task. And the display unit130can be positioned so as not to hinder the opening or closing of any latches, hatches, windows, or doors in the cab. In some examples, the display can be mounted on the exterior of the cab5, such as to a rollover protection structure (ROPS), via a mounting device. This can enable the operator to view the display while operating the controls of the work vehicle. In other examples, the display unit130can be positioned in the cab5, for example, by using a vertical support pillar that forms the structure of the ROPS. The vertical support pillar can be located in front of the operator or behind the operator while the operator is seated in the cab5. Additionally or alternatively, the display unit130can be positioned in the cab using a cross member that spans across a vertical support pillar. The cross member can be positioned in front of, behind, or on the sides of the operator while the operator is seated in the cab5. The display unit130can additionally or alternatively be positioned on a supporting structure inside the cab5for the controls (e.g., pilot controls) used to operate the work vehicle1. The controls can control the rotation of the work vehicle1; the boom20, arm, or work tool24; or any combination of these. In some examples, the display unit130can be an existing monitor in the work vehicle that typically shows other information, such as operating parameters (e.g., engine speed, fuel level, and engine temperature) of the work vehicle. In such an example, the monitor may be modified to perform the functionality of the display unit130.

The camera device121of the virtual guidance apparatus100is operable to obtain an image of the loader10of the associated work vehicle1and to generate loader image data representative of the obtained image of the loader10. The display unit130is operatively coupled with the guidance control unit110, and receives the loader image data and the visual aid data213. The display unit130displays on a screen132of the display unit that is viewable by the user from a user station of the associated work vehicle1a viewable image of the loader10of the associated work vehicle based on the loader image data. The display unit130also displays on the screen132a first virtual alignment target image based on the visual aid data213. In the example embodiment, the first virtual alignment target image is a static display and is rendered on the screen132superimposed over the dynamic viewable image of the loader. The first virtual alignment target image is representative of a first selectable target location on the screen132for alignment of a first visual marker portion of the viewable image of the loader.

In accordance with an embodiment the first virtual alignment target image representative of the first selectable target location to be displayed on the screen is digitally compared by logic executed by the processor of the guidance control unit with the first visual marker portion of the viewable image of the loader in order to determine that the loader has reached a desired conformation relative to the tractor and/or relative to the ground during the automatic or semi-automatic RTP operational functionality off the subject system.

It is to be appreciated that although only a single camera device121is shown for ease of illustration mounted to the work vehicle1at the position shown, the camera device121shown is merely representative of a set of one or more cameras that may include several connected cameras that can be mounted on or near the vehicle or anywhere as may be necessary or desired such as for example on other areas of the work vehicle or on portions of the loader itself in order to obtain as many images of one or more targets on the boom and/or on the tool carrier as may be necessary and/or desired for providing images of a full relevant range of movement or motion of the loader.

In an embodiment, the virtual guidance apparatus may include a plurality of such camera devices121comprising the optical system120of the example embodiment, each being provided on and/or oriented towards a segment of the loader10in the form of an articulated boom20. In particular and in accordance with an example embodiment, a camera may be provided for each pivot joint and/or each degree of freedom of movement of the articulated boom20. The example embodiment described herein includes for ease of description and understanding only a single camera device121for a loader having a single pivot or degree of freedom of movement.

FIG.2is a block diagram that illustrates a representative virtual guidance apparatus or controller100according to the example embodiments. The controller is suitable for executing embodiments of one or more software systems or logic modules that perform the virtual guidance method for assisting an operator of an associated work vehicle to maneuver portions of the associated work vehicle for material handling of an associated workpiece according to the subject application such as may be useful when traveling with the vehicle without a load, when maneuvering the loader to couple an implement onto a tool carrier on the free end of a boom of the loader, when maneuvering the vehicle with the attached implement prior to picking up a load, for efficient and safe material handling and load manipulation when picking up a load, and for efficient and safe material handling and load manipulation when placing the load onto an associated storage rack, platform or the like.

The example system includes a bus202or other communication mechanism for communicating information, and a processor204coupled with the bus for processing information. The computer system includes a main memory, such as random access memory (RAM)206or other dynamic storage device for storing information and instructions to be executed by the processor204, and read only memory (ROM)208or other static storage device for storing static information and instructions for the processor204. A storage device210is also suitably provided for storing visual aid logic211comprising instructions for execution by the processor, and other information including for example data and instructions for execution by the processor for obtaining and displaying visual aids on a screen of the display unit that is viewable from an operator's seat of the associated work vehicle. The visual aids are displayed for helping to guide the user in operating the loader to position a boom relative to the work vehicle, to position the boom relative to level ground, to position the tool carrier22on the free end23thereof relative to the one or more various attachments or implements24to be coupled with the carrier22, to position the one or more various attachments or implements24relative to the boom and/or relative to the ground or to an associated load to be picked up, to show a preview of a path tool carrier22, to assist in positioning the workpiece load to be manipulated, and to drop off the load in or on an associated storage rack, platform or the like. The storage device210may further store visual aid data213used for generating in the example embodiment one or more static virtual alignment target images superimposed on a dynamic viewable image of the loader displayed on the screen132, wherein the one or more virtual alignment target images displayed on the screen132are representative of selectable target locations on the screen132for alignment of visual marker portions of the viewable image of the loader. The visual marker portions of the viewable image of the loader may be representative of images of one or more physical markers such as for example crosshair markers, manufacturer's logos, or the like that are provided on or at areas of the boom and/or on or at areas of the tool carrier on the end of the boom for example.

The example embodiments described herein are related to the virtual guidance apparatus or controller100performing a method providing visual aids that assist an operator of an associated work vehicle to maneuver a loader of the associated work vehicle to couple an implement or other attachment onto a tool carrier of the loader, and for material handling of an associated workpiece load using the implement on the loader. According to one implementation, information for the visual aids to be displayed on the screen of the display unit are provided by the virtual guidance apparatus or controller100in response to the processor204executing the visual aid logic211comprising one or more sequences of instructions of logic modules contained in main memory206. Such instructions may be read into main memory206from another computer-readable medium, such as storage device210. Execution of the sequences of the visual aid logic211instructions contained in storage device210and/or main memory206causes the processor204to perform the process steps described herein. In an alternative implementation, hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention. Thus implementations of the example embodiments are not limited to any specific combination of hardware circuitry, software, logic, or combinations of hardware, software, and/or logic.

In accordance with the descriptions herein, the term “computer-readable medium” as used herein refers to any non-transitory media that participates in providing visual aid logic211instructions to the processor204for execution. Such a non-transitory medium may take many forms, including but not limited to volatile and non-volatile media. Non-volatile media includes, for example, optical or magnetic disks. Volatile media includes dynamic memory for example and does not include transitory signals, carrier waves, or the like. Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punch cards, papertape, any other physical medium with patterns of holes, a RAM, PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, or any other tangible non-transitory medium from which a computer can read.

In addition and further in accordance with the descriptions herein, the term “logic”, as used herein with respect to the Figures, includes hardware, firmware, software in execution on a machine, and/or combinations of each to perform a function(s) or an action(s), and/or to cause a function or action from another logic, method, and/or system. Logic may include a software controlled microprocessor, a discrete logic (e.g., ASIC), an analog circuit, a digital circuit, a programmed logic device, a memory device containing instructions, and so on. Logic may include one or more gates, combinations of gates, or other circuit components.

The virtual guidance apparatus or controller100further includes a communication interface218coupled with the bus202which provides a two-way data communication coupling to a network link220that is connected to local network212such as for example a local network of the work vehicle1such as a Controller Area Network (CAN) bus or the like. The communication interface218may be a controller area network (CAN) card to provide a data communication connection to a compatible CAN bus. As another example, communication interface218may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. For example, communication interface218may be an integrated services digital network (ISDN) card or a modem to provide a data communication connection to a corresponding type of telephone line. Wireless links may also be implemented. In any such implementation, communication interface218may be a wireless receiver/transmitter, i.e. a transceiver operable to send and receive electrical, electromagnetic, radio frequency (RF), and/or optical signals that carry data streams such as digital data streams representing various types of information.

Network link220typically provides data communication through one or more networks to other data devices. For example, network link220may provide a connection through a local network to a diagnostic host computer (not shown) of the like for supporting configuration of the system as desired or necessary. An Internet Service Provider (ISP)226may provide data communication services indirectly through the Internet via the network212or directly through the network link220.

The example virtual guidance apparatus or controller100can send messages and receive data, including program code, through the network(s), network link220and communication interface218. In the Internet-connected example embodiment, the virtual guidance apparatus or controller100is operatively connected with a plurality of external public, private, governmental or commercial servers (not shown) configured to execute a web application in accordance with example embodiments. The example virtual guidance apparatus or controller100suitably includes several subsystems or modules to perform the virtual operator's guidance as set forth herein. A benefit of the subject application is to provide improved guidance by displaying guidance instructions on a screen132of a display unit130that is viewable from an operator's seat of the associated work vehicle for helping the operator to view current positions of the loader and tool mechanisms as well as to visualize preferred or recommended paths for movement of the loader and tool mechanisms to safely and efficiently couple the tool with the tool carrier and to safely and efficiently load, unload, or otherwise handle material. An output device215may also be provided such as in the form of a sound generating device such as a speaker to help improve guidance assistance by generating audible guidance instructions in the form of audible instructions and/or suitable instructional noises such as beeps, voice messages or the like that can be heard from an operator's seat of the associated work vehicle for helping the operator to be alerted to current positions of the mechanisms as well as to be audibly instructed of preferred or recommended paths for movement of the loader and tool mechanisms to safely and efficiently load, unload, or otherwise couple the desired implement24with the carrier22, and also to handle the material using the implement24received on the carrier22.

The example embodiment of the virtual guidance apparatus further includes an input device214operatively coupled with the guidance control unit. The input device214may be used during a training of the virtual guidance apparatus for receiving a boom position training signal representative of the first predetermined target location on the screen for alignment of the boom target device in the image of the loader to establish the predetermined relative physical position between the boom portion of the loader and the ground supporting the associated work vehicle. In an example embodiment the input device214may include a touchscreen portion134of the display unit130, a pointer device216operatively coupled with the guidance control unit, or any other device or means of communicating training and other information to the control unit110of the virtual guidance assist apparatus100.

In an example embodiment during system training, the operator may first position the boom20to a desired position relative to the vehicle and/or relative to the ground, then drag and drop a virtual indicia movable on the touchscreen134using a pointer or finger onto a selected portion of the image of boom on the screen132while the boom is in the desired position, wherein the virtual indicia comprises the boom position training signal representative of a selectable target location on the screen for alignment of the boom target device in the viewable image of the loader to establish a desired relative physical position between the boom portion of the loader and the vehicle or to the ground supporting the associated work vehicle. In an example embodiment, the virtual indicia may be a crosshair for example. In a further example embodiment, the virtual indicia may be a visual symbol such as a logo, for example. In yet a further example, the virtual indicia may be a static image capture of the boom disposed in a selected desired position relative to the vehicle and/or relative to the ground wherein a subsequent superposition of the static captured image of the boom in the selected desired position with dynamic images of the boom being moved during use of the vehicle cold be used to establish an accurate boom repositioning to the desired position upon movement of the boom to realize a coincidence of the dynamic boom image being displayed on the screen with the static boom target image also being displayed.

Further in an example embodiment, the operator may position the tool carrier22in a desired position relative to the boom20, then drag and drop the virtual indicia movable on the touchscreen134onto a selected portion of the image of boom on the screen132while the tool carrier22is in a desired position relative to the boom20, wherein the virtual indicia comprises the carrier position training signal representative of a selectable target location on the screen for alignment of the carrier target device in the viewable image of the loader to establish a desired relative physical position between the carrier portion of the boom and the vehicle, the ground supporting the associated work vehicle, and/or relative to the boom20. In an example embodiment, the virtual indicia may be a crosshair for example. In a further example embodiment, the virtual indicia may be a visual symbol such as a logo, for example. In yet a further example, the virtual indicia may be a static image capture of the tool carrier22disposed in the desired position relative to the boom20.

Virtual Alignment Target Image Aids Useful to Position Boom Relative to Work Vehicle and/or Relative to Ground

FIGS.3and4show virtual alignment target image visual image aids displayed by the virtual guidance apparatus100in the form of guiding lines useful to assist in positioning the boom20relative to ground supporting the associated work vehicle in accordance with an example embodiment. Although the guiding lines of the example embodiment are cross-hair-type guiding lines, as will be described further herein, it is to be appreciated that other forms of guiding lines, shapes, or other forms or combination of forms such as for example, a silhouette of the boom of the loader arm in the one or more desired position(s), a visual symbol such as a fanciful symbol or a manufacturer's logo or the like may be used as necessary or desired. With reference now to those Figures and with continued reference toFIGS.1and2, as described above, a camera device121operatively coupled with the guidance control unit110is mounted on the associated work vehicle and is operable to obtain an image of the loader10of the associated work vehicle1and to generate loader image data representative of the obtained image of the loader10. The display unit130operatively coupled with the guidance control unit110receives the loader image data and the visual aid data213stored in the memory device210and displays a viewable image300of the loader10of the associated work vehicle based on the loader image data. The viewable image300is displayed on the screen132of the display unit130that is viewable from an operator's seat of the associated work vehicle. The display unit130also displays a first virtual alignment target image310superimposed on the viewable image300of the loader obtained by the camera. The first virtual alignment target image310is representative of a first predetermined target location320on the screen for alignment of a first visual marker portion350of the viewable image300of the loader10.

As will be described below, the first visual marker portion350is a portion of the viewable image300corresponding to an image of a physical boom target device360disposed on the boom and imaged by the camera device121. The viewable image300in the example embodiment is dynamically displayed on the display unit130as the user operates the vehicle to move the loader10and therefore this image together with the first visual marker portion350of the viewable image300change together with the movement of the loader10being imaged by the camera device121. However, the first virtual alignment target image310superimposed on the moving image300including the visual marker portion350of the moving image is statically displayed on the display unit130. In that way, an alignment of the first visual marker portion350of the viewable image300with the first virtual alignment target image310effected by the operator moving the loader10relative to the associated work vehicle1results in the loader being manipulated to the desired position. In an example embodiment, the alignment of the first visual marker portion350of the viewable image300with the first virtual alignment target image310is obtained by achieving a coincidence on the screen132of the display unit130between the first visual marker portion350and the first virtual alignment target image310. In this regard,FIG.3shows a malalignment between the first visual marker portion350and the first virtual alignment target image310while the loader10is being manipulated by the operator of the vehicle and while it is being imaged by the camera device121.FIG.4shows an alignment in the form of a coincidence between the first visual marker portion350and the first virtual alignment target image310indicating that the loader10has been manipulated by the operator of the vehicle to its desired location relative to the vehicle and/or relative to the ground.

In an example embodiment, the first visual marker portion350is a portion of the viewable image300corresponding to an image of a physical target device360disposed on the boom and/or on the tool carrier and imaged by the camera device121wherein the physical boom target device360has one or more physical properties that lend themselves for determination by the virtual guidance apparatus100based on the obtained image of a pose, inclination, orientation, or the like of the physical target device360relative to the camera device121and, hence, also of the pose, inclination, orientation, or the like of the boom and/or of the tool carrier relative to the work vehicle and/or relative to the ground beneath the vehicle owing to a mounting of the physical boom target device360on the boom at a predetermined relative position between the target device and the boom. Physical target devices having such one or more physical properties that lend themselves for determination by the virtual guidance apparatus100based on the obtained image of a pose, inclination, orientation, or the like of the physical target device may include for example signage carrying positional indicia at predefined locations on the signage wherein examples include bar codes, Quick Response (QR) codes having position patterns at the corners of the QR code label, markers, or the like. Manufacturer's logos, or the like may also be provided on or at the boom and/or on or at the tool carrier on the end of the boom for imaging by the camera device121to determine from the obtained image a pose, inclination, orientation, or the like of the boom and/or of the tool carrier relative to the work vehicle and/or relative to the ground beneath the vehicle owing to a mounting of the physical boom target device in the example form of a manufacturer's logo or the like on the boom at a predetermined relative position between the target device and the boom and/or on the tool carrier at a predetermined relative position between the target device and the tool carrier.

In an example embodiment, the camera together with the logic executed by the processor of the guidance control unit of the apparatus may operate as a RTP system utilized to automatically return the loader, boom arm, and implements such as a bucket to a pre-stored position in response to operator input. The camera together with the logic executed by the processor effectively provides for directly or indirectly monitoring the position of the bucket relative to the body of the tractor in lieu of the need for physical implement position sensors such as linear transducers or the like to monitor the stroke position of cylinders. The camera provides the boom stroke position and other data related to the inclination of the implement on the boom in an essentially continual manner. In this regard, the motion of the cylinders37,39,47, and49may be controlled by the virtual guidance apparatus100automatically and/or semi-automatically in response to generalized operator commands such as for example return to position (RTP) commands wherein for example the virtual guidance apparatus100may function in an RTP mode to automatically return the boom20overall, portions of the boom20such as for example the lift arms36,38and/or the implement arms46,48, the various attachments, tools or implements24carried on the tool carrier22, to one or more pre-stored position(s) in response to operator input. In accordance with an embodiment, the system100digitally determines an alignment in the form of a digital coincidence between the first visual marker portion350and the first virtual alignment target image310indicating that the loader10has been manipulated by the operator of the vehicle to its desired location relative to the vehicle and/or relative to the ground.

In the example, both the first virtual alignment target image310as well as the physical boom target device360on the loader represented as the first visual marker portion350may comprise similar cross-hair-type guiding lines so that a coincidence or near identical overlaying may be obtained. However, in further example embodiments, an alignment without exact coincidence between the first visual marker portion350of the viewable image300and the first virtual alignment target image310may be sufficient for establishing that the loader is in the desired position.

For ease of use for the operator and in accordance with a further example embodiment, the display unit130also simultaneously displays an enlarged image300′ of the viewable image300of the loader10described above on the screen132of the display unit that is viewable from an operator's seat of the associated work vehicle. The enlarged image300′ corresponds to a portion of the field of view of the camera device121directed to the first virtual alignment target image310. For ease of reference by the operator, the enlarged image300′ and the regular viewable image300of the loader10are displayed in a screen-within-a-screen fashion. The display unit130also displays an enlarged first virtual alignment target image310′ (FIG.4) superimposed on the image of the loader obtained by the camera when the loader is in the desired position. The enlarged first virtual alignment target image310′ is representative of a first predetermined target location320′ on the screen for alignment of a first visual marker portion350′ of the image of the loader.

Movement of the loader10relative to the associated work vehicle1from the position shown inFIG.3to the position shown in inFIG.4to establish an alignment and preferably a coincidence between the first visual marker portion350of the viewable image300of the loader10displayed on the screen with the first virtual alignment target image310displayed at the first selectable target location320on the screen132corresponds to a first desired relative physical position between the loader10and the associated work vehicle1. Equivalently, movement of the loader10relative to the associated work vehicle1from the position shown inFIG.3to the position shown in inFIG.4to establish a coincidence between the first visual marker portion350of the viewable image300of the loader10displayed on the screen with the first virtual alignment target image310displayed at the first selectable target location320on the screen132corresponds to a first desired relative physical position between the loader10and the ground supporting the associated work vehicle1.

In the example embodiment the display unit is operable to dynamically display the image of the loader of the associated work vehicle obtained by the camera device on the screen of the display unit that is viewable from an operator's seat of the associated work vehicle. Further in the example embodiment the display unit is operable to statically display the first virtual alignment target superimposed on the image of the loader dynamically displayed on the screen. In particular, in the example embodiment, the display unit130is operable to receive the loader image data as the loader10is moved, and to dynamically display the viewable image300of the loader10of the associated work vehicle1on the screen132of the display unit130, and the display unit130is further operable to statically display the first virtual alignment target image310superimposed on the viewable image300of the loader10dynamically displayed on the screen132.

In an example embodiment, the first virtual alignment target image310displayed by the display unit130is representative of a first plurality of selectable target locations320,320′ on the screen132for alignment of a corresponding plurality of articulated boom portions350,350′ of the viewable image300,500,700,800of the loader10, wherein movement of multiple joints of the loader10relative to the associated work vehicle1to sequentially establish a coincidence between the plurality of articulated boom portions350,350′ of the viewable image300of the loader10displayed on the screen132with the first plurality of selectable target locations320,320′ on the screen132corresponds to a first desired relative physical position between one or more of a tool carrier22on an end of a boom20of the loader10and the associated work vehicle1, and/or the tool carrier22on the end of the boom20of the loader10and the ground supporting the associated work vehicle1.

In an example embodiment, the first virtual alignment target image310displayed by the display unit130is representative of a selectable boom target location320′ on the screen132for alignment of a boom portion350′ of the viewable image300,500,700,800of the loader, wherein movement of the loader10relative to the associated work vehicle1to establish a coincidence between the boom portion350′ of the viewable image300,500,700,800of the loader displayed on the screen132with the selectable boom target location320′ on the screen132corresponds to a first desired relative physical position between one or more of a boom20of the loader10and the associated work vehicle1, and/or the boom20of the loader10and the ground supporting the associated work vehicle1.

In an example embodiment, the physical boom target device360has one or more physical properties that lend themselves for determination by the virtual guidance apparatus100based on the obtained image of a pose, inclination, orientation, or the like of the physical boom target device360relative to the camera device121and, hence, also of the pose, inclination, orientation, or the like of the boom and/or of the tool carrier relative to the work vehicle and/or relative to the ground beneath the vehicle owing to a mounting of the physical boom target device360on the boom at a predetermined relative position between the target device and the boom. Physical target devices having such one or more physical properties that lend themselves for determination by the virtual guidance apparatus100based on the obtained image of a pose, inclination, orientation, or the like of the physical target device may include for example signage carrying positional indicia at predefined locations on the signage wherein examples include bar codes, QR codes having position patterns at the corners of the QR code label, markers, or the like. Manufacturer's logos, or the like may also be provided on or at the boom and/or on or at the tool carrier on the end of the boom for imaging by the camera device121to determine from the obtained image a pose, inclination, orientation, or the like of the boom and/or of the tool carrier relative to the work vehicle and/or relative to the ground beneath the vehicle owing to a mounting of the physical boom target device in the example form of a manufacturer's logo or the like on the boom at a predetermined relative position between the target device and the boom and/or on the tool carrier at a predetermined relative position between the target device and the tool carrier.

Further in the example embodiment the virtual guidance apparatus includes a physical boom target device360attached with a torsion support tube the boom10of the loader10of the associated work vehicle1.

The example embodiment of the virtual guidance apparatus further includes an input134,214,216(FIG.2) operatively coupled with the guidance control unit. The input may be used during a training of the virtual guidance apparatus for receiving a boom position training signal representative of the first predetermined target location on the screen for alignment of the boom target device in the image of the loader to establish the predetermined relative physical position between the boom portion of the loader and the ground supporting the associated work vehicle.

In an example embodiment the input134,214,216may include a touchscreen portion134of the display unit130or a pointer device216(FIG.2) operatively coupled with the guidance control unit. The memory210of the guidance control unit110may store visual aid data213corresponding to the boom position training signal received by the input during the training. In particular and in accordance with an example embodiment, the input134,214,216comprises one or more of a touchscreen portion134of the display unit130and/or a pointer device216operatively coupled with the guidance control unit110, and the memory210of the guidance control unit110stores the training data received from the operator via the input134,214,216as the visual aid data213corresponding to the target device training signal received by the input during the operator training of the system100.

Further in the example embodiment, the output device215the system may generate a confirmation sound annunciating the coincidence of the first predetermined target location320with the first visual marker portion350of the image of the loader as shown for example inFIG.4.

Virtual Alignment Target Image Aids Useful to Position a Tool Carrier Relative to Boom

FIGS.5A,5B, and6A,6Bshow visual aids and visual aid images useful to position a tool carrier on a free end of the boom relative to the boom in accordance with an example embodiment. The virtual alignment target image aids of the example embodiment ofFIGS.5A and6Bare cross-hair-type guiding lines590, and the virtual alignment target image aids of the example embodiment ofFIGS.5B and6Bcomprise a visual symbol such as the outline590′ of a manufacturer's logo591that is familiar to the vehicle operator and that provides reassuring comfort during operation of the vehicle.FIG.5Ashows an implement coupled with a tool carrier of a boom in a starting position illustrating a crosshairs visual aid useful to position the tool carrier together with the attached implement relative to the boom in accordance with an example embodiment.FIG.5Bshows the implement coupled with the tool carrier of the boom in the starting position ofFIG.5Aand illustrating a visual symbol visual aid useful to position the tool carrier together with the attached implement relative to the boom in accordance with an example embodiment.FIG.6Ashows the implement coupled with the tool carrier of the boom in a desired working position illustrating the crosshairs visual aid useful to position the tool carrier together with the attached implement relative to the ground and ready for engaging a load in accordance with an example embodiment.FIG.6Bshows the implement coupled with the tool carrier of the boom in the desired working position ofFIG.6Aand illustrating the visual symbol visual aid useful to position the tool carrier together with the attached implement relative to the ground in accordance with an example embodiment. It is to be appreciated however that other forms of guiding lines, shapes, or other forms or combination of forms such as for example, a silhouette of the boom of the loader arm in the one or more desired position(s), other comforting visual symbols, logos or the like may be used as necessary or desired.

With reference now to those Figures and as described above, a camera device121mounted on the associated work vehicle is operable to obtain an image of the loader10of the associated work vehicle1, and the display unit130displays the viewable image500of the loader obtained by the camera device on the screen132of the display unit that is viewable from an operator's seat of the associated work vehicle. The display unit130also displays a second virtual alignment target image510,510′ superimposed on the image of the loader obtained by the camera. The second virtual alignment target510,510′ is representative of a second predetermined target location520,520′ on the screen for alignment of a tool carrier portion of the image of the loader. In the example embodiment the camera device121is mounted on a top surface (e.g., roof) of the cab5of the tractor2. In further example embodiments, the camera device121may be mounted on a hydraulic cylinder powering a mechanical self-leveling loader and, accordingly, movement of the various linkages of the loader mechanism changes the relative position of the camera device relative to the loader torsion support member as the orientation of the tines of the load mechanism relative to the ground is also changed. It is to be appreciated that although only a single camera device121is shown for ease of illustration mounted to the work vehicle1at the position shown, the camera device121shown is merely representative of a set of one or more cameras that may include several connected cameras that can be mounted on or near the vehicle or anywhere as may be necessary or desired such as for example on other areas of the work vehicle or on portions of the loader itself, on the vehicle, on the boom, on the carrier, and/or on the implement fir example in order to obtain as many images of one or more targets on the boom and/or on the tool carrier as may be necessary and/or desired for providing images of a full relevant range of movement or motion of the loader.

The display unit130also simultaneously displays an enlarged image502of the viewable image500described above of the loader obtained by the camera device on the screen132of the display unit that is viewable from an operator's seat of the associated work vehicle. The enlarged image502and the regular viewable image500of the loader are displayed in a screen-within-a-screen fashion. The display unit130also displays an enlarged second virtual alignment target image512,512′ superimposed on the image obtained by the camera. The enlarged second virtual alignment target512,512′ is representative of a first predetermined target location522,522′ on the screen for alignment of a second portion552,552′ of the image of the loader.

Movement of the tool carrier portion of the loader relative to the associated work vehicle from the position shown inFIG.5to the position shown in inFIG.6to establish a coincidence between the tool carrier portion of the image of the loader displayed on the screen and the second predetermined target location520on the screen corresponds to a first predetermined relative physical position between the tool carrier portion of the loader and the ground supporting the associated work vehicle. In the example embodiment, the second virtual alignment target image510,510′ superimposed on the viewable image500of the loader displayed on the screen132is representative of a second selectable target location520,520′ on the screen132for alignment of a tool carrier portion550of the viewable image500of the loader, wherein movement of a carrier portion22of the loader relative10to the associated work vehicle to establish a coincidence between the tool carrier portion550of the viewable image500of the loader displayed on the screen132with the second virtual alignment target image510,510′ displayed at the second selectable target location520,520′ on the screen132corresponds to a desired relative physical position between one or more of the carrier portion22of the loader10and the associated work vehicle1, and/or the carrier portion22of the loader10and the ground supporting the associated work vehicle1.

The second virtual alignment target510displayed by the display unit is representative of a second predetermined target location520on the screen for indirect alignment of tines of the tool carrier portion of the image of the loader.

Movement of the various mechanisms of the loader but without moving the loader relative to the associated work vehicle to establish a coincidence between the tines of the tool carrier portion of the image of the loader displayed on the screen and the second predetermined target location520on the screen corresponds to a first predetermined relative physical position between the tines of the tool carrier portion of the loader and the ground supporting the associated work vehicle.

In an example embodiment, the second virtual alignment target510displayed by the display unit is representative of a second predetermined target location520on the screen for indirect alignment of a bucket in the tool carrier portion of the image of the loader, wherein movement of the loader relative to the associated work vehicle to establish a coincidence between the bucket in the tool carrier portion of the image of the loader displayed on the screen and the second predetermined target location on the screen corresponds to a first predetermined relative physical position between the bucket in the tool carrier portion of the loader and the ground supporting the associated work vehicle. In particular and in accordance with an example embodiment, the second virtual alignment target image510displayed by the display unit130is representative of a second selectable target location on the screen132for indirect alignment of a bucket in the tool carrier portion of the viewable image300,500,700,800of the loader, wherein movement of the loader relative to the associated work vehicle to establish a coincidence between the bucket25in the tool carrier portion of the viewable image of the loader displayed on the screen with the second selectable target location on the screen132corresponds to a desired relative physical position between one or more of the bucket25attached with the tool carrier22of the loader10and the associated work vehicle1, and/or the bucket25on the tool carrier22of the loader10and the ground supporting the associated work vehicle1.

In addition in accordance with the example embodiment, the virtual guidance apparatus includes an input214operatively coupled with the guidance control unit. The input may be used during a training of the virtual guidance apparatus for receiving a tool carrier position training signal representative of the second predetermined target location on the screen for alignment of the tool carrier target device in the image of the loader to establish the predetermined relative physical position between the tool carrier portion of the loader and the ground supporting the associated work vehicle.

In an example embodiment the input214may include a touchscreen portion of the display unit130or a pointer device216(FIG.2) operatively coupled with the guidance control unit. The memory210of the guidance control unit110may store data corresponding to the tool carrier position training signal received by the input during the training. In particular and in accordance with an example embodiment, the input134,214,216comprises one or more of a touchscreen portion134of the display unit130and/or a pointer device216operatively coupled with the guidance control unit110, and the memory210of the guidance control unit110stores training data corresponding to the tool carrier training position signal received by the input134,214,216during the training.

Further in the example embodiment, the output device215the system may generate a confirmation sound annunciating the coincidence of the second predetermined target location520with the second portion550of the image of the loader as shown for example inFIG.4.

In accordance with an embodiment and during RTP operation, the system100digitally determines an alignment in the form of a digital coincidence between the tool carrier portion550of the viewable image500of the loader displayed on the screen132with the second virtual alignment target image510,510′ displayed at the second selectable target location520,520′ on the screen132which corresponds to the desired relative physical position between one or more of the carrier portion22of the loader10and the associated work vehicle1, and/or the carrier portion22of the loader10and the ground supporting the associated work vehicle1.

Visual Aid Images Showing Preview Paths

As mentioned above, visual aids are provided to the operator to assist in positioning the loader of the work vehicle relative to a tool or implement, to assist in positioning the tool carried on the loader with respect to a load, to assist in moving a load carried by the tool or implement relative to a desired placement location, and/or to assist in moving the load from a load placement location by using the tool or implement carried on the loader. In some example embodiments, one or more preview paths are presented to the operator having one or more guidelines displayed over the one or more preview paths and/or overlying the one or more preview paths for movement of the loader relative to the tool, for movement of the vehicle carrying the tool relative to a load, for movement of the vehicle carrying the load relative to a desired unloading location, and/or for movement of the vehicle when removing the load from a load placement location such as from a rack or the like.

In an example embodiment during system training of preview paths, the operator may first position the tool carrier on the boom20to a desired position relative to the tool or implement intended to be attached to the tool carrier, or position the tool relative to the load to be processed, or position the load relative to the unload location, then drag and drop virtual indicia movable on the touchscreen134using a pointer or finger onto a selected portion of the image of boom on the screen132while the selected equipment is in the desired position, wherein the virtual indicia comprises training signals representative of selectable target location on the screen for presentation of one or more guidelines to be displayed on the screen during tool and load handling.

In a further example embodiment and as will be described in greater detail below, the visual aid images showing the preview paths as a pair of preview path guidelines may be generated by the processor204executing the logic211to generate the preview path guidelines extending from one or more virtual planes at the face of the tool carrier, at the interface between the tool carrier and the tool, at the face of the load, and/or at the face of the unloading location.

In an embodiment, the visual aid images showing the one or more preview paths are provided in the form of guidelines extending between a physical feature of the tool carrier such as a physical marker on the tool carrier and a physical feature of the tool or implement intended to be attached to the tool carrier such as a physical marker on the tool. In a further embodiment, the preview of the path is representative of a direct virtual path of the tool carrier. In an embodiment the preview path is provided in the form of guidelines extending between a coupling feature of the tool carrier and a coupling feature of the tool or implement intended to be attached to the tool carrier. In a still further embodiment, the preview of the path is representative of an indirect virtual path of the tool carrier that is not directly physically visible to the operator, wherein the operator may operate the work vehicle based on the displayed virtual path preview for correctly moving the work vehicle relative to the tool or implement intended to be attached to the tool carrier. In an embodiment the preview path is provided in the form of guidelines extending between a geometrical feature of the tool carrier and a physical feature of the tool or implement intended to be attached to the tool carrier such as a physical marker on the tool or implement intended to be attached to the tool carrier. In a further embodiment the preview path is provided in the form of guidelines extending between a virtual feature of the tool carrier and a geometrical feature of the tool or implement intended to be attached to the tool carrier such as a physical feature of the tool or implement intended to be attached to the tool carrier offset from a physical coupling feature of the tool or implement intended to be attached to the tool carrier.

In an embodiment, the visual aid images showing the one or more preview paths are provided in the form of guidelines extending between a physical feature of the implement or tool such as a physical marker on the implement or tool and a physical feature of the target or load such as a physical marker on the target or load. In a further embodiment, the preview of the path is representative of a direct virtual path of the tool carrier or of an implement attached with the tool carrier. In an embodiment the preview path is provided in the form of guidelines extending between a coupling feature of the implement or tool and a coupling feature of the target or load. In a still further embodiment, the preview of the path is representative of an indirect virtual path of an implement attached with the tool carrier that is not directly physically visible to the operator, wherein the operator may operate the work vehicle based on the displayed virtual path preview for correctly moving the work vehicle relative to the load. In an embodiment the preview path is provided in the form of guidelines extending between a geometrical feature of the implement or tool and a physical feature of the target or load such as a physical marker on the target or load. In a further embodiment the preview path is provided in the form of guidelines extending between a virtual feature of the implement or tool and a geometrical feature of the target or load such as a physical feature of the target or load offset from a physical coupling feature of the target or load.

After the load is engaged and raised using the loader10with the implement24coupled therewith, further visual aids described herein are developed by the system and provided to the operator of the associated work vehicle1for assisting in properly placing the load at a desired location such as on associated storage rack, platform or the like. In an embodiment a preview path is provided in the form of guidelines extending between a physical feature of the load as a physical marker on the load and a physical feature of the target location such as a physical marker on the target location. In a further embodiment, the preview of the path is representative of a direct virtual path of the load attached carried by the tool. In an embodiment the preview path is provided in the form of guidelines extending between a coupling feature of the load and a coupling feature of the target location. In a still further embodiment, the preview of the path is representative of an indirect virtual path of the carried load that is not directly physically visible to the operator, wherein the operator may operate the work vehicle based on the displayed virtual path preview for correctly moving the work vehicle carrying the load relative to the desired unloading location. In an embodiment the preview path is provided in the form of guidelines extending between a geometrical feature of the load and a physical feature of the target location such as a physical marker on the target location. In a further embodiment the preview path is provided in the form of guidelines extending between a virtual feature of the load and a geometrical feature of the target location such as a physical feature of the target location offset from a physical coupling feature of the target location.

In an embodiment, the visual aid images showing the one or more preview paths are provided in the form of guidelines extending between a physical feature of the load or of the tool such as a physical marker on the load or tool and a physical feature of a load placement location such as a physical marker disposed on a rack, shelf or the like configured to receive the load therein or thereon. In a further embodiment, the preview of the path is representative of a direct virtual path of the tool or of the load carried by the tool. In an embodiment the preview path is provided in the form of guidelines extending between a coupling feature of the tool or load and a coupling feature of the load placement location. In a still further embodiment, the preview of the path is representative of an indirect virtual path of a load attached or otherwise carried by the tool that is not directly physically visible to the operator, wherein the operator may operate the work vehicle based on the displayed virtual path preview for correctly moving the workpiece load carried by the load relative to the load placement location. In an embodiment the preview path is provided in the form of guidelines extending between a geometrical feature of the load or tool and a physical feature of the load placement location such as a physical marker disposed on a rack, shelf or the like configured to receive the load therein or thereon. In a further embodiment the preview path is provided in the form of guidelines extending between a virtual feature of the load or tool and a geometrical feature of the load placement location such as a physical feature of the placement location offset from a physical structure or area of the placement location. In an embodiment, the visual aid images showing the one or more preview paths are provided in the form of guidelines extending between a physical feature of the load or tool such as a physical marker on the implement or tool and a physical feature of the load placement location such as a physical marker on the target load placement location. In a further embodiment, the preview of the path is representative of a direct virtual path of the tool or of a load carried by the tool. In an embodiment the preview path is provided in the form of guidelines extending between a coupling feature of the load or tool and a coupling feature of the load placement location. In a still further embodiment, the preview of the path is representative of an indirect virtual path of a load carried by the tool that is not directly physically visible to the operator, wherein the operator may operate the work vehicle based on the displayed virtual path preview for correctly moving the load relative to the load placement location. In an embodiment the preview path is provided in the form of guidelines extending between a geometrical feature of the load or tool and a physical feature of the target load placement location such as a physical marker disposed on a rack, shelf or the like configured to receive the load therein or thereon. In a further embodiment the preview path is provided in the form of guidelines extending between a virtual feature of the load tool and a geometrical feature of the target load placement location such as a physical feature of the target load placement location offset from a physical coupling feature of the load placement location.

FIG.7Ashows example preview path visual aids that are useful to illustrate a virtual preview path of an implement attached with the tool carrier on the end of the boom in accordance with an example embodiment and by way of example of the preview path visual aids described above. In this embodiment, the preview path is provided in the form of guidelines extending between a physical feature of the implement or tool such as a physical marker on the implement or tool and a physical feature of the target or load such as a physical marker on the target or load. With reference now to that Figure and as described above, the camera device121mounted on the associated work vehicle is operable to obtain an image of the loader10of the associated work vehicle1. The display unit130displays the viewable image700of the loader obtained by the camera device on the screen132of the display unit that is viewable from an operator's seat of the associated work vehicle. The display unit130also displays on the screen132a third virtual alignment target image710superimposed on the viewable image700of the loader10. The third virtual alignment target image710is representative of a preview path712,714on the screen132of movement to be followed by a tool carrier portion22of the loader10for forward movement of the associated work vehicle1.

The third virtual alignment target image710representative of the preview path712,714displayed on the screen132essentially provides a virtual directional heading image710superimposed on the viewable image of the loader based on the image obtained by the camera, wherein the virtual directional heading image is representative of a preview path on the screen of a movement path to be followed by a tool carrier of the loader for forward movement of the associated work vehicle.

In addition in accordance with the example embodiment, the third virtual alignment target is representative of a pair of spaced apart preview path guidelines712,714of movement on the screen to be followed by tines of an implement attached with the tool carrier portion of the image of the loader for forward movement of the associated work vehicle. In particular and in accordance with an embodiment, the third virtual alignment target image710superimposed on the viewable image700of the loader is representative of a pair of spaced apart preview path guidelines712,714of movement on the screen132to be followed by one or more of tines28of the tool carrier portion22of the image of the loader for forward movement of the associated work vehicle1, and/or a surface of the tool carrier22acting as a pilot for forward movement of the associated work vehicle1.

In the example embodiment shown, the preview of the path is representative of an actual path of one or more implements attached with the tool carrier22on the boom. However, it is to be appreciated that in a further embodiment, the preview of the path is representative of a direct virtual path of one or more implements, tools or other attachments such as for example a bucket on the boom. In a still further embodiment, the preview of the path is representative of an indirect virtual path of an implement attached with the tool carrier22that is not directly physically visible to the operator such as for example a gripper, a grapple, or a bail hugger, wherein the operator may operate the work vehicle based on the virtual path preview for correctly moving the work vehicle towards the load.

Further in accordance with the example embodiment shown, the display unit130also displays a fourth virtual alignment target set800(see alsoFIG.8) superimposed on the image of the workpiece and the implement attached with the tool carrier. The fourth virtual alignment target set may include destination target indicia860,862physically disposed on the workpiece as shown or superimposed on the image of the workpiece (not shown), and presence target indicia820,822superimposed on the image of the implement attached with the tool carrier. The destination target indicia860,862and the presence target indicia820,822are representative of a preview path of movement on the screen to be followed by the associated work vehicle moving forward to effectively couple, prior to lifting the associated workpiece with the loader, the associated workpiece with the implement attached with the tool carrier portion for the presence target indicia820,822being coincident with the destination target indicia860,862. It is to be appreciated that during use in the example embodiment the destination target indicia860,862remain fixed in place relative to the view of the load702displayed on the screen132of the display unit130and viewable by the operator of the tractor2, and the image of the presence target indicia820,822remain fixed in place relative to the loader and also relative to the guidelines712,714. In this way the operator is provided with visual feedback of confirmation that the tool is in a desired position for lifting the load when the presence target indicia820,822and the destination target indicia860,862are made to mutually overly on the screen132of the display unit130by the operator moving the vehicle.

The destination target indicia860,862physically disposed on the workpiece may have one or more physical properties that lend themselves for determination by the virtual guidance apparatus100based on an obtained image of a pose, inclination, orientation, or the like of the physical target indicia860,862relative to the camera device121and, hence, also of the pose, inclination, orientation, or the like of a virtual load connecting plane706disposed at the face of the load702. The virtual load connecting plane706is perpendicular with the ground. Physical target devices having such one or more physical properties that lend themselves for determination by the virtual guidance apparatus100based on the obtained image of a pose, inclination, orientation, or the like of the physical target device may include for example signage carrying positional indicia at predefined locations on the signage wherein examples include bar codes, QR codes having position patterns at the corners of the QR code label, markers, or the like. Manufacturer's logos, or the like may also be provided on or at the load and/or on or at a selected storage location such as a storage rack, platform or the like for imaging by the camera device121to determine from the obtained image a pose, inclination, orientation, or the like of the load and/or of the storage location relative to the work vehicle and/or relative to the tool carried by the vehicle owing to a mounting of the physical destination target indicia device(s) in the example form of a manufacturer's logo or the like on the load at a predetermined relative position on the load and/or on a storage location or storage apparatus such as a storage rack at one or more predetermined positions.

In addition in accordance with the example embodiment, the virtual guidance apparatus includes an input214operatively coupled with the guidance control unit. The input may be used during a training of the virtual guidance apparatus for receiving a path preview training signal representative of the preview path on the screen of movement to be followed by the tool carrier portion of the image of the loader for forward movement of the associated work vehicle.

In an example embodiment the input214may include a touchscreen portion of the display unit130or a pointer device216(FIG.2) operatively coupled with the guidance control unit. The memory210of the guidance control unit110may store data corresponding to the path preview training signal received by the input during the training. In the example embodiments herein the virtual alignment target image(s) such as for example the virtual alignment target image710ofFIG.7Asuperimposed on the viewable image700of the loader10and other virtual alignment target images as will be described below virtual alignment target image may be inputted by the operator during a training of the virtual guidance apparatus for receiving a path preview training signal representative of the preview path on the screen of movement to be followed by the tool carrier portion of the image of the loader for forward movement of the associated work vehicle. It is to be appreciated that the operator may train the system on learning the virtual alignment target image(s) by the operator dragging and dropping the virtual alignment target image(s) onto the screen in alignment with the tool as desired by the operator, and by the operator dragging and dropping the presence target indicia820,822and the destination target indicia860,862onto the screen when the tool is in the desired operation relative to the load. Thereafter the preview path visual aids are automated by the system recalling the stored guidelines and indicia at suitable times during loader operation.

In accordance with an embodiment the preview path visual aids to be displayed on the screen may be digitally compared by logic executed by the processor of the guidance control unit with visual marker portions of the viewable image of the loader in order to determine that the loader has reached a desired conformation relative to the tool, that the tool has reached a desired conformation relative to the load, and/or that the load has reached a desired conformation relative to a load placement location such as s rack, shelf, or the like.

It is still further to be appreciated that the motion of the tractor2may be controlled by the virtual guidance apparatus100automatically and/or semi-automatically in response to generalized operator commands such as for example return to position (RTP) commands wherein for example the virtual guidance apparatus100may function in an RTP mode to automatically return the boom20overall, portions of the boom20such as for example the lift arms36,38and/or the implement arms46,48, the various attachments, tools or implements24carried on the tool carrier22, steering mechanisms of the vehicle, etc. to one or more pre-stored position(s) in response to operator input.

FIG.7Bshows preview path visual aids720useful to illustrate a virtual preview path of an implement attached with the tool carrier on the end of the boom in accordance with an example embodiment, wherein in the embodiment the preview path visual aids720are provided in the form of guidelines722,724extending between coupling features726,728of the implement or tool24and coupling features727,729of the target or load702. In the example embodiment, the guidelines722,724extend perpendicularly from a virtual tool connecting plane704disposed at the interface between the tool carrier and the tool. The virtual tool connecting plane704is perpendicular with the ground. In addition, the guidelines722,724also extend perpendicularly from the virtual load connecting plane706disposed at the face of the load702. The virtual load connecting plane706is perpendicular with the ground.

The third virtual alignment target image720representative of the preview path722,724displayed on the screen132essentially provides a virtual directional heading image720superimposed on the viewable image of the loader based on the image obtained by the camera, wherein the virtual directional heading image is representative of a preview path on the screen of a movement path to be followed by a tool carrier of the loader for forward movement of the associated work vehicle.

Further in accordance with the example embodiment shown, the display unit130also displays an additional virtual alignment target set, such as the set800described above in connection withFIG.7A, superimposed on the image of the workpiece and the implement attached with the tool carrier. The fourth virtual alignment target set of the example embodiment shown inFIG.7Bmay include destination target indicia870,872physically disposed on the workpiece as shown or superimposed on the image of the workpiece (not shown), and presence target indicia830,832superimposed on the image of the implement attached with the tool carrier. The destination target indicia870,872and the presence target indicia830,832are representative of a preview path of movement on the screen to be followed by the associated work vehicle moving forward to effectively couple, prior to lifting the associated workpiece with the loader, the associated workpiece with the implement attached with the tool carrier portion for the presence target indicia830,832being coincident with the destination target indicia870,872. The destination target indicia870,872and the presence target indicia830,832are disposed on the guidelines722,724extending between coupling features726,728of the implement or tool24and coupling features727,729of the target or load702.

FIG.7Cshows preview path visual aids730useful to illustrate a virtual preview path of an implement attached with the tool carrier on the end of the boom in accordance with an example embodiment, wherein in the example embodiment the preview path visual aids730are provided in the form of guidelines732,734extending between geometrical features736,738of the implement or tool24and physical features737,739of the target or load702. The physical features737,739of the target or load702may be physical markers on the target or load for example. In the example embodiment, the guidelines732,734extend perpendicularly from a virtual tool connecting plane704disposed at the interface between the tool carrier and the tool. The virtual tool connecting plane704is perpendicular with the ground. In addition, the guidelines732,734also extend perpendicularly from a virtual load connecting plane706disposed at the face of the load702. The virtual load connecting plane706is perpendicular with the ground.

The physical features737,739of the target or load702may be physical markers on the target or load for example and may have one or more physical properties that lend themselves for determination by the virtual guidance apparatus100based on an obtained image of a pose, inclination, orientation, or the like of the physical target indicia737,739relative to the camera device121and, hence, also of the pose, inclination, orientation, or the like of a virtual load connecting plane706disposed at the face of the load702. The virtual load connecting plane706is perpendicular with the ground. Physical target devices having such one or more physical properties that lend themselves for determination by the virtual guidance apparatus100based on the obtained image of a pose, inclination, orientation, or the like of the physical target device may include for example signage carrying positional indicia at predefined locations on the signage wherein examples include bar codes, QR codes having position patterns at the corners of the QR code label, markers, or the like. Manufacturer's logos, or the like may also be provided on or at the load and/or on or at a selected storage location such as a storage rack, platform or the like for imaging by the camera device121to determine from the obtained image a pose, inclination, orientation, or the like of the load and/or of the storage location relative to the work vehicle and/or relative to the tool carried by the vehicle owing to a mounting of the physical destination target indicia device(s) in the example form of a manufacturer's logo or the like on the load at a predetermined relative position on the load and/or on a storage location or storage apparatus such as a storage rack at one or more predetermined positions.

The third virtual alignment target image730representative of the preview path732,734displayed on the screen132essentially provides a virtual directional heading image730superimposed on the viewable image of the loader based on the image obtained by the camera, wherein the virtual directional heading image is representative of a preview path on the screen of a movement path to be followed by a tool carrier of the loader for forward movement of the associated work vehicle.

Further in accordance with the example embodiment shown, the display unit130also displays an additional virtual alignment target set, such as the set800described above in connection withFIG.7A, superimposed on the image of the workpiece and the implement attached with the tool carrier. The fourth virtual alignment target set of the example embodiment shown inFIG.7Cmay include destination target indicia880,882physically disposed on the workpiece as shown or superimposed on the image of the workpiece (not shown), and presence target indicia840,842superimposed on the image of the implement attached with the tool carrier. The destination target indicia880,882and the presence target indicia840,842are representative of a preview path of movement on the screen to be followed by the associated work vehicle moving forward to effectively couple, prior to lifting the associated workpiece with the loader, the associated workpiece with the implement attached with the tool carrier portion for the presence target indicia840,842being coincident with the destination target indicia880,882. The destination target indicia880,882and the presence target indicia840,842are disposed on the guidelines732,734extending between geometrical features736,738of the implement or tool24and physical features737,739of the target or load702.

FIG.7Dshows preview path visual aids740useful to illustrate a virtual preview path of an implement attached with the tool carrier on the end of the boom in accordance with an example embodiment, wherein in the example embodiment the preview path visual aids740are provided in the form of guidelines742,744extending between virtual features746,748of the implement or tool and geometrical features747,749of the target or load702such as for example one or more physical features of the target or load such as the top opposite left and right edges of the load702offset from a physical coupling features727,729(FIG.7B) of the target or load702. In the example embodiment, the guidelines742,744extend perpendicularly from a virtual tool connecting plane704disposed at the interface between the tool carrier and the tool. The virtual tool connecting plane704is perpendicular with the ground. In addition, the guidelines742,744also extend perpendicularly from a virtual load connecting plane706disposed at the face of the load702. The virtual load connecting plane706is perpendicular with the ground.

The third virtual alignment target image740representative of the preview path742,744displayed on the screen132essentially provides a virtual directional heading image740superimposed on the viewable image of the loader based on the image obtained by the camera, wherein the virtual directional heading image is representative of a preview path on the screen of a movement path to be followed by a tool carrier of the loader for forward movement of the associated work vehicle.

Further in accordance with the example embodiment shown, the display unit130also displays an additional virtual alignment target set, such as the set800described above in connection withFIG.7A, superimposed on the image of the workpiece and the implement attached with the tool carrier. The fourth virtual alignment target set of the example embodiment shown inFIG.7Dmay include destination target indicia890,892physically disposed on the workpiece as shown or superimposed on the image of the workpiece (not shown), and presence target indicia850,852superimposed on the image of the implement attached with the tool carrier. The destination target indicia890,892and the presence target indicia850,852are representative of a preview path of movement on the screen to be followed by the associated work vehicle moving forward to effectively couple, prior to lifting the associated workpiece with the loader, the associated workpiece with the implement attached with the tool carrier portion for the presence target indicia850,852being coincident with the destination target indicia890,892. The destination target indicia890,892and the presence target indicia850,852are disposed on the guidelines742,744extending between virtual features746,748of the implement or tool and geometrical features747,749of the target or load702such as for example one or more physical features of the target or load such as the top opposite left and right edges of the load702offset from a physical coupling features727,729(FIG.7B) of the target or load702.

Visual Aid Images Useful to Position Load to be Manipulated

FIG.8shows further visual aids useful to position the implement attached with the tool carrier on the end of the boom relative to a load to be manipulated in accordance with an example embodiment. With reference now to that Figure together withFIG.7Aand as described above, a camera device121mounted on the associated work vehicle is operable to obtain an image of the loader10of the associated work vehicle1, and the display unit130displays the image800of the loader obtained by the camera device on the screen132of the display unit that is viewable from an operator's seat of the associated work vehicle. The display unit130also displays a fourth virtual alignment target810superimposed on the image of the workpiece and the implement attached with the tool carrier. The fourth virtual alignment target may include destination target indicia860,862physically disposed on the workpiece as shown or superimposed on the image of the workpiece (not shown), and presence target indicia820,822superimposed on the image of the implement attached with the tool carrier. The destination target indicia860,862and the presence target indicia820,822are representative of a preview path of movement on the screen to be followed by the associated work vehicle moving forward to effectively couple, prior to lifting the associated workpiece with the loader, the associated workpiece with the implement attached with the tool carrier portion for the presence target indicia820,822being coincident with the destination target indicia860,862.

In accordance with the example embodiment, the vehicle operator may operate the work vehicle to move forward using as guidance the pair of spaced apart preview path guidelines712,714described above until a coincidence is established on the display unit130between the presence target indicia820,822and the destination target indicia860,862. In the example embodiment, the presence target indicia820,822effectively moves together with the image on the display unit130of the loader and tool relative to the background surroundings being displayed and, similarly, the destination target indicia860,862effectively remains stationary together with the image on the display unit130of the load702relative to the background surroundings being displayed. At the conclusion of movement of the work vehicle carrying the tool along the guidelines712,714the coincidence is established on the display unit130between the presence target indicia820,822and the destination target indicia860,862. This provides visual confirmation to the operator that the tractor and tool are properly positioned and/or otherwise aligned relative to the load, and that the load is ready for lifting with the tool on the loader.

In addition in accordance with the example embodiment, the virtual guidance apparatus includes an input214operatively coupled with the guidance control unit. The input may be used during a training of the virtual guidance apparatus for receiving destination and presence training signals representative of the preview path of movement on the screen to be followed by the associated work vehicle moving forward to effectively couple prior to lifting the associated workpiece with the loader the associated workpiece with the implement attached with the tool carrier portion for the presence target indicia being coincident with the destination target indicia. The training data received includes in the example embodiment data representative of the presence target indicia820,822and data representative of the destination target indicia860,862.

In an example embodiment the input214may include a touchscreen portion of the display unit130or a pointer device216(FIG.2) operatively coupled with the guidance control unit. The memory210of the guidance control unit110may store data corresponding to the destination and presence training signals and/or data received by the input during the training.

It is to be appreciated that the showings ofFIG.8are equivalently applicable to embodiments herein wherein visual aids in the form of one or more preview paths are provided for assisting an operator of a work vehicle to visualize and align portions of a loader of a work vehicle and of material handling attachments detachably coupled to the loader relative to the work vehicle itself, relative to the ground under the work vehicle, and to help visualize the position of one or more attachments on the loader and of material to be loaded or manipulated such as to approach a load and/or tool, pick up the tool and/or the load using the tool, and deposit or otherwise place the load or any other material at a desired location such as at or on the ground at a selected location, at or on a storage rack, platform or the like.

UsingFIG.8as a general template of embodiments herein wherein visual aids in the form of one or more preview paths are provided for assisting an operator of a work vehicle and with additional reference toFIG.7B, in accordance with the example embodiments, the display unit130also displays an additional virtual alignment target set, such as the set800described above in connection withFIG.7A, superimposed on the image of the workpiece and the implement attached with the tool carrier. The fourth virtual alignment target set of the example embodiment shown inFIG.7Bmay include destination target indicia870,872physically disposed on the workpiece as shown or superimposed on the image of the workpiece (not shown), and presence target indicia830,832superimposed on the image of the implement attached with the tool carrier. The destination target indicia870,872and the presence target indicia830,832are representative of a preview path of movement on the screen to be followed by the associated work vehicle moving forward to effectively couple, prior to lifting the associated workpiece with the loader, the associated workpiece with the implement attached with the tool carrier portion. This desired end relative positon is confirmed when there is a coincidence on the display unit130between the presence target indicia830,832and the destination target indicia870,872. The destination target indicia870,872and the presence target indicia830,832are disposed on the guidelines722,724extending between coupling features726,728of the implement or tool24and coupling features727,729of the target or load702.

With continued use ofFIG.8as a general template of embodiments herein wherein visual aids in the form of one or more preview paths are provided for assisting an operator of a work vehicle and with additional reference toFIG.7C, in accordance with the example embodiments, the display unit130also displays an additional virtual alignment target set, such as the set800described above in connection withFIG.7A, superimposed on the image of the workpiece and the implement attached with the tool carrier. The fourth virtual alignment target set of the example embodiment shown inFIG.7Cmay include destination target indicia880,882physically disposed on the workpiece as shown or superimposed on the image of the workpiece (not shown), and presence target indicia840,842superimposed on the image of the implement attached with the tool carrier. The destination target indicia880,882and the presence target indicia840,842are representative of a preview path of movement on the screen to be followed by the associated work vehicle moving forward to effectively couple, prior to lifting the associated workpiece with the loader, the associated workpiece with the implement attached with the tool carrier portion. This desired end relative positon is confirmed when there is a coincidence on the display unit130between the presence target indicia840,842and the destination target indicia880,882. The destination target indicia880,882and the presence target indicia840,842are disposed on the guidelines732,734extending between geometrical features736,738of the implement or tool24and physical features737,739of the target or load702.

With continued use ofFIG.8as a general template of embodiments herein wherein visual aids in the form of one or more preview paths are provided for assisting an operator of a work vehicle and with additional reference toFIG.7D, in accordance with the example embodiments, the display unit130also displays an additional virtual alignment target set, such as the set800described above in connection withFIG.7A, superimposed on the image of the workpiece and the implement attached with the tool carrier. The fourth virtual alignment target set of the example embodiment shown inFIG.7Dmay include destination target indicia890,892physically disposed on the workpiece as shown or superimposed on the image of the workpiece (not shown), and presence target indicia850,852superimposed on the image of the implement attached with the tool carrier. The destination target indicia890,892and the presence target indicia850,852are representative of a preview path of movement on the screen to be followed by the associated work vehicle moving forward to effectively couple, prior to lifting the associated workpiece with the loader, the associated workpiece with the implement attached with the tool carrier portion for the presence target indicia850,852being coincident with the destination target indicia890,892. The destination target indicia890,892and the presence target indicia850,852are disposed on the guidelines742,744extending between virtual features746,748of the implement or tool and geometrical features747,749of the target or load702such as for example one or more physical features of the target or load such as the top opposite left and right edges of the load702offset from a physical coupling features727,729(FIG.7B) of the target or load702.

As described above and in accordance with further example embodiments, the visual aid images showing the one or more preview paths are provided in the form of guidelines extending between a physical feature of the load or of the tool such as a physical marker on the load or tool and a physical feature of a load placement location such as a physical marker disposed on a rack, shelf or the like configured to receive the load therein or thereon. In a further embodiment, the preview of the path is representative of a direct virtual path of the tool or of the load carried by the tool. In an embodiment the preview path is provided in the form of guidelines extending between a coupling feature of the tool or load and a coupling feature of the load placement location. In a still further embodiment, the preview of the path is representative of an indirect virtual path of a load attached or otherwise carried by the tool that is not directly physically visible to the operator, wherein the operator may operate the work vehicle based on the displayed virtual path preview for correctly moving the workpiece load carried by the load relative to the load placement location. In an embodiment the preview path is provided in the form of guidelines extending between a geometrical feature of the load or tool and a physical feature of the load placement location such as a physical marker disposed on a rack, shelf or the like configured to receive the load therein or thereon. In a further embodiment the preview path is provided in the form of guidelines extending between a virtual feature of the load or tool and a geometrical feature of the load placement location such as a physical feature of the placement location offset from a physical structure or area of the placement location. In an embodiment, the visual aid images showing the one or more preview paths are provided in the form of guidelines extending between a physical feature of the load or tool such as a physical marker on the implement or tool and a physical feature of the load placement location such as a physical marker on the target load placement location. In a further embodiment, the preview of the path is representative of a direct virtual path of the tool or of a load carried by the tool. In an embodiment the preview path is provided in the form of guidelines extending between a coupling feature of the load or tool and a coupling feature of the load placement location. In a still further embodiment, the preview of the path is representative of an indirect virtual path of a load carried by the tool that is not directly physically visible to the operator, wherein the operator may operate the work vehicle based on the displayed virtual path preview for correctly moving the load relative to the load placement location. In an embodiment the preview path is provided in the form of guidelines extending between a geometrical feature of the load or tool and a physical feature of the target load placement location such as a physical marker disposed on a rack, shelf or the like configured to receive the load therein or thereon. In a further embodiment the preview path is provided in the form of guidelines extending between a virtual feature of the load tool and a geometrical feature of the target load placement location such as a physical feature of the target load placement location offset from a physical coupling feature of the load placement location.

Virtual Guidance Method

FIG.9is a flow diagram showing a virtual guidance method900for assisting an operator of an associated work vehicle to maneuver portions of the associated work vehicle for material handling of an associated workpiece in accordance with an example embodiment.

The virtual guidance method900comprises obtaining910by a camera device mounted on the associated work vehicle an image of a loader of the associated work vehicle and, in particular, to obtaining the image of the loader of the associated work vehicle in the context of the surroundings particularly the surroundings where the implement attached with the tool carrier will be performing the work; and displaying920on a screen of a display unit such as for example an overhead display that is viewable from an operator's seat of the associated work vehicle and that is operatively coupled with a guidance control unit comprising a processor and a memory operatively coupled with the processor: the image of the loader of the associated work vehicle obtained by the camera device; and a first virtual alignment target superimposed on the image of the loader obtained by the camera, the first virtual alignment target being representative of a first predetermined target location on the screen for alignment of a first visual marker portion of the image of the loader, wherein movement of the loader relative to the associated work vehicle to establish a coincidence between the first visual marker portion of the image of the loader displayed on the screen and the first predetermined target location on the screen corresponds to a first predetermined relative physical position between: the loader and the associated work vehicle; and/or the loader and ground supporting the associated work vehicle.

If the boom is determined in step930to be properly positioned based on the above, the virtual guidance method further comprises displaying940on the screen of the display unit: a second virtual alignment target superimposed on the image of the loader obtained by the camera, the second virtual alignment target being representative of a second predetermined target location on the screen for alignment of a tool carrier portion of the image of the loader, wherein movement of the tool carrier portion of the loader relative to the associated work vehicle to establish a coincidence between the tool carrier portion of the image of the loader displayed on the screen and the second predetermined target location on the screen corresponds to a first predetermined relative physical position between the tool carrier portion of the loader and the ground supporting the associated work vehicle.

If the tool carrier is determined in step950to be properly positioned, the virtual guidance method further comprises displaying960on the screen of the display unit: a third virtual alignment target superimposed on the image of the loader obtained by the camera, the third virtual alignment target being representative of a preview path on the screen of movement to be followed by a tool carrier portion of the image of the loader for forward movement of the associated work vehicle towards tool or implement for coupling the tool or implement with the tool carrier. In a further example embodiment the third virtual alignment target is representative of preview path on the screen of movement to be followed by a tool or implement portion of the image of the loader for forward movement of the associated work vehicle towards a load or workpiece such as for example after the tool is coupled so that the load may be properly approached and lifted. In a further example embodiment the third virtual alignment target is representative of preview path on the screen of movement to be followed by a load or workpiece carried on the tool or implement portion of the image of the loader for forward movement of the associated work vehicle carrying the load towards the load placement location such as a rack or the like. In a further example embodiment the third virtual alignment target is representative of preview path on the screen of movement to be followed by a tool or implement portion of the image of the loader for forward movement of the associated work vehicle towards a load or workpiece at a load placement location such as when seeking to retrieve a load from a storage location.

As shown inFIG.8for example, a virtual load engagement target image810is superimposed on the viewable image of the workpiece and tool carrier. The virtual load engagement target image includes destination target indicia860,862superimposed on the image of the workpiece and presence target indicia820,822superimposed on the image of the tool carrier. The destination and presence target indicia are representative of preview paths of movement on the screen to be followed by the associated work vehicle moving forward to, for example, effectively couple prior to lifting the associated workpiece with the loader the associated workpiece with an implement on the tool carrier for the presence target indicia820,822being coincident with the destination target indicia860,862in the viewable image. In addition, the destination and presence target indicia are representative in an example embodiment of preview paths of movement on the screen to be followed by the associated work vehicle moving forward to, for example, effectively place the tool and load in proper alignment for lifting the load. In addition, the destination and presence target indicia are representative in an example embodiment of preview paths of movement on the screen to be followed by the associated work vehicle moving forward to, for example, effectively place the load in proper alignment for lifting the load.

If the operator determines in step970that the preview path was followed, the virtual guidance method further comprises: obtaining by the camera device an image of the tool carried on the loader of the associated work vehicle; obtaining by the camera device an image of the associated workpiece adjacent to the tool; and displaying980on the screen of the display unit: the image of the tool carried on the loader of the associated work vehicle obtained by the camera device; the image of the associated workpiece adjacent to the tool; and a fourth virtual alignment target superimposed on the image of the workpiece and tool carrier, the fourth virtual alignment target comprising destination target indicia superimposed on the image of the workpiece and presence target indicia superimposed on the image of the tool or tool carrier, the destination and presence target indicia being representative of a preview path of movement on the screen to be followed by the associated work vehicle moving forward to effectively couple prior to lifting the associated workpiece with the loader the associated workpiece with the tool portion for the presence target indicia being coincident with the destination target indicia.

The virtual guidance method further comprises: obtaining by the camera device an image of the load carried by the tool; obtaining by the camera device an image of the associated storage location; and displaying980on the screen of the display unit: the image of the load, the image of the storage location; and a fourth virtual alignment target superimposed on the image of the load and storage location, the fourth virtual alignment target comprising destination target indicia superimposed on the image of the storage location and presence target indicia superimposed on the image of the load, the destination and presence target indicia being representative of a preview path of movement on the screen to be followed by the associated work vehicle moving forward to effectively place the load into or on the storage location such as a rack for example, for the presence target indicia being coincident with the destination target indicia.

If it is determined in step990that the tool carrier was properly manipulated relative to the tool or implement, that the tool was properly manipulated relative to the load, and/or that the load was properly manipulated relative to the storage location, the method ends, concludes, or is terminated.

It is to be understood that other embodiments will be utilized and structural and functional changes will be made without departing from the scope of the present invention. The foregoing descriptions of embodiments of the present invention have been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Accordingly, many modifications and variations are possible in light of the above teachings. It is therefore intended that the scope of the invention be limited not by this detailed description.