Patent ID: 12214725

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

A tow vehicle, such as, but not limited to a car, a crossover, a truck, a van, a sports-utility-vehicle (SUV), and a recreational vehicle (RV) may be configured to tow a trailer. The tow vehicle connects to the trailer by way of a trailer hitch. It may be difficult to driver the vehicle-trailer system in a rearward direction. Therefore, it is desirable to have a system that provides the driver with a panoramic view of the rear and side of the trailer which allows the driver to have a wider range of motion because of the area the driver is able to see.

Referring toFIGS.1A,1B, and2, in some implementations, a vehicle-trailer system100includes a tow vehicle102attached to a trailer106. The tow vehicle102includes a vehicle tow ball104supported by a vehicle hitch bar105. The vehicle tow ball104is coupled to the trailer106by way of a trailer hitch coupler108supported by a trailer hitch bar109of the trailer106. The tow vehicle102may include a drive system110that maneuvers the tow vehicle102across a road surface based on drive commands having x, y, and z components, for example. As shown, the drive system110includes a front right wheel112,112a, a front left wheel112,112b, a rear right wheel112,112c, and a rear left wheel112,112d. The drive system110may include other wheel configurations as well. The drive system110may also include a brake system (not shown) that includes brakes associated with each wheel112,112a-d, and an acceleration system (not shown) that is configured to adjust a speed and direction of the tow vehicle102. In addition, the drive system110may include a suspension system (not shown) that includes tires associates with each wheel112,112a-d, tire air, springs, shock absorbers, and linkages that connect the tow vehicle102to its wheels112,112a-dand allows relative motion between the tow vehicle102and the wheels112,112a-d.

The tow vehicle102may move across the road surface by various combinations of movements relative to three mutually perpendicular axes defined by the tow vehicle102: a transverse axis XV, a fore-aft axis YV, and a central vertical axis ZV. The transverse axis XVextends between a right-side and a left-side of the tow vehicle102. A forward drive direction along the fore-aft axis YVis designated as Fv, also referred to as a forward motion. In addition, an aft or rearward drive direction along the fore-aft direction YVis designated as RV, also referred to as rearward motion. In some examples, the tow vehicle102includes a suspension system (not shown), which when adjusted causes the tow vehicle102to tilt about the XVaxis and or the YVaxis, or move along the central vertical axis ZV.

Moreover, the trailer106follows the tow vehicle102across the road surface by various combinations of movements relative to three mutually perpendicular axes defined by the trailer106: a trailer transverse axis XT, a trailer fore-aft axis YT, and a trailer central vertical axis ZT. The trailer transverse axis XTextends between a right-side and a left-side of the trailer106along a trailer turning axle107. In some examples, the trailer106includes a front axle (not shown) and rear axle107. In this case, the trailer transverse axis XTextends between a right-side and a left-side of the trailer106along a midpoint of the front and rear axle (i.e., a virtual turning axle). A forward drive direction along the trailer fore-aft axis YTis designated as FT, also referred to as a forward motion. In addition, a trailer aft or rearward drive direction along the fore-aft direction YTis designated as RT, also referred to as rearward motion. Therefore, movement of the vehicle-trailer system100includes movement of the tow vehicle102along its transverse axis XV, fore-aft axis YV, and central vertical axis ZV, and movement of the trailer106along its trailer transverse axis XT, trailer fore-aft axis YT, and trailer central vertical axis ZT. Therefore, when the tow vehicle102makes a turn as it moves in the forward direction Fv, then the trailer106follows along. While turning, the tow vehicle102and the trailer106form a trailer angle α being an angle between the vehicle fore-aft axis YVand the trailer fore-aft axis Yr.

The tow vehicle102may include a user interface120. The user interface120may include a display122, a knob, and a button, which are used as input mechanisms. In some examples, the display122may show the knob and the button. While in other examples, the knob and the button are a knob button combination. In some examples, the user interface120receives one or more driver commands from the driver via one or more input mechanisms or a touch screen display122and/or displays one or more notifications to the driver. The user interface120is in communication with a controller140. In some examples, the display122displays an image133of an environment of the tow vehicle102which includes the trailer articulation view152as shown inFIG.6.

The tow vehicle102may include a sensor system130to provide reliable and robust driving. The sensor system130may include different types of sensors that may be used separately or with one another to create a perception of the environment of the tow vehicle102and the trailer106that is used for the tow vehicle102to drive and aid the driver in make intelligent decisions based on objects and obstacles detected by the sensor system130. The sensor system130may include the one or more cameras132,132a-dsupported by the vehicle-trailer system100. In some implementations, the tow vehicle102includes a rear vehicle camera132a(i.e., a first camera) that is mounted to provide a view of a rear-driving path for the tow vehicle102, or in other words, the rear vehicle camera132acaptures images133of a rear environment of the tow vehicle102. Additionally, the sensor system130includes a trailer rear camera132b(i.e., a second camera) that is mounted to provide a view of a rear-driving path for the trailer106, or in other words, the rear trailer camera132bcaptures images133of a rear environment of the trailer106. In some examples, the sensor system130also includes side trailer camera132c,132d(i.e., third camera and fourth camera) each mounted to provide a side images133of the side environment of the trailer106. In some examples, additional one or more cameras132are positioned on a front of the tow vehicle102to capture a forward-driving path of the tow vehicle102and/or on the sides of the tow vehicle102to capture a side environment of the tow vehicle102.

In some implementation, the rear vehicle camera132a, the rear trailer camera132b, and the side trailer cameras132c,132dinclude a fisheye lens having an ultra-wide-angle lens that produces strong visual distortion intended to create a wide panoramic or hemispherical image. Fisheye cameras capture images having an extremely wide angle of view. Moreover, images captured by the fisheye camera have a characteristic convex non-rectilinear appearance. Other types of cameras may also be used to capture the images133.

The sensor system130may also include other sensors134that detect the vehicle motion, i.e., speed, angular speed, position, etc. The other sensors134may include an inertial measurement unit (IMU) configured to measure the vehicle's linear acceleration (using one or more accelerometers) and rotational rate (using one or more gyroscopes). In some examples, the IMU also determines a heading reference of the tow vehicle102. Therefore, the IMU determines the pitch, roll, and yaw of the tow vehicle102. The other sensors134may also include, but are not limited to, radar, sonar, LIDAR (Light Detection and Ranging, which can entail optical remote sensing that measures properties of scattered light to find range and/or other information of a distant target), LADAR (Laser Detection and Ranging), ultrasonic, HFL (High Resolution 3D Flash LIDAR), etc. In some implementations, the sensor system130may provide external sensor data received from other systems or vehicles, such as by way of V2X communication or any other communication.

The controller140includes a computing device (or processor)142(e.g., central processing unit having one or more computing processors) in communication with non-transitory memory144(e.g., a hard disk, flash memory, random-access memory) capable of storing instructions executable on the computing processor(s)142. The controller may be supported by the tow vehicle102, the trailer106, or both the tow vehicle102and the trailer106. In some examples, the controller140executes an imaging system150that provides the driver of the tow vehicle102with a panoramic view152of the environment behind the trailer106.

The imaging system150receives images133from one or more camera132and provides a panoramic view152of the rear environment of the trailer106. The imaging system150solves the difficulties that the driver faces when backing up the vehicle-trailer system100by showing a panoramic view152of the rear environment of the trailer106on the display122. The panoramic view152includes images133captured by the rear trailer camera132band the side trailer cameras132c,132d. The panoramic view152may be stitched to or overlain on an image133acaptured by the rear vehicle camera132awithin a boundary124of a trailer representation126on the display122as shown inFIG.6.

Referring toFIG.3, the imaging system150receives images133,133bfrom the rear trailer camera132b(i.e., second camera), images133,133cfrom the right-side trailer camera132c, and images133,133dfrom the left-side trailer camera132d. The imaging system150stitches the received images133b,133c,133dto create a panoramic view152shown inFIG.4. In some examples, the panoramic view152is displayed on the display122. In other examples, the panoramic view152is displayed within the boundary124of the trailer representation126of an image133acaptured by the rear vehicle camera132aon the display122as shown inFIG.6. In other words, the display122shows the image133acaptured by the rear vehicle camera132aand projects the panoramic view152onto a trailer body representation of the trailer106within the image133a.

Referring toFIGS.5A and5B, in some implementations, the controller140determines the trailer angle α based on sensor system data131from the sensor system130. For example, the controller140determines the trailer angle α based on one or more images133areceived from the rear vehicle camera132aand/or images received from a camera positioned on a front portion of the trailer106. Additionally or alternatively, the controller140may determine the trailer angle α based on sensor data134received from one or more sensors134supported by the rear vehicle and/or the front portion of the trailer106.

In some implementations, the imaging system150selects a viewing area154of the panoramic view152for display based on the trailer angle α. For example, referring toFIG.5A, when the tow vehicle102is moving in a straight direction (forward or backward), then the imaging system150selects the viewing area154of the panoramic view152that includes the image133bfrom the rear trailer camera132band a section of each of the side images133c,133dcaptured by the side trailer cameras132c,132d. Referring toFIG.5B, when the tow vehicle102is turning towards the right (forward or backward direction), then the imaging system150selects the viewing area154of the panoramic view152that includes the image133c(or a section thereof) from the right-side trailer camera132cand a section of the image133bfrom the rear trailer camera132b. Similarly, when the tow vehicle102is turning towards the left (forward or backward direction), then the imaging system150selects the viewing area154of the panoramic view152that includes the image133d(or a section thereof) from the left-side trailer camera132dand a section of the image133bfrom the rear trailer camera132b. The imaging system150utilizes the trailer angle α to scroll the panoramic view154and display a natural image or viewing area154to the driver while driving in reverse as shown inFIGS.5A and5B.

In some implementations, the driver may adjust the viewing area154displayed on the display122by way of the user input120. Therefore, the driver may adjust the viewing area154shown inFIG.5Ato view more of the image133ccaptured by the right-side trailer camera132cas shown inFIG.5B. Other adjustments may also be possible.

FIG.5provides an example arrangement of operations for a method700of providing a panoramic view152of an environment behind a trailer106of a vehicle-trailer system100shown inFIGS.1A-6. At block702, the method700includes receiving, at data processing hardware140,142, a first image133,133bfrom a rear trailer camera132,132bpositioned on a rear portion of the trailer106. The first image133,133bcaptures a rearward environment of the trailer106. At block704, the method700includes receiving, at the data processing hardware140,142, a second image133,133cfrom a right-side trailer camera132,132cpositioned on a right-side portion of the trailer106. The second image133,133ccaptures a right-side environment of the trailer106. At block706, the method700includes receiving, at the data processing hardware140,142, a third image133,133dfrom a left-side trailer camera132,132dpositioned on a left-side portion of the trailer106. The third image133,133ccaptures a left-side environment of the trailer106. At block708, the method700includes determining, at the data processing hardware140,142, a panoramic view152based on the first image133,133b, the second image133,133c, and the third image133,133d. In some examples, the method700determines the panoramic view152by stitching the first image133,133b, the second image133,133c, and the third image133,133d. Other methods of determining the panoramic view may also be used. At block710, the method700includes determining, at the data processing hardware140,142, a trailer angle α based on sensor system data131received from a sensor system130. The trailer angle α being an angle between a vehicle fore-aft axis YVand a trailer fore-aft axis YT. At block712, the method700includes determining, at the data processing hardware140,142, a viewing area154within the panoramic view152based on the trailer angle α. At block714, the method700includes sending, from the data processing hardware140,142to a display122in communication with the data processing hardware140,142, instructions156to display the viewing area154.

In some implementations, the method700also includes receiving a fourth image133,133afrom a rear vehicle camera132,132apositioned on a rear portion of a vehicle102of the vehicle-trailer system100. The fourth image133,133acaptures and includes a trailer representation126of the trailer106. The method700may also include projecting the panoramic view152or the viewing area154of the panoramic view152within a boundary124of the trailer representation126of the fourth image133,133a. Therefore, in some examples, the instructions156to display the viewing area154of the panoramic view152include displaying the viewing area154of the panoramic view152within the boundary124of the trailer representation126of the fourth image133,133a.

In some examples, the rear trailer camera132,132b, the right-side trailer camera132,132c, the left-side trailer camera132,132d, and the rear vehicle camera132,132aeach include a fisheye lens. The camera132may include other types of cameras132.

In some implementations, the method700also includes receiving a driver input via a user interface120and adjusting the viewing area154of the panoramic view152displayed on the display122based on the driver input. The method700may include receiving a driver input via a user interface120indicative of a driver request to view the panoramic view152and adjusting the viewing area154to include the panoramic view152based on the driver input.

Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor and can be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, apparatus and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal used to provide machine instructions and/or data to a programmable processor.

Implementations of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Moreover, subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The terms “data processing apparatus”, “computing device” and “computing processor” encompass all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to suitable receiver apparatus.

Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multi-tasking and parallel processing may be advantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the disclosure. Accordingly, other implementations are within the scope of the following claims.