Patent Publication Number: US-2021188350-A1

Title: System for road slope compensation using camera information and method thereof

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
     This application claims priority from and the benefit of Korean Patent Application No. 10-2019-0170245, filed on Dec. 18, 2019, which is hereby incorporated by reference for all purposes as if set forth herein 
     BACKGROUND 
     Field 
     Exemplary embodiments inventive concepts relate to a road slope compensation system using camera information, and a method thereof, and more particularly, relate to a road slope compensation system using camera information that determines a road slope based on camera information and lateral acceleration of a vehicle, estimates the road slope, and assists an Advanced Driving Assistance System (ADAS) driving convenience controller in order to drive the vehicle to the center of a lane on a road where the road slope is present, and a method thereof. 
     Discussion of the Background 
     ADAS refers to an advanced driver assistance system for assisting a driver&#39;s driving, and is configured to sensing situations in front of a vehicle, to determine situations based on the sensed result, and to control the behavior of a vehicle based on the situation determination. In particular, a lane tracking assistance system refers to a system that automatically controls a steering device to drive in the center of a lane. 
     The lane tracking assistance system may grasp the location of a vehicle in a lane by using a sensor recognizing a lane, may calculate the required steering angle necessary to position the vehicle in the center of the lane, and then may calculate a steering torque for tracking the required steering angle to control the steering of the vehicle depending on the calculated steering torque. However, there is a difference between the rotation torque and restoration torque of the steering system when a road slope or dispersion of a steering system occurs. As the steering angle is generated in the direction of the large rotation torque, the vehicle may be inclined to one side. 
     There is a prior art disclosed as Korean Registered Patent Publication No. 10-1567207 (Patent Document 1). 
     The above information disclosed in this Background section is only for enhancement of understanding of the background of the invention and, therefore, it may contain information that does not constitute prior art. 
     SUMMARY 
     Features of the inventive concepts will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the inventive concepts. 
     An aspect of the inventive concepts provides a road slope compensation system using camera information that determines a road slope based on camera information and lateral acceleration of a vehicle, estimates the road slope, compensates the estimated road slope to an ADAS driving convenience system, and thus prevents the vehicle from being inclined to the road slope in a section where a road slope is present, thereby securing the driving stability of the vehicle by driving the vehicle in the middle of the lane on a road having the road slope, and a method thereof. 
     The technical problems to be solved by the present inventive concept are not limited to the aforementioned problems, and any other technical problems not mentioned herein will be clearly understood from the following description by those skilled in the art to which the inventive concepts pertains. 
     According to an aspect of the inventive concepts, a road slope compensation system using camera information may include a camera module configured to capture and obtain a front image of a vehicle, a vehicle sensor device configured to sense and obtain state information of the vehicle, a compensation calculation device configured to calculate a compensation yawrate, using a sensor signal delivered from the vehicle sensor device and a heading error by the front image delivered from the camera module, when the vehicle is driving on a road having a road slope, and a correction controller configured to calculate a final yawrate by applying a compensation yawrate delivered from the compensation calculation device, to a target yawrate calculated by using the sensor signal, a line curvature yawrate by the front image, and a current heading, and calculating a final steering torque by the final yawrate to control a steering device. 
     In an embodiment, the compensation calculation device may calculate an estimated heading, using the line curvature yawrate and the current heading, may calculate the heading error, using the estimated heading and the current heading, and may calculate a compensation yawrate, using the heading error. 
     In an embodiment, the curvature yawrate may be calculated by following Equation “Curvature Yawrate=Measured Yawrate−(Curvature*Vs)” (Here, the measured yawrate is a measured value by a yawrate sensor, a curvature is a curvature radius of the road having the road slope, and Vs is a speed of the vehicle). 
     In an embodiment, the estimated heading may be calculated by following Equation “Estimated Heading=f to   t1  Curvature Yawrate dt+Current Heading” (t 0  is a point in time when line control is activated, and t 1  is a point in time when a reset condition is determined). 
     In an embodiment, the heading error may be calculated by following Equation “Heading error=Estimated Heading−Current Heading”. 
     In an embodiment, the compensation yawrate may be calculated by following Equation “Compensation Yawrate=Heading error*Compensation Gain”. 
     In an embodiment, the final yawrate may be calculated by following Equation 
     “Final Yawrate=Target Yawrate+Compensation Yawrate”. 
     In an embodiment, the correction controller may control a steering device after calculating the final steering torque by using the compensation yawrate to compensate for a performance difference between left tracking and right tracking of the vehicle, which is caused by the road slope. 
     In an embodiment, the final steering torque may be calculated by following Equation “Final steering Torque=Existing torque+Compensation Torque”, and the compensation Torque may be calculated by following Equation “Compensation Torque=Gain*Compensation Yawrate”. 
     According to an aspect of the inventive concepts, a road slope compensation method using camera information may include calculating a compensation yawrate, using a sensor signal delivered from a vehicle sensor device sensing and obtaining state information of a vehicle, and a heading error by a front image delivered from the camera module capturing and obtaining the front image of the vehicle, when the vehicle is driving on a road having a road slope, and calculating, by a correction controller, a final yawrate by applying a compensation yawrate delivered from the compensation calculation device, to a target yawrate calculated by using the sensor signal, a line curvature yawrate by the front image, and a current heading, and calculating a final steering torque by the final yawrate to control a steering device. 
     In an embodiment, the calculating of the compensation yawrate may include calculating an estimated heading, using a sensor signal, the line curvature yawrate by the front image, and the current heading, calculating the heading error, using the estimated heading and the current heading, and calculating a compensation yawrate, using the heading error. 
     In an embodiment, the calculating of the compensation yawrate may include calculating the curvature yawrate by following Equation “Curvature Yawrate=Measured Yawrate−(Curvature*Vs)” (Here, the measured yawrate is a measured value by a yawrate sensor; a curvature is a curvature radius of the road having the road slope, and Vs is a speed of the vehicle). 
     In an embodiment, the calculating of the compensation yawrate may include calculating the estimated heading by following Equation “Estimated Heading=∫ t0   t1  Curvature Yawrate dt+Current Heading” (t 0  is a point in time when line control is activated, and t 1  is a point in time when a reset condition is determined). 
     In an embodiment, the calculating of the compensation yawrate may include calculating the heading error by following Equation “Heading error=Estimated Heading−Current Heading”. 
     In an embodiment, the calculating of the compensation yawrate may include calculating the compensation yawrate by following Equation “Compensation Yawrate=Heading error*Compensation Gain”. 
     In an embodiment, the controlling of the steering device may include calculating the final yawrate by following Equation “Final Yawrate=Target Yawrate+Compensation Yawrate”. 
     In an embodiment, the controlling of the steering device may include controlling, by the correction controller, the steering device after calculating a final steering torque by using the compensation yawrate to compensate for a performance difference between left tracking and right tracking of the vehicle, which is caused by the road slope. 
     In an embodiment, the controlling of the steering device may include calculating the final steering torque by following Equation “Final steering Torque=Existing torque+Compensation Torque”, and calculating the compensation torque by following Equation “Compensation Torque=Gain*Compensation Yawrate”. 
     It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention, and together with the description serve to explain the principles of the invention. 
       The above and other objects, features and advantages of the inventive concepts will be more apparent from the following detailed description taken in conjunction with the accompanying drawings: 
         FIG. 1  is a block diagram illustrating a road slope compensation system using camera information according to an embodiment of the inventive concepts; 
         FIG. 2  is a view illustrating generation of lateral acceleration in a vehicle, to which a road slope compensation system using camera information is applied, due to a road slope on a road according to an embodiment of the inventive concepts; 
         FIGS. 3A, 3B, 3C, and 3D  are graphs illustrating progress status of a vehicle before a road slope compensation system using camera information is applied, according to an embodiment of the inventive concepts; 
         FIGS. 4A, 4B, 4C, and 4D  are graphs illustrating progress status of a vehicle after a road slope compensation system using camera information is applied, according to an embodiment of the inventive concepts; and 
         FIG. 5  is a flowchart illustrating a road slope compensation method using camera information according to an embodiment of the inventive concepts. 
     
    
    
     DETAILED DESCRIPTION OF THE ILLUSTRATED EMBODIMENTS 
     The invention is described more fully hereinafter with reference to the accompanying drawings, in which embodiments of the invention are illustrated. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure is thorough, and will fully convey the scope of the invention to those skilled in the art. Like reference numerals in the drawings denote like elements. 
     Various advantages and features of the present invention and methods accomplishing thereof will become apparent from the following description of embodiments with reference to the accompanying drawings. However, the present invention is not be limited to the embodiments set forth herein but may be implemented in many different forms. The present embodiments may be provided so that the disclosure of the present invention will be complete, and will fully convey the scope of the invention to those skilled in the art and therefore the present invention will be defined within the scope of claims. Like reference numerals throughout the description denote like elements. 
     Unless defined otherwise, it is to be understood that all the terms (including technical and scientific terms) used in the specification has the same meaning as those that are understood by those who skilled in the art. Further, the terms defined by the dictionary generally used should not be ideally or excessively formally defined unless clearly defined specifically. It will be understood that for purposes of this disclosure, “at least one of X, Y, and Z” can be construed as X only, Y only, Z only, or any combination of two or more items X, Y, and Z (e.g., XYZ, XYY, YZ, ZZ). Unless particularly described to the contrary, the term “comprise”, “configure”, “have”, or the like, which are described herein, will be understood to imply the inclusion of the stated components, and therefore should be construed as including other components, and not the exclusion of any other elements. 
     Hereinafter, exemplary embodiments of the present invention will be described in more detail with reference to the accompanying drawings. 
     inventive concepts In the drawings, the same reference numerals will be used throughout to designate the same or equivalent elements. In addition, a detailed description of well-known features or functions will be ruled out in order not to unnecessarily obscure the gist of the inventive concepts. 
     In describing elements of exemplary embodiments of the inventive concepts, the terms first, second, A, B, (a), (b), and the like may be used herein. These terms are used to distinguish one element from another element, but do not limit the corresponding elements irrespective of the nature, order, or priority of the corresponding elements. Furthermore, unless otherwise defined, all terms including technical and scientific terms used herein are to be interpreted as is customary in the art to which this invention belongs. It will be understood that terms used herein should be interpreted as having a meaning that is consistent with their meaning in the context of the inventive concepts and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein. 
     Hereinafter, various embodiments of the inventive concepts will be described in detail with reference to  FIGS. 1 to 4 . 
       FIG. 1  is a block diagram illustrating a road slope compensation system using camera information according to an embodiment of the inventive concepts.  FIG. 2  is a view illustrating generation of lateral acceleration in a vehicle, to which a road slope compensation system using camera information is applied, due to a road slope on a road according to an embodiment of the inventive concepts.  FIG. 3  is a graph illustrating a progress status of a vehicle before a road slope compensation system using camera information is applied, according to an embodiment of the inventive concepts.  FIG. 4  is a graph illustrating a progress status of a vehicle after a road slope compensation system using camera information is applied, according to an embodiment of the inventive concepts. 
     Referring to  FIG. 1 , a road slope compensation system using camera information according to an embodiment of the inventive concepts may include ADAS including Autonomous Emergency Braking (AEB), Lane Keep Assist System (LKAS), Advanced Smart Cruise Control (ASCC), Active Blind Spot Detection (ABSD), Around View Monitor (AVM), and the like; the road slope compensation system may include a camera module  100 , a vehicle sensor device  300 , a correction controller  500 , a compensation calculation device  700 , and a steering device  900 . 
     The camera module  100  may capture and obtain a front image of a vehicle. The vehicle sensor device  300  may be configured to sense information about a vehicle equipped with ADAS and information about an external object, and may include radar, LiDAR, a speed sensor, a heading sensor, a torque sensor, a lateral acceleration sensor, a yawrate sensor, and the like. 
     The steering device  900  may include a steering angle sensor that measures the steering angle of a steering wheel. The steering device  900  may receive the steering angle calculated by the correction controller  500 , and may steer a wheel by adjusting the steering angle of the steering wheel depending on the calculated steering angle. 
     The steering device  900  may be implemented with Motor Driven Power Steering (MDPS). 
     The correction controller  500  may be included in an Electronic Control Unit (ECU). The correction controller  500  may calculate the difference between a target yawrate and the yawrate measured by a yawrate sensor, and may modify the target yawrate using the calculated difference when a vehicle is driving on a road having a road slope. The steering of the steering wheel may be controlled to implement the modified target yawrate. 
     That is, the correction controller  500  may determine whether a vehicle is driving on a road having a threshold road slope, using a line curvature calculated based on the line information captured by the camera module  100  and a vehicle curvature sensed by the vehicle sensor device  300 . Next, when the absolute value of the difference (|vehicle curvature−line curvature|) between the vehicle curvature and the line curvature exceeds a threshold, the correction controller  500  may determine that there is a possibility of a road having a threshold road slope. Also, when the determination is repeated more than the specific number of times, the correction controller  500  may determine that the vehicle is driving on the road having a threshold road slope. 
     Then, the correction controller  500  may control the operation of the steering device  900  for road slope compensation when the vehicle is driving on the road having the threshold road slope. 
     The correction controller  500  measures the current yawrate through a yawrate sensor, and calculates the difference between the measured yawrate and the target yawrate as a compensation value (measured yawrate−target yawrate). In addition, the steering device  900  may be controlled to modify the target yawrate, using the difference between the current yawrate and the target yawrate. 
     That is, the modified target yawrate may be calculated by Equation 1. 
       Modified Target Yawrate=Target Yawrate+(Compensation Value×Gain)  [Equation 1]
 
     Afterward, to implement the modified target yawrate, the steering device  900  may control the steering angle of the steering wheel. 
     However, referring to  FIG. 2 , in the case of a road having the threshold road slope, the vehicle may receive additional lateral acceleration force. 
     For example, when the lateral acceleration is estimated for a road slope of 1.5%, about 0.3767 deg/s yawrate may occur for normal straight driving in the vehicle at 80 KPH (roughly 50 MPH). 
     That is, additional lateral acceleration due to road slope occurs as compared to a horizontal road, and therefore embodiments described herein having a threshold road slope compensate for the existing target yawrate by estimating the lateral acceleration (angular velocity) occurring at a measured road slope within the threshold slope. Embodiments estimate and compensate for the target yawrate using compensation and using the actual heading (Reference) by the camera module  100 , by calculating vehicle-based heading through the yawrate estimation of the vehicle in a state where the vehicle is moving. 
     When a vehicle is driving on the road in the threshold road slope, the compensation calculation device  700  may calculate the compensation yawrate, using the sensor signal delivered from the vehicle sensor device  300  and the heading error by the front image delivered from the camera module  100 . 
     For example, the compensation calculation device  700  may calculate the estimated heading using the line curvature yaw rate and the current heading of the vehicle, may calculate a heading error, using the estimated heading and the current heading, and may calculate a compensation yawrate, using the heading error. 
     The curvature yawrate may be calculated by Equation 2. 
       Curvature Yawrate=Measured Yawrate−(Curvature* Vs )  [Equation 2]
 
     Here, the measured yawrate may be the measured value by a yawrate sensor. The curvature may be a curvature radius of the road having a threshold road slope; and Vs may be the speed of a vehicle. 
     Because the road curvature of 3000˜5000 R obtained from a straight road has a significant effect on the estimation of the heading, the yawrate may be calculated in consideration of the curvature. 
     In addition, when a cut-in of the preceding vehicle occurs, the curvature may be invalid because the view-range is small, and thus whether to reflect the curvature yawrate may be determined by using the gradient direction and the sign of a road curvature. 
     The estimated heading may be calculated by Equation 3. 
       Estimated Heading=∫ t0   t1  Curvature Yawrate  dt +Current Heading  [Equation 3]
 
     Here, t 0  is a point in time when line control is activated; t 1  is a point in time when a reset condition is determined. 
     For reference, the reset condition may be a case that line control activation is started, a case of a curved road with a curvature radius of 5000R or less, a case that a vehicle is located in the middle of the lane, a case that the past direction of the difference in yawrate tracking performance for a set time period such as four (4) seconds is different from the direction of the heading estimation error, or a case that it is time to switch to a driver&#39;s hands-on state. 
     The heading error may be calculated by Equation 4. 
       Heading Error=Estimated Heading−Current Heading  [Equation 4]
 
     In the meantime, the moving average of, for example, 5 seconds is applied to the heading error signal, which is configured to reflect the compensation yaw rate. In this way, a small delay may occur, but the heading error signal is applied to compensate for a constant average value. Also, the effect on an error from estimating the heading error due to the external disturbance may be reduced by the average. 
     The compensation yawrate is calculated by Equation 5. 
       Compensation Yawrate=Heading Error*Compensation Gain  [Equation 5]
 
     Furthermore, when line control is activated, the operating condition of the compensation yawrate is a case where a vehicle speed is not less than 40 KPH or a case that the yawrate compensation is in the direction for assistance toward the center of the line; when the line control activation is terminated, the release condition of the compensation yawrate is a case where a vehicle speed is not greater than 35 KPH or a case that the yawrate compensation is not in the direction for assistance toward the center of the line. 
     As described above, when the compensation yawrate is calculated, the compensation yawrate is transmitted to the correction controller  500 . 
     The correction controller  500  may calculate the final yawrate by applying the compensation yawrate delivered from the compensation calculation device  700  to the target yawrate calculated using the line curvature yawrate and the current heading. 
     The final yawrate may be calculated by Equation 6. 
       Final Yawrate=Target Yawrate+Compensation Yawrate  [Equation 6]
 
     In the meantime, a performance difference between the vehicle&#39;s left tracking and right tracking may occur due to the force generated by the threshold road slope. That is, the generation amount of left yawrate is different from the generation amount of right yawrate, and thus the desired torque may be different. 
     Accordingly, to compensate for the performance difference between the vehicle&#39;s left tracking and right tracking, which is caused by the threshold road slope, the correction controller  500  may control the steering device  900  after calculating the final steering torque using the compensation yawrate. 
     The final steering torque may be calculated by Equation 7. 
       Final Steering Torque=Existing Torque+compensation Torque  [Equation 7]
 
     At this time, the compensation torque may be calculated by Equation 8. 
       Compensation Torque=Gain*Compensation Yawrate  [Equation 8]
 
     Referring to  FIGS. 3A to 3D , in the case of not applying the compensation yawrate according to an embodiment of the inventive concepts,  FIG. 3A  illustrates a current yawrate  11  and a target yawrate  13  of a vehicle;  FIG. 3B  illustrates a desired torque  21  calculated in real time for the current yawrate  11  to follow the target yawrate  13 ; the steering device  900  may be controlled such that the desired torque  21  is applied to the steering. 
       FIG. 3C  illustrates a camera heading signal  31  recognized by the camera module  100  and a predicted heading estimation signal  33 .  FIG. 3D  illustrates the location of a vehicle in a line, and illustrates a line  41 , a left wheel  43  of the vehicle, and a right wheel  45  of the vehicle. 
     Referring to  FIGS. 4A to 4D , in the case of applying the compensation yawrate according to an embodiment of the inventive concepts,  FIG. 4A  illustrates a finally-compensated target yawrate  15  by adding the current yawrate  11 , the target yawrate  13 , and the compensation yawrate of a vehicle. This indicates that the vehicle may be further moved to the right. 
       FIG. 4B  illustrates the desired torque  21  calculated in real time such that the current yawrate  11  better follows the compensated target yawrate  15 . 
       FIG. 4C  illustrates the camera heading signal  31  recognized by the camera module  100  and the predicted heading estimation signal  33 .  FIG. 4D  illustrates the location of a vehicle in a line, and illustrates the line  41 , the left wheel  43  of the vehicle, and the right wheel  45  of the vehicle. 
     As a result, when the compensation yawrate according to an embodiment of the inventive concepts is applied, the vehicle close to the left side on the road slope may drive while further moving to the right side to drive in the middle of the road. 
     Hereinafter, a road slope compensation method using camera information according to another embodiment of the inventive concepts will be described in detail with reference to  FIG. 5 .  FIG. 5  is a flowchart illustrating a road slope compensation method using camera information according to an embodiment of the inventive concepts. 
     Hereinafter, it is assumed that a road slope compensation method using camera information of  FIG. 1  performs the process of  FIG. 5 . 
     For example, the method determines the yawrate error estimation performance by applying an activation of a vehicle&#39;s line control, a straight road with a curvature radius of 5000R or more, within the target yawrate of ±1 deg/s, a driver&#39;s hands-off state, or the moving average of the yawrate Error of tracking performance for 4 seconds (S 110 ). 
     Next, the compensation calculation device  700  calculates the estimated heading, using the sensor signal delivered from the vehicle sensor device  300  sensing and obtaining state information of a vehicle, the line curvature yawrate by the front image delivered from the camera module  100  capturing and obtaining the front image of the vehicle, and the current heading when the vehicle is driving on a road having a threshold road slope. 
     Then, the method calculates a heading error, using the estimated heading and the current heading (S 120 ). 
     Next, the method calculates a compensation yawrate, using the heading error (S 130 ). 
     Subsequently, in the correction controller  500 , the method calculates the final yawrate by applying the compensation yawrate delivered from the compensation calculation device  700  to the target yawrate calculated using the sensor signal, the line curvature yawrate by the front image, and the current heading (S 140 ). 
     Subsequently, the method may calculate the final steering torque by the final yaw rate (S 150 ), and then may control the steering device (S 160 ). 
     As described above, a road slope compensation system using camera information, and a method thereof determines a threshold road slope based on camera information and lateral acceleration of a vehicle, estimates the road slope, compensates the estimated road slope to an ADAS driving convenience system, and thus prevents the vehicle from being inclined to the road slope in a section where a threshold road slope is present, thereby securing the driving stability of the vehicle by driving the vehicle in the middle of the lane on a road having the threshold road slope. 
     In the meantime, according to an embodiment of the inventive concepts, a road slope compensation method using camera information according to operations S 110  to S 160  may be programmed and stored in a computer-readable medium. 
     Hereinabove, although the inventive concepts has been described with reference to exemplary embodiments and the accompanying drawings, the inventive concepts is not limited thereto, but may be variously modified and altered by those skilled in the art to which the inventive concepts pertains without departing from the spirit and scope of the inventive concepts claimed in the following claims. 
     Therefore, embodiments of the inventive concepts are not intended to limit the technical spirit of the inventive concepts, but provided only for the illustrative purpose. The scope of protection of the inventive concepts should be construed by the attached claims, and all equivalents thereof should be construed as being included within the scope of the inventive concepts. 
     Embodiments described herein determine a threshold road slope based on camera information and lateral acceleration of a vehicle, estimate the road slope, compensate the estimated road slope to an ADAS driving convenience system, and thus prevent the vehicle from being inclined to the road slope in a section where a threshold road slope is present, thereby securing the driving stability of the vehicle by driving the vehicle in the middle of the lane on a road having the threshold road slope. 
     Besides, a variety of effects directly or indirectly understood through the specification may be provided. 
     Hereinabove, although the inventive concepts has been described with reference to exemplary embodiments and the accompanying drawings, the inventive concepts is not limited thereto, but may be variously modified and altered by those skilled in the art to which the inventive concepts pertains without departing from the spirit and scope of the inventive concepts claimed in the following claims.