Patent ID: 12188957

DETAILED DESCRIPTION

In the following description, numerous specific details are set forth to provide a more thorough understanding of the various embodiments. However, it will be apparent to one of skilled in the art that the inventive concepts may be practiced without one or more of these specific details.

System Overview

FIG.1is a conceptual block diagram of an optical sensor compensation system100configured to implement one or more aspects of the various embodiments. As shown, the optical sensor compensation system100includes a computing device110, sensor(s)120, actuator130, and optical sensor140. Computing device110includes one or more processing unit(s)112, and memory114. Memory114stores a compensation application116and an actuator controller118. In some embodiments, sensor(s)120, actuator130, optical sensor140, and computing device110, are implemented at a vehicle.

Sensor(s)120can include any type of device that is capable of receiving and/or transmitting sensor data, including for example accelerometer data, angular velocity data, and so forth. More generally, the sensor data can include angular data of the vehicle, including but not limited to angular velocity, angular rotation or displacement relative to one or more axes and/or planes, and/or the like. In some embodiments, sensor(s)120include one or more sensors that provide other data, such as location data, image data, temperature data, etc. Computing device110can compute an amount of angular rotation or displacement corresponding to a roll of a vehicle on which sensor(s)120are mounted based on the acquired sensor data. Non-limiting examples of sensor(s)120include accelerometers, gyroscopes, magnetometers, computing devices, smartphones, navigation devices, imaging devices, Internet of Things (IoT) devices, radiofrequency identification (RFID) devices, traffic devices, global positioning devices, etc. In various embodiments, sensor(s)120can communicate with computing device110via a wired or wireless connection. In some embodiments, sensors120include an inertial measurement unit (IMU) that includes one or more sensors such as one or more accelerometer(s), one or more gyroscope(s), and/or one or more magnetometer(s).

As noted above, computing device110includes processing unit(s)112and memory114. Computing device110can be a device that includes one or more processing units112, such as a system-on-a-chip (SoC). In some embodiments, computing device110can be a head unit or other component included in a vehicle system. Generally, computing device110can be configured to coordinate the overall operation of optical sensor compensation system100. The embodiments disclosed herein contemplate any technically-feasible system configured to implement the functionality of optical sensor compensation system100via computing device110.

In various embodiments, computing device110can be located in various environments including, without limitation, road and/or land vehicle environments (e.g., consumer vehicle, commercial vehicle, bicycle, motorcycle, wheeled drone, etc.), aerospace and/or aeronautical vehicle environments (e.g., airplane, helicopter, spaceship, glider, aerial drone, etc.), nautical and submarine vehicle environments (e.g., boat, yacht, submarine, personal watercraft, nautical or submarine drone, etc.), and so forth.

Processing unit(s)112can be any technically-feasible form of processing device configured to process data and execute program code. Processing unit(s)112could include, for example, and without limitation, a system-on-chip (SoC), a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a field-programmable gate array (FPGA), and so forth. Processing unit(s)112includes one or more processing cores. In operation, processing unit112can be a primary processor of computing device110, controlling and coordinating operations of other system components.

Memory114can include a memory module or a collection of memory modules. In various embodiments, processing unit(s)112can execute compensation application116and/or actuator controller118to implement the overall functionality of the computing device110and, thus, to coordinate the operation of the optical sensor compensation system100as a whole. For example, and without limitation, sensor data acquired via the sensors120can be processed by compensation application116to generate a command. Processing unit(s)112can execute actuator controller118to drive actuator130based on the command generated by compensation application116. In various embodiments, compensation application116can control the timing of sending the command to actuator controller118. For example, compensation application116can send a command that includes an amount of angular rotation of a vehicle about a longitudinal axis of the vehicle to actuator controller118, driving actuator130to orient optical sensor140by an amount that counters the amount of roll by the vehicle about the longitudinal axis. In some examples, compensation application116can send the command within 10 μs of receiving the roll data from sensor(s)120. In some embodiments, compensation application116and actuator controller118are stored in a storage medium (not shown) of computing device110and loaded into memory114for execution. In some embodiments, compensation application116and actuator controller118can be combined into one application. Operation of compensation application116and actuator controller118is further described below.

Actuator130controls the orientation of optical sensor140. In various embodiments, actuator130controls the orientation of optical sensor140along at least one axis (e.g., a longitudinal axis that is parallel to the longitudinal axis of the vehicle). In some embodiments, actuator130and actuator controller118can be included in a separate optical sensor orientation system (not shown). In such instances, actuator controller118can receive a command from computing device110and drive actuator130to rotate optical sensor140according to an angle (e.g., an amount of angular rotation of the vehicle about the longitudinal axis of the vehicle) included in the command. In some embodiments, actuator130controls one or more stabilizer components (not shown) that modify the orientation of optical sensor140relative to the road surface. Actuator130includes various mechanical, electro-mechanical, and/or other components (e.g., gears, actuators, hydraulic systems, pneumatic systems, etc.) that physically change the orientation of optical sensor140.

Optical sensor(s)140includes at least one sensor that acquires optical data. In various embodiments, optical sensor140acquires optical data relating to the environment of a vehicle. In various embodiments, optical sensor140can include one or more cameras, such as RGB cameras, infrared cameras, thermal cameras, night vision cameras, depth cameras, and/or camera arrays, which include two or more of such cameras. Other optical sensors can include imagers and laser sensors. Further, optical sensor140can include one or more components of an imaging system, such as one or more components of a RADAR, a LiDAR system, and/or the like. In various embodiments, a vehicle can include one or more optical sensors140. For example, a vehicle could include a forward-facing optical sensor (for capturing the environment in front of the vehicle) and/or a rear-facing optical sensor (for capturing the environment to the rear of the vehicle).

In various embodiments, optical sensor140is coupled to actuator130. For example, optical sensor140could be mounted on a frame or a shaft that is coupled to and rotatable by actuator130. In some embodiments, the vehicle could include multiple actuators130(e.g., an actuator130for each optical sensor140).

In various embodiments, optical sensor140can send optical data to computing device110and/or another processing system. In such instances, computing device110and/or the other processing system analyzes the image data. For example, compensation system100could include an image processing system150that processes images to identify entities (e.g., objects, people, animals). In such instances, optical sensor140could send optical data in the form of a captured image to image processing system150, where image processing system150analyzes the captured image using any technical feasible technique(s) (e.g., object recognition, computer optical vision, machine learning, etc.) in order to identify entities in the image. Image processing system150could send image analysis results (e.g., the objects recognized in the image) to a vehicle control assistance system160. Image processing system150can be implemented via hardware, software, and/or a combination of hardware and software.

In some embodiments, compensation system100further includes a vehicle control assistance system160. Vehicle control assistance system160can include one or more applications or sub-systems that provides assistance or take-over (e.g., information to a human driver, automated or autonomous vehicle operation) regarding operation of the vehicle, such as a driver assistance system, an autonomous driving system, an automated cruise control system, a collision avoidance system, and/or the like. Vehicle control assistance system160Vehicle control assistance system160can use the image analysis results to provide assistance or take-over based on the results (e.g., an alert to a human driver, autonomously brake and/or steer the vehicle). Vehicle control assistance system160can be implemented via hardware, software, and/or a combination of hardware and software. Image processing system150and/or vehicle control assistance system160can be an application or a sub-system within computing device110or implemented elsewhere in the vehicle (e.g., another application or sub-system in the vehicle head unit or in-vehicle computing system). In some embodiments, image processing system150and vehicle control assistance system160can be combined.

A vehicle can include a driver assistance system that helps a user operate the vehicle safely and assists in certain tasks associated with the vehicle, enabling the user to enjoy the ride more. Additionally or alternatively, a vehicle can include an autonomous driving system that operates the vehicle with little or no driver intervention. Computer optical vision systems have been key components in driver assistance systems and other vehicle systems, such as autonomous systems for four-wheel cars and trucks. When included in a driver assistance system, the computer optical vision system helps the user by providing feedback. Such feedback can include information about how quickly to enter a corner, or an alert to notify a driver who is not paying close attention to the road. In autonomous systems, the computer optical vision system enables the autonomous driving system to safely navigate an environment based on data acquired about the environment.

A computer optical vision system includes one or more optical sensors (e.g., still cameras, video cameras, LiDAR systems, etc.) that captures images of the surrounding environment. The computer optical vision system includes an image processing system that receives the one or more images captured by the optical sensors as an input and processes the image in order to operate the vehicle. For example, an autonomous vehicle could employ a computer optical vision system to accurately detect entities (e.g., vehicles, street buildings, pedestrians, road signs, etc.) in order to assist the autonomous vehicle to drive as safely as a human. Similarly, a driver assistance system could detect entities and notify a human driver of potential hazards.

Techniques for Roll Effect Correction

FIG.2are diagrams illustrating principal axes for a four-wheel vehicle and a motorcycle, according to various embodiments. As shown inFIG.2, a four-wheel vehicle200has a lateral axis, a longitudinal axis, and a vertical axis. Four-wheel vehicle200rarely rolls or rarely rolls appreciably around the longitudinal axis during operation. As used herein, a roll of a vehicle is a rotation of the vehicle around the longitudinal axis, thereby causing an amount of angular rotation relative and orthogonal to a vertical axis of the vehicle, where that vertical axis is perpendicular to a horizontal plane and/or a plane of the road surface on which the vehicle is travelling. Instead, vehicle200typically rotates (i.e., is steered) around the vertical axis (e.g., the z-axis or heave axis). Vehicle200can also experience pitch around the lateral axis relative to a horizontal plane (e.g., when travelling uphill or downhill) and/or relative to the plane of the road surface. The vehicle can provide yaw control by controlling the motion around the vertical axis. Thus, optical sensors mounted to the front and/or rear of a four-wheel vehicle (e.g., where the field of view of the optical sensor is directed toward the front or rear of the vehicle) with a view-up orientation that is upright and parallel to the vertical axis of the vehicle have a view-up orientation that remains substantially perpendicular to the horizontal plane and/or the road surface. Image processing system150generally accurately detects entities in such images.

FIG.3illustrates an example image of a road environment captured without a roll effect, according to various embodiments. As shown, image300is an image captured by a forward-facing optical sensor mounted on a vehicle. Image300is captured while the vehicle has no appreciable roll, and accordingly the view-up orientation of the optical sensor at the moment of capture, and correspondingly the view-up orientation of image300, is substantially perpendicular to the road surface. Accordingly, image300is substantially free of a roll effect.

Image300has a predefined aspect ratio302. Image300includes a road304in front of the vehicle. A vehicle310is on road304in front of the vehicle. Another vehicle306is on the opposite road, across a median316. A tree308is located on the median316. A traffic light312is located on the side of road304. Image processing system150processing image can analyze image300and accurately recognize and detect vehicles306and310, tree308, and traffic light312, as indicated by recognition bounding boxes314.

Computer optical vision systems can also be implemented in other vehicles that provide controls along one or two other axes. For example, returning toFIG.2, a motorcycle210also has a lateral axis, a longitudinal axis, and a vertical axis. Vehicles such as motorcycle210, submarines, aerial vehicles (including quadcopters), boats, and so forth, can roll around the longitudinal axis. Vehicle210can provide roll control to manage the amount of roll. Thus, the view-up orientations of optical sensors mounted to the front and/or rear of vehicle210, mounted with a view-up orientation that is upright and parallel to the vertical axis of vehicle210, are not perpendicular to the road surface while vehicle210is rolled about the longitudinal axis of vehicle210.

Because a typical image processing system is trained using images where the view-up orientation of the optical sensor is substantially perpendicular to the road surface and/or the horizontal plane, the image processing system detects entities less accurately in images where the view-up orientation of the optical sensor is not perpendicular to the road surface and/or the horizontal plane. While such images can be corrected post-capture (e.g., by rotating the images and then fitting the images into the predefined aspect ratio required by the image processing system), such correction has the drawback of causing portions of the original image to be cropped off, resulting in less information for the image processing system.

FIG.4illustrates an example image of the road environment captured with a roll effect, according to various embodiments. As shown, image400is an image of the same road environment as in image300, captured by the same forward-facing optical sensor as with image300. Image400is captured while the vehicle is rolled counter-clockwise about the longitudinal axis the vehicle relative to the plane of the surface of road304, and accordingly the view-up orientation of the optical sensor at the moment of capture, and correspondingly the view-up orientation of image400, is not perpendicular to the road surface but is angled. Accordingly, image400has a roll effect.

Image400has a predefined aspect ratio402that is the same as aspect ratio302. Shown in image300is road304in front of the vehicle. Image400includes vehicle310on road304in front of the vehicle. Image400also includes a portion of vehicle306on the opposite road across a median416, and a portion of tree308on median416. A traffic light312is located on the side of road304.

FIG.5illustrates the example image ofFIG.4with roll effect correction according to conventional techniques.FIG.5illustrates an image500that can be generated by rotating image400opposite the direction of the roll of the vehicle and then fitting the rotated image into aspect ratio502, which is the same aspect ratio as aspect ratio302/402. Thus, the view-up orientation of image500is substantially perpendicular to the surface of road304. Because the rotated image is fitted into aspect ratio502, portions506of the rotated image400that are outside of aspect ratio502are cropped off and not included in image500. Instead, image500includes portions504that are blank (e.g., fully white or black) and correspond to portions of the aspect ratio502that are not covered by any part of the rotated image400fitted into aspect ratio502and are instead filled with blank content (e.g., a single color). Accordingly, image500contains less information than image300or400. An image processing system can process image500and, because of the reduced amount of information in the image, detect entities less accurately and/or detect fewer entities in the image. For example, as shown inFIG.5, image processing system150would detect just vehicle310and traffic light312in image500, indicated by bounding boxes314. Meanwhile, significant portions of vehicle306and tree308are not captured in image500and are accordingly not detected by image processing system150.

To address the above drawbacks, optical sensor compensation system100performs roll effect correction prior to capture of an image. Compensation application116acquires sensor data from sensors120, determines an amount of roll of the vehicle (e.g., an amount of angular rotation around the longitudinal axis of the vehicle), and generates an actuator command based on the amount of roll. Compensation application116sends the actuator command to actuator controller118, which drives actuator130based on the actuator command to modify the orientation of optical sensor140to counteract the amount of roll. Optical sensor140, with the modified orientation, can capture one or more images, which can be processed by image processing system150.

In operation, compensation application116acquires sensor data from sensors120. Compensation application116receives sensor data that can be used to detect an amount of roll of the vehicle about the longitudinal axis of the vehicle from sensors120(e.g., from an IMU and/or an accelerometer). The sensor data can include a roll angle parameter that directly measures an amount of roll (e.g., the amount of angular rotation relative to the axis perpendicular to the road surface) and a direction of roll (e.g., rolling clockwise or counterclockwise about the longitudinal axis of the vehicle relative to the vertical axis). Additionally or alternatively, the sensor data can include one or more parameters (e.g., angle relative to a horizontal plane, angular velocity, etc.) that can indirectly indicate rolling and can be used to calculate the amount and direction of roll.

Compensation application116can determine an amount and direction of roll based on the sensor data and accordingly determine or detect whether the vehicle is rolling. For example, if the sensor data includes a roll angle parameter, compensation application116can detect whether the vehicle is rolling from the roll angle parameter. Additionally or alternatively, compensation application116can calculate the amount and direction of roll from the sensor data and detect whether the vehicle is rolling from the calculated amount and direction of roll. In some embodiments, compensation application116detects rolling that exceeds a threshold and disregards rolling that does not exceed the threshold. For example, compensation application116could disregard amounts of roll that are smaller than a threshold (e.g., angular rotation of less than 5 degrees relative to the vertical axis).

In some embodiments, compensation application116filters the sensor data as part of the determination of the roll angle and direction. In some other embodiments, sensors120(e.g., accelerometer, IMU) can filter the sensor data before sending the sensor data (e.g., the roll amount and direction) to compensation application116. The filtering can be performed with a low-pass filter, for example. In these embodiments, filtering the sensor data can remove noise amongst the sensor data and provide a more-accurate determination of the amount of roll. For example, sensors120could generate noise within the sensor data due to vibrations and jerks felt by the vehicle (e.g., driving over an uneven road surface).

To remove noise, sensors120and/or compensation application116can compute a simple moving average for incoming sensor data. For example, when sensors120filters incoming sensor data, with each iteration in a code loop, sensors120could drop the oldest value from the previous n datapoints and replaces that value with the latest reading. Sensors120can use Equation 1 to compute an average for the amount of roll (e.g., angular rotation of the roll).

dsm=1n⁢∑i=0n-1dm-i(1)

Where dsmis the simple moving average at current data point dm, and n is the number of previous data points taken into consideration. In one example, sensors120or compensation application116can compute the angular rotation of the roll as an average of the previous twenty data points acquired by the accelerometer. In some embodiments, sensors120and/or compensation application116can perform exponential smoothing on the sensor data in lieu of calculating a simple moving average. More generally, sensors120and/or compensation application116can aggregate, using any suitable algorithm or technique, any number of roll angle values in the sensor data before optical sensor140captures an image to determine an amount and direction of roll, so that the orientation of optical sensor140can be modified before the image is captured.

Using the new reading, sensors120can compute a new aggregate value. This new aggregate value is sent to compensation application116, which uses the new aggregate value to generate an actuator command for actuator controller118. Using aggregated values to drive the motion controller can ensure smooth operation of the motor.

In some embodiments, compensation application116acquires sensor data at a more frequent rate than the frame rate of optical sensor140. For example, the frame rate of optical sensor140could be 30 frames per second, and sensors120could provide sensor data to compensation application116at a rate of 300 data points per second. When determining an amount and direction of roll before capture of a next frame by optical sensor140, compensation application116can use the data points of sensor data acquired between that next frame and the immediately preceding frame of optical sensor140. Those data points can be optionally processed (e.g., aggregated, average, moving average, exponentially smoothed) to determine the amount and direction of roll.

After determining the amount and direction of roll, compensation application116can generate an actuator command. The actuator command can include an angle by which to rotate optical sensor140and a direction of rotation. The angle and direction included in the actuator command are generated to counter the amount and direction of roll of the vehicle. For example, if the vehicle is rolling counterclockwise 30 degrees, the actuator command would include an angle of 30 degrees clockwise by which to rotate a forward-facing optical sensor140. Compensation application116sends the actuator command to actuator controller118. In some embodiments, if compensation application116detects that the vehicle is not rolling (e.g., the amount of roll is below a threshold or is otherwise zero or minimal), compensation application116can omit generating an actuator command.

Actuator controller118receives the actuator command and drives actuator130based on the actuator command. Actuator controller118activates actuator130to rotate optical sensor140by the angle and direction included in the actuator command, thereby re-orienting optical sensor140so that the view-up orientation of optical sensor140is substantially perpendicular to the road surface and/or the horizontal plane.

In some embodiments, compensation application116and actuator controller118can perform the roll effect correction process for multiple optical sensors. For example, if the vehicle includes a forward-facing optical sensor and a rear-facing optical sensor, compensation application116could determine the amount and direction of roll of the vehicle, and then determine respective actuator commands for the forward-facing optical sensor and the rear-facing optical sensor. Actuator controller118could drive respective actuators130based on the respective actuator commands. Additionally or alternatively, compensation application116can generate a command, and the actuator controller118drives the respective actuators130based on the command, with actuator controller118being responsible for interpreting the command into the proper amounts and directions of rotation based on whether the particular actuator is rotating a forward-facing optical sensor or a rear-facing optical sensor.

FIG.6illustrates a conceptual diagram of roll effect correction by modifying an orientation of an optical sensor, according to various embodiments.FIG.6illustrates a front view of a motorcycle602that can roll about its longitudinal axis. Vehicle604has a forward-facing optical sensor140(e.g., a camera604) that faces out toward the front environment of motorcycle602with a field of view aligned parallel to the longitudinal axis of motorcycle602. When motorcycle602rolls, motorcycle602rolls by an angular rotation amount ⊖x608and in a direction (e.g., clockwise from motorcycle602facing forward as shown) relative to a vertical axis606that is perpendicular to road surface612. Compensation application116can determine the angular rotation amount ⊖x608and the direction of roll, and generate a corresponding actuator command. Actuator controller118can drive an actuator130on-board motorcycle602to rotate610camera604by angular rotation amount ⊖xand in a direction (e.g., counterclockwise from camera604facing forward as shown) that counters the direction of the roll of motorcycle602, thereby maintaining the view-up orientation of camera604to be perpendicular to road surface612.

It should be appreciated that the angular rotation amount of the roll can be expressed as an angle relative to the vertical axis or as an angle relative to a horizontal plane or a plane of the road surface. In various embodiments, sensors120and/or compensation application116can determine an angle relative to the vertical axis from an angle relative to a horizontal plane or a plane of the road surface, or vice versa.

FIG.7illustrates a flow diagram of method steps to compensate for the roll effect for an optical sensor, according to various embodiments. Although the method steps are described with respect to the systems ofFIGS.1-6, persons skilled in the art will understand that any system configured to perform the method steps, in any order, falls within the scope of the various embodiments.

As shown, a method700begins at step702, where compensation application116acquires sensor data from one or more sensors120(e.g., IMU, accelerometer) on a vehicle.

At step704, compensation application116determines an amount and direction of roll of the vehicle. Compensation application116can read the data points in the sensor data and determine an amount of roll and a direction of roll directly from the sensor data (e.g., from a roll angle parameter in the sensor data). Additionally or alternatively, compensation application116can calculate an amount and direction of roll from the sensor data. Further additionally or alternatively, compensation application116can determine an amount and direction of roll from multiple data points in the sensor data. In some embodiments, sensors120and/or compensation application116can aggregate and/or filter data points in the sensor data before determining the amount and direction of roll. In some embodiments, compensation application116can determine if the amount of roll is less than a threshold amount (e.g., is zero, is less than a threshold amount of roll). If compensation application116determines that the amount of roll is less than the threshold amount, then compensation application116can omit steps706and708below.

At step706, compensation application116generates an actuator command based on the amount and direction of roll. The actuator command includes an angle and a direction that is intended to counter the amount and direction of roll of the vehicle determined in step704. For example, the actuator command can include an amount of rotation equal to the amount of roll of the vehicle, but in a direction opposite to the direction of roll of the vehicle. If compensation application116determines that the amount of roll is less than the threshold amount, then compensation application116can, instead of omitting steps706and708as described above, generate a null or no-op command.

At step708, compensation application116causes an actuator to modify an orientation of an optical sensor on the vehicle based on the actuator command. Compensation application116sends the actuator command generated in step706to an actuator controller118. Actuator controller118drives an actuator130based on the actuator command to modify an orientation of an optical sensor140(e.g., rotating optical sensor140to maintain a view-up orientation perpendicular to the road surface). If the command is a null or no-op command, actuator controller118can receive the null or no-op command and take no action in response.

At step710, computing device110captures an image using the optical sensor. Computing device110and/or image processing system150can capture an image using optical sensor140whose orientation is modified in step708. At step712, image processing system150processes the captured image to recognize one or more entities in the image. Image processing system150in computing device110or elsewhere in the vehicle can process the image captured in step710to recognize one or more objects, etc. in the image. Image processing system150can send the processing results to a vehicle control assistance system160.

In sum, the disclosed techniques compensate for the amount and direction of roll in a structure (e.g., a vehicle) when capturing optical data via a forward-facing or rear-facing optical sensor mounted on the vehicle. The techniques include acquiring sensor data from sensors on the vehicle. A compensation application processes the sensor data to determine an amount and direction of roll of the vehicle and to generate an actuator command based on the determined amount and direction of roll. An actuator controller drives an actuator based on the actuator command to modify the orientation of a forward-facing or rear-facing optical sensor mounted on the vehicle and coupled to the actuator, thereby re-orienting the view-up orientation of the optical sensor to be substantially perpendicular to the road surface while the vehicle is undergoing roll. The optical sensor whose orientation is modified by the disclosed techniques can then capture optical data, and those optical data, which exhibit reduced or eliminated roll effect due to the modified orientation of the optical sensor, can be provided to an image processor for processing.

At least one technical advantage of the disclosed techniques relative to the prior art is that the effect of vehicle roll on forward-facing and/or rear-facing optical data can be compensated for without cropping off information from the optical data. Accordingly, more orientation-appropriate optical data information can be provided to image processing and computer vision systems, resulting in higher recognition accuracy by those image processing and computer vision systems and thereby resulting in more effective driver assistance, autonomous driving, and/or the like. These technical advantages provide one or more technological advancements over prior art approaches.

1. In some embodiments, a computer-implemented method for controlling an optical sensor mounted on a vehicle comprises receiving sensor data from at least one sensor associated with the vehicle; detecting an amount of roll of the vehicle based on the sensor data; generating a command based on the detected amount of roll; and controlling an orientation of the optical sensor based on the command.

2. The method of clause 1, wherein controlling the orientation of the optical sensor comprises driving an actuator coupled to the optical sensor.

3. The method of clauses 1 or 2, wherein the actuator rotates the optical sensor based on the command.

4. The method of any of clauses 1-3, wherein the sensor data comprises angular data of the vehicle.

5. The method of any of clauses 1-4, further comprising determining a direction of roll of the vehicle based on the sensor data.

6. The method of any of clauses 1-5, wherein the amount of roll comprises an amount of angular rotation of the vehicle about a longitudinal axis of the vehicle.

7. The method of any of clauses 1-6, wherein generating the command based on the detected amount of roll comprises determining whether the detected amount of roll exceeds a threshold.

8. The method of any of clauses 1-7, wherein detecting the amount of roll based on the sensor data comprises aggregating a plurality of data points included in the sensor data.

9. The method of any of clauses 1-8, wherein detecting the amount of roll based on the sensor data comprises filtering the sensor data.

10. The method of any of clauses 1-9, wherein the command comprises an amount of angular rotation that counters the detected amount of roll.

11. In some embodiments, one or more non-transitory computer-readable storage media include instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of receiving sensor data from at least one sensor associated with a vehicle; detecting an amount of roll based on the sensor data; generating a command based on the detected amount of roll; and controlling an orientation of an optical sensor mounted on the vehicle based on the command.

12. The non-transitory computer-readable storage media of clause 11, wherein controlling the orientation of the optical sensor comprises driving an actuator coupled to the optical sensor.

13. The non-transitory computer-readable storage media of clauses 11 or 12, wherein detecting the amount of roll based on the sensor data comprises detecting the amount of roll based on a plurality of data points in the sensor data between a next frame and an immediately preceding frame of the optical sensor.

14. The non-transitory computer-readable storage media of any of clauses 11-13, wherein the steps further comprise determining a direction of roll of the vehicle based on the sensor data.

15. The non-transitory computer-readable storage media of any of clauses 11-14, wherein the command comprises an amount of angular rotation that counters the detected amount of roll, and a direction that counters the direction of roll.

16. The one or more non-transitory computer-readable storage media of any of clauses 11-15, wherein the steps further comprise generating a second command based on the detected amount of roll; and controlling an orientation of a second optical sensor mounted on the vehicle based on the second command.

17. In some embodiments, a system comprises at least one sensor; an optical sensor; memory storing an application; and one or more processors that, when executing the application, is configured to receive sensor data from the at least one sensor; detect an amount of roll based on the sensor data; generate a command based on the detected amount of roll; and control an orientation of the optical sensor based on the command.

18. The system of clause 17, further comprising an actuator coupled to the optical sensor, and wherein the actuator is configured to rotate the optical sensor in response to the command.

19. The system of clauses 17 or 18, wherein the system is implemented at a vehicle, and the sensor data comprises angular data of the vehicle.

20. The system of any of clauses 17-19, wherein the at least one sensor comprises an inertial measurement unit.

Any and all combinations of any of the claim elements recited in any of the claims and/or any elements described in this application, in any fashion, fall within the contemplated scope of the present disclosure and protection.

The descriptions of the various embodiments have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.

Aspects of the present embodiments may be embodied as a system, method or computer program product. Accordingly, aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “module,” a “system,” or a “computer.” In addition, any hardware and/or software technique, process, function, component, engine, module, or system described in the present disclosure may be implemented as a circuit or set of circuits. Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

Aspects of the present disclosure are described above with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine. The instructions, when executed via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions/acts specified in the flowchart and/or block diagram block or blocks. Such processors may be, without limitation, general purpose processors, special-purpose processors, application-specific processors, or field-programmable gate arrays.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

While the preceding is directed to embodiments of the present disclosure, other and further embodiments of the disclosure may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.