Patent Description:
Algorithms designed to automatically film moving objects using cameras attached to robotic equipment struggle to achieve the same performance of human operators in dynamically changing scenes. The main visible problems are:.

Professional automatic cameras may have, in addition to actuators to pan and tilt the camera, also one or several actuators connected to control a setting of the camera optics, like for example the lens aperture (iris), the focal length (zoom), or the focus distance, or any other meaningful setting of the camera or of an objective connected to the camera. Such actuators are often denoted as "ring actuators", referring to the conventional form of such controls, and this practice will be followed in the following description and claims, but the invention may include variants in which a "ring actuator" acts on a control that is not ring-shaped.

Precise actuation of the ring controls is often a weak point of conventional automatic cameras. An important aspect of the present invention lies in the accuracy of these operations. <CIT> discloses an automatic camera head with robotic control of pan, tilt, roll, focus, iris signal, track and other degrees of freedom. <CIT> describes a remote-control pan head allowing easy switching between local and network control. <CIT> discloses a remote-control system for zoom, focus and iris, <CIT> discloses a method to determine the focal length of a moving camera.

According to the invention, these aims are achieved by the system herein described and claimed.

In particular, the invention arises from an analysis of the limitation of the known solutions that are identified in a lack of control of the camera movements by the automatic camera head of the prior art, both for what pan and tilt are concerned, but also for objective adjustments in focus, zoom, and iris.

As <FIG> illustrates, the invention relates to an automatic camera head for panning and tilting a video camera, comprising a base <NUM>, that can be mounted on a fixed support such as a tripod, on a movable platform, or on a vehicle, possibly an automatic robot. The pedestal height of the tripod may also be settable by a suitable automatic control.

The base <NUM> has attached an intermediate element <NUM>, also called "pan platform", that can be rotated about the pan axis <NUM>. The rotation is determined by the motor <NUM> and the rotation angle is precisely captured and read by an encoder.

According to an important aspect of the invention the motor <NUM> is a brushless DC motor that drives directly the pan platform <NUM> without a timing belt. This solution has the advantage of being quiet and allow very fast rotation speed. The motor is driven by a specialized electronic driver that allows control of the robot both in speed and angular position, including the possibility of locking the motor at any desired angle. In a variant, a brushless DC motor drives pan platform <NUM> through a strain-wave gear or another low-backlash reduction.

The speed of the electric motor and the reduction ratio of the strain-wave gear are chosen in consideration of the desired panning angular speed. In a typical realization, the maximum panning speed may be of <NUM> degree/s, which is achievable with a motor speed of <NUM> rpm and a <NUM>÷<NUM> reduction, but this is not an essential feature of the invention. The reduction ratio may be comprised between <NUM>÷<NUM> and <NUM>÷<NUM>.

The intermediate element <NUM> may have a "U" shape, as shown in <FIG>, or a "L" shape, or any other suitable shape. The "U" shape allows to journal the tilt platform between two bearings, for superior stiffness. The "L" shape is lighter and easier to carry. An embodiment involving an L-shaped pan platform will be disclosed further on, in reference to <FIG>.

The encoder yields a resolution of at least <NUM> points per revolution, or equivalently, better than <NUM> bits of resolution, and preferably more. <NUM>-bit and <NUM>-bit encoders, yielding more than <NUM><NUM>, respectively more than <NUM><NUM><NUM> codes per revolution, have provided satisfactory results. This result can be obtained by an accurate interpolator digitizing two quadrature sinusoidal analogue signal generated by the encoder by one or more high-resolution ADC and performing the necessary trigonometric calculation in a specially-programmed FPGA. Other structures are however possible for the interpolator. The speed of the encoder must be high enough to capture the motion of the panning platform at maximal speed without errors.

Preferably the motor <NUM> and the axle supporting the intermediate element are hollow, to simplify cable routing.

The panning platform <NUM> is attached to a camera holder <NUM> that can rotate about a tilt axis <NUM>, orthogonal to the pan axis <NUM>. Conventionally, the pan axis is vertical, and the tilt axis is horizontal and, in the present disclosure, the terms "horizontal" and "vertical" are used to denote the conventional positions of the pan and tilt axis when the camera head of the invention is in operation, but the invention is not limited to this disposition and, indeed, the base <NUM> could be mounted on an inclined platform such that the axis <NUM> is sloping.

The rotation about the tilt axis <NUM> is guaranteed by one or two motors <NUM> and, again, the tilt angle is read by a high-speed encoder. As for the pan axis, the tilt motor or motors <NUM> are preferably DC brushless motors with a hollow axle, driving the camera holder directly, or through a strain-wave gear of suitable reduction. The nominal speeds for the tilt axis may be essentially the same as those of the pan axis, or slightly inferior. The instantaneous tilt angle is read by high precision encoders having a resolution comparable with that of the encoders on the pan axis, for example or <NUM> codes per revolution, preferably at least one million codes per revolution, and adequate speed.

A third axis for roll movements, orthogonal to pan and tilt axis, could also be added. The main purpose of the roll axis would be the compensation of misalignments, since deliberate roll movements are seldom used in cinematography. Accordingly, the angle limits, speed and dynamic properties of the roll axis can be considerably lower than those of pan and tilt axes.

<FIG> relate to a ring actuator that is fixed to the camera holder <NUM> or to the camera itself by means of the clamp <NUM> and operates on a ring on the camera's objective by means of the gear <NUM>. The purpose of this actuator is to act on any possible ring control of an optical objective, typically on an iris, focus, or zoom control.

The angle of the gear <NUM> is also read by a suitable encoder or deduced by step counting. Preferably, the ring controllers include a torque measurement unit that reads and makes available the mechanical torque on the ring. The torque measurement unit could be a suitable mechanical sensor, or else the torque could be derived from the electrical operating parameters of the motor <NUM>. The speed required for the ring movements are generally lower than those needed for zoom and pan and the ring actuator may include a reduction means, like the timing belt <NUM>, or a gearbox, or another suitable arrangement. The electronic unit <NUM> is arranged to drive the motor <NUM> and interface with the encoder.

Importantly, the control of the pan-tilt motions as well as of the objective rings are the task of a controller (not drawn) that has access to the angles measured by the encoders. In a preferred variant of the invention, the controller has a calibration mode that determines the physical limit of motion of the actuators, be they the pan-tilt actuators or the ring ones, by moving the respective motors and monitoring the torque to detect an increase that signals that the ring has been pushed to the end of its permitted range or, when the torque is not read directly, recording the physical limits of the actuator from the encoders.

In a further calibration phase, the controller determines a map between the setting of each actuator, especially the zoom ring, and the field of view of the objective. This is done, at each desired position of the ring, by taking two or several images of the same scene while panning/and or tilting the head. The angle of the head can be varied continuously or in steps, and the angle at which each image is captured is precisely known by means of the encoders.

An automatic vision software is used in this step of the calibration to identify key features in each image and pair them across images taken at different pan/tilt setting. In this way the software can determine the displacement of the key feature in the image for each angular change and ring setting, thus reconstructing the field of view and a pixel-angle relationship for each setting of the zoom ring. This mapping is used in an automatic tracking system to treat tracking problems in physical 3d space and not in 2d pixels of the image.

This feature is particularly advantageous because it allows a precise control of the field of view, by robotic means. In particular, nonlinearities between ring setting and focal length can be automatically accounted for and corrected.

The calibration can be extended also to the focus ring, by an automatic analysis of images taken at different focus setting, determining the distance of key features that are in sharp focus. This can be obtained either by taking an image that contain points whose distances are previously known, or by determining the distance of the key features by any other suitable means. If the camera is mounted on a dolly, a movable platform, or on an automatic pedestal, the distance of key points could be determined by parallax shifts between two images taken from two different position spaced apart by a known distance.

The mapping between focus ring angle and distance of focus is repeated for different setting of the iris.

The automatic calibration may include the determination and adjustment of dynamic parameters, by changing the angle of the pan rotation axis and/or of the tilt rotation axis, and capturing video data during the change of angle, by a video camera mounted on the camera holder. A processor analyses the video data to extract dynamic parameters, such as vibrations, jerkiness, and damping, possibly adapting a dynamic model of the camera/camera head assembly and determines an allowed range of dynamic parameters for the pan rotation axis and/or the tilt rotation axis based on the captured video data.

Importantly, the automatic calibration outlined above is executed after every readjustment or change in the camera rig (for example after changing a lens or adding a teleprompter, or any other accessory). Preferably, the software resources determining the calibration are totally or in part embedded in the automatic head firmware, for fully autonomous operation.

In a preferred embodiment the automatic camera head of the invention is equipped with a modular platform for offline rigging platform attachable to the camera and releasably connected to the camera holder. The camera and lens actuators can be attached and fixed to the rigging platform which can be easily attached to the camera holder on the pan-tilt unit without tools.

According to another variant visible on <FIG>, the intermediate element <NUM> of the automatic camera head of the invention is "L" shaped with a horizontal arm connected to the pan motor and a vertical arm connected to the tilt motor, and a handle <NUM> at the top end of the vertical arm. Advantageously, the handle <NUM> is easily reachable and above the centre of mass, such that the head can be easily lifted and carried.

Preferably, the camera head has a communication interface connectable to a computer or to a network of computers and arranged to drive the pan axis and the tilt axis and/or the ring actuators according to directives received from an external system. In a favourable variant the camera head has an input device, such as the button <NUM>, and is arranged set the camera head in a manual state in which the motors are put in a zerotorque mode.

In the manual state, all the automatic movement are inhibited, and all the directives received from the communication interface are ignored or suspended. The pan and tilt axes, as well as other degrees of freedom of the camera are locked or may be moved manually. The button <NUM> is a safety feature, and preferably combined with appositive visual feedback, for example a light in the button or close to it that signals when the camera head is in the manual state, hence safe to approach.

Importantly, the button <NUM> allows to approach the camera and perform necessary manual adjustments without depowering or resetting the whole system, hence without losing an existing calibration state, which would be undesirable in a live production setup.

<FIG> shows the offline rigging platform mounted on the camera head and disassembled in its constituents. The riffing platform includes a slide <NUM> arranged to slide linearly on the support <NUM>. A pair of mating dovetail surfaces, or a similar arrangement of collaborating surfaces on the slide <NUM> and on the support <NUM> ensures smooth relative motion between these two parts. The support <NUM> is attachable to the tilt platform <NUM>, and the slide <NUM> has slots, holes, or other suitable attachment points to receive and hold a video camera (not drawn).

The rigging platform includes an extension with two bars <NUM>, held in a parallel configuration by the end-caps <NUM>, into which they fit precisely. The bars <NUM> are used to attach one, two, or more ring actuators by the respective clamps <NUM> (see <FIG>). The spacer <NUM> can be changed, or stacked with other compatible spacers, to modify the vertical distance between the bars <NUM> and the optical axis of the camera, according to the needs, and specifically according to the diameter of the objective lens.

Claim 1:
An automatic camera head for panning and tilting a video camera, comprising a base (<NUM>), an intermediate element (<NUM>) connected to said base (<NUM>) and rotatable relative to said base (<NUM>) about a pan rotation axis (<NUM>), a camera holder (<NUM>) connected to said intermediate element (<NUM>) and rotatable relative to said intermediate (<NUM>) element about a tilt rotation axis (<NUM>) orthogonal to the pan rotation axis (<NUM>), a pan actuator (<NUM>) arranged to turn the intermediate (<NUM>) element about the pan rotation axis (<NUM>), a tilt actuator (<NUM>) arranged to turn the camera holder (<NUM>) about the tilt rotation axis (<NUM>), wherein the pan actuator (<NUM>) and/or the tilt actuator (<NUM>) include DC brushless motors driving the intermediate element respectively the camera holder directly, or via a strain-wave reduction gear, characterised in that
the automatic camera head further comprises at least one zoom ring actuator (<NUM>) arranged to act on a zoom control of an objective of the camera,
the automatic camera head has a control unit arranged to execute a calibration process and to determine a map between a setting of the zoom control and the field of view of the objective with the steps of:
- moving the zoom control by the zoom ring actuator;
- determining limit angles in the zoom control;
- determining a field of view of the objective at a plurality of positions of the zoom control, and in that
determining the field of view of the objective includes:
- taking at least a first image and a second image of a scene by the camera at different pan and/or tilt angles:
- identifying and pairing key features in the first and second images,
- determining the field of view based on the separation of paired key features in the first and second images.