Patent Description:
Image sensor systems are common in factory production lines for automation or labor saving in inspecting and managing products. A typical known system includes a camera and a controller connected to each other with a cable (refer to Patent Literature <NUM>). A recent processor-integrated image sensor combines a camera with a controller to perform processes from imaging control to image processing in a single device. Such a processor-integrated image sensor is also referred to as a smart camera, and may incorporate an illuminator and a lens.

A recent image sensor includes an optical system incorporating a liquid lens. Patent Literature <NUM> describes an imaging optical system shown in <FIG> including multiple solid lenses (<NUM>, <NUM>, <NUM>), an aperture stop (<NUM>), and a liquid lens arranged in order of distance from the object. The literature also describes the solid lenses preferably formed from glass or plastic. The liquid lens is a varifocal lens having the refractive power adjustable by changing the application voltage. An optical system incorporating the liquid lens includes no mechanical moving parts, and thus allows faster autofocus (AF) and has a longer service life (or an infinite service life) than typical motor-driven optical systems.

Patent Literature <NUM>: <CIT> <CIT> describes an optical information reader having a focus function and a zoom function using liquid lenses. The reader comprises a lens module having a plurality of liquid lenses. Each liquid lens may be provided with a temperature sensor. <CIT> describes a camera module including a lens capable of adjusting the focal distance using electrical energy. The module comprises a first lens unit, a second lens unit, a liquid lens unit and a lens housing. A temperature sensor arranged in a groove of the lens housing is capable of measuring the ambient temperature of the liquid lens unit.

For the above processor-integrated image sensor, the inventors have noted the use of an optical system combining a liquid lens with plastic lenses, which are lighter and less expensive than glass lenses. An image sensor may be attached to a moving object (e.g., the end of a robot arm) to capture images while the position, the posture, and the focus position of the image sensor are being changed as intended. For such use, the image sensor being lightweight or enabling fast AF can provide a high added value.

However, the inventors exploring such structures have faced an issue of temperature compensation in the optical system. Liquid lenses and plastic lenses are more temperature-dependent than, for example, glass lenses, and may have the optical properties greatly dependent on temperature. In particular, a processor-integrated image sensor includes an image sensor body including components that generate much heat (e.g., a processor and a drive circuit). The image sensor in operation may have a temperature a dozen degrees Celsius higher than ambient temperature, and may undergo temperature-dependent changes that are not negligible. In such circumstances, the image sensor may include a temperature compensator for monitoring the lens temperature and adaptively correcting the optical properties for stable performance. However, temperature sensors to be installed for individual lenses or structures for mechanically adjusting the intervals between the lenses can structurally complicate the optical system and increase the weight and cost, and thus cannot be used.

In response to the above issue, one or more aspects of the present invention are directed to an image sensor that includes an optical system combining a liquid lens with plastic lenses including temperature sensors and a controller, as defined by claim <NUM>, and achieves accurate temperature compensation with a simple structure.

An image sensor according to a first aspect of the present invention includes the features according to claim <NUM>. The image sensor with this structure includes the optical system combining the liquid lens with the plastic lenses and achieves accurate temperature compensation with a simple structure.

For example, the controller may control the voltage to be applied from the electrode to the liquid lens in accordance with the temperature measured by the temperature sensors to change a refractive power of the liquid lens so as to cancel a temperature-dependent change in properties of the first plastic lens and the second plastic lens. The liquid lens allows temperature compensation in the entire optical system, thus increasing the reliability and stability of the image sensor. This also eliminates any additional special temperature compensator or any structure for mechanically adjusting the intervals between the lenses.

The temperature sensors may be located on a substrate on which the electrode is formed. The substrate is commonly used by the electrode for applying a voltage to the liquid lens and the temperature sensor. Thus, the structure is simpler, includes fewer components, and is less costly. The electrode (substrate) are to be used to apply a voltage to the liquid lens and thus located near the liquid lens. Thus, the temperature sensors can be easily designed to be adjacent to the liquid lens.

In the above aspects of the present invention, the "temperature sensor located near the liquid lens" refers to a plurality of temperature.

The structure including one temperature sensor may be simplest and cost-effective, however, does not fall within the scope of the claims. The structure including multiple temperature sensors use measured temperatures at multiple positions and thus increase the accuracy of temperature compensation According to the independent claim, multiple temperature sensors are arranged along the optical axis of the optical unit to determine measured temperatures at multiple positions along the temperature gradient. This allows accurate estimation of the temperature gradient and the lens temperatures.

The optical unit may include a lens barrel supporting the optical system, and the lens barrel may include a passage connecting a space on the object side of the liquid lens in the lens barrel with a space on the image side of the liquid lens. The passage allows warmer air in the space on the image side to be replaced with the air in the space on the object side. This reduces the temperature gradient and the temperature differences between the first plastic lens, the liquid lens, and the second plastic lens. The temperatures at both ends of the temperature gradient (the positions of the first and second plastic lenses) may be estimated from the measured temperature at an intermediate position (the position of the liquid lens) along the temperature gradient. The estimation is likely to be more accurate with a system having a small temperature gradient than with a system having a large temperature gradient. The passage reduces the temperature gradient, thus increasing the accuracy of temperature compensation.

The image sensor may further include a heat insulator between the optical unit and the controller. The heat insulator reduces heat transfer from the controller to the optical unit. This reduces the temperature rise on the image side of the optical unit, and thus reduces the temperature gradient in the optical unit.

An image sensor according to one or more aspects of the present invention includes an optical system combining a liquid lens with plastic lenses including temperature sensors and a controller, as defined by claim <NUM>, and achieves accurate temperature compensation with a simple structure.

An example use of the present invention will now be described. <FIG> is a schematic diagram of an image sensor according to an example, not falling within the scope of the claims.

An image sensor <NUM> mainly includes an imaging device <NUM>, an optical unit <NUM>, and a controller <NUM>. The optical unit <NUM> guides light to the imaging device <NUM>. The optical unit <NUM> includes an optical system <NUM>, a flexible substrate <NUM>, and a lens barrel <NUM>. The optical system <NUM> includes a first plastic lens <NUM>, a liquid lens <NUM>, and a second plastic lens <NUM> arranged in order of distance from the object. The flexible substrate <NUM> includes electrodes <NUM> for applying a voltage to the liquid lens <NUM>, and a temperature sensor <NUM>. The lens barrel <NUM> is a housing supporting the optical system <NUM>. The controller <NUM> controls the imaging device <NUM> and the optical unit <NUM> and performs image processing and other computation. The imaging device <NUM> and the controller <NUM> are inside a housing for the image sensor body.

The controller <NUM> monitors the temperature of the optical unit <NUM> using the temperature sensor <NUM> during the operation of the image sensor <NUM>. The controller <NUM> controls the voltage to be applied from the electrodes <NUM> to the liquid lens <NUM> in accordance with the temperature measured by the temperature sensor <NUM> to adjust the refractive power of the liquid lens <NUM>.

The liquid lens <NUM> has the refractive power adjusted to change the focus position. For example, the liquid lens <NUM> has the focus position adjusted in accordance with the distance from the image sensor <NUM> to the object measured by a range sensor (not shown). This allows fast active AF. The liquid lens <NUM> in the present example has the refractive power adjusted also for temperature compensation in the optical system <NUM>. More specifically, the liquid lens <NUM> has the refractive power changed to cancel the temperature-dependent change in the properties of the first plastic lens <NUM> and the second plastic lens <NUM>. Thus, the optical system <NUM> has the optical properties maintained constant as a whole independently of temperature.

The optical system <NUM> with this structure incorporates the liquid lens <NUM> and thus allows faster AF and has a longer service life than a motor-driven optical system. The plastic lenses <NUM> and <NUM> as solid lenses are combined with the liquid lens <NUM> to reduce the weight and the cost of the optical system <NUM>, thus reducing the weight and the cost of the entire image sensor <NUM>.

The liquid lens <NUM> and the plastic lenses <NUM> and <NUM> are more temperature-dependent than, for example, glass lenses, and have the optical properties greatly dependent on temperature. Thus, the liquid lens <NUM> in the present example receives a voltage controlled in accordance with the temperature measured by the temperature sensor <NUM> to have the optical properties (refractive power) adjusted adaptively. This allows temperature compensation in the entire optical system <NUM> including the liquid lens <NUM> and the plastic lenses <NUM> and <NUM>. The liquid lens <NUM> allows temperature compensation in the entire optical system <NUM>, thus increasing the reliability and stability of the image sensor <NUM>. This also eliminates any additional special temperature compensator or any structure for mechanically adjusting the intervals between the lenses.

The temperature sensor <NUM> in the present example is adjacent to the liquid lens <NUM>. The temperature sensor <NUM> at this location allows accurate detection or estimation of the temperature of the liquid lens <NUM>, thus allowing the liquid lens <NUM> to serve as a more accurate temperature compensator. The liquid lens <NUM> is typically more temperature-dependent than the plastic lenses <NUM> and <NUM>. Thus, the liquid lens <NUM> may serve as an accurate temperature compensator to increase the accuracy of temperature compensation in the entire optical system <NUM> including the liquid lens <NUM> and the plastic lenses <NUM> and <NUM>.

The temperature sensor <NUM> may be adjacent to the liquid lens <NUM> also for the reason below. An electrical component such as the temperature sensor <NUM> is to be installed in the optical unit <NUM> and electrically connected to the controller <NUM> in the image sensor body. The electrodes <NUM>, which are adjacent to the liquid lens <NUM> to apply a voltage to the liquid lens <NUM>, allow design that can easily incorporate an additional component for electrically connecting the electrical component.

The liquid lens <NUM> in the present example is located between the two plastic lenses <NUM> and <NUM>. This structure may increase the accuracy of temperature compensation. The image sensor body typically includes components that generate much heat, such as a processor, a drive circuit, a power integrated circuit (IC), and a coil component, which are hereafter collectively referred to as a heating element. Thus, the image side of the optical unit <NUM> is susceptible to their heat. In contrast, the object side of the optical unit <NUM> is away from the heating element and is thus dependent on ambient temperature around the image sensor <NUM>. During the operation of the image sensor <NUM>, the optical unit <NUM> has a temperature gradient at which the temperature gradually decreases from the image side to the object side of the optical unit <NUM>. Thus, the lenses <NUM> to <NUM> included in the optical system <NUM> have different temperatures. The lens barrel <NUM> (lens support) is to be formed from a plastic material to have a coefficient of linear expansion similar to that of the plastic lenses <NUM> and <NUM>. However, the plastic material typically has low thermal conductivity and tends to maintain the above temperature gradient (or in other words, the temperature differences between the lenses) over time. The optical unit <NUM> is expected to have a temperature difference between the image side and the object side of a dozen degrees Celsius or higher, depending on the design. The temperature difference also depends on the heating element temperature and the environmental temperature. For example, the heating element generates more heat with more frequent image capture or under a greater processing load.

<FIG> schematically shows the temperature gradient along the optical axis. In <FIG>, P1 indicates the position of the first plastic lens <NUM>, P2 indicates the position of the second plastic lens <NUM>, P3 indicates the position of the liquid lens <NUM>, and the vertical axis indicates the temperature at each position. The figure schematically shows the temperature gradually decreasing from the image side to the object side.

In the present example, the liquid lens <NUM> is located between the two plastic lenses <NUM> and <NUM>, and the temperature sensor <NUM> is adjacent to the liquid lens <NUM>. Thus, the temperature of the liquid lens <NUM> at the position P3 can be determined accurately. The temperatures of the plastic lenses <NUM> and <NUM> (the temperatures at the positions P1 and P2) are estimated from the measured value obtained by the temperature sensor <NUM>. Thus, the estimation accuracy tends to decrease at a greater distance from the position P3. <FIG> shows error bars indicating the error in temperature at the positions P1 and P2 estimated from the measured value at the position P3.

In the structure including the first plastic lens, the second plastic lens, and the liquid lens arranged in order of distance from the object with the temperature sensor installed adjacent to the liquid lens, the temperature of the liquid lens may be accurately measured but the temperature of the first plastic lens, nearest the object, may be inaccurately estimated and may have large error at the position P1 as shown in <FIG>. A similar issue may arise for the liquid lens, the first plastic lens, and the second plastic lens arranged in order of distance from the object.

Such comparison and examination have revealed the structure in the present example including the liquid lens <NUM> between the two plastic lenses <NUM> and <NUM>, and the temperature sensor <NUM> adjacent to the liquid lens <NUM>. In this structure, the temperature sensor <NUM> is located not far from either the first plastic lens <NUM> or the second plastic lens <NUM>. The temperature sensor <NUM> measures the temperature at an intermediate position (the position P3) along the temperature gradient at which the temperature gradually decreases from the image side (the second plastic lens <NUM>) to the object side (the first plastic lens <NUM>). Thus, the temperature sensor <NUM> adjacent to the liquid lens <NUM> alone can determine the temperature state of the liquid lens <NUM>, and also allows estimation of the temperature states of the two plastic lenses <NUM> and <NUM> with satisfactory accuracy.

The image sensor <NUM> according to the present embodiment includes the optical system <NUM> combining the liquid lens <NUM> with the plastic lenses <NUM> and <NUM> and achieves accurate temperature compensation with a simple structure.

<FIG> shows an example sensor system including the image sensors according to the embodiment of the present invention. A sensor system <NUM> according to the present embodiment inspects and manages products <NUM> on, for example, a production line. The sensor system <NUM> includes multiple image sensors <NUM> and an information processor <NUM>. The information processor <NUM> is connected to the image sensors <NUM> through an industrial network <NUM> such as Ethernet for Control Automation Technology (EtherCAT), and transmits and receives data to and from the image sensors <NUM> through the network <NUM>. In the example in <FIG>, three image sensors <NUM> are installed to capture images of the products <NUM> carried on a conveyor <NUM>. However, any other number of image sensors <NUM> may be installed. A large factory may include tens, hundreds, or more image sensors. The sensor system <NUM> may include image sensors <NUM> attached to moving objects such as robot arms. Each image sensor <NUM> may capture images of a product <NUM> from different angles while changing its position, posture, and focus position.

The industrial image sensor <NUM> is used for various image-based processes. Examples include recording images of objects to be inspected, recognizing shapes, detecting edges and measuring widths and numbers, measuring areas, determining color features, labeling and segmentation, object recognition, reading barcodes and two-dimensional codes, optical character recognition (OCR), and individual identification. The processor-integrated image sensor (smart camera) according to the present embodiment combines the imaging system with the processing system. In some embodiments, the imaging system may be separated from the processing system in an image sensor. The optical unit described above may be included in such an image sensor. The image sensor <NUM> is also referred to as, for example, a vision sensor or a vision system.

<FIG> and <FIG> show an image sensor according to a first example. <FIG> is a perspective view of the image sensor according to the first example showing its main part. <FIG> is a cross-sectional view of the image sensor according to the first example showing its main part, not falling within the scope of the claims.

The optical unit <NUM> includes the optical system <NUM> combining the two plastic lenses <NUM> and <NUM> with the liquid lens <NUM>. The first plastic lens <NUM>, the liquid lens <NUM>, and the second plastic lens <NUM> are arranged in order from the object side and assembled on the lens barrel <NUM>. The lenses <NUM>, <NUM>, and <NUM> are respectively fastened with holder rings <NUM>, <NUM>, and <NUM>. The lens barrel <NUM> is formed from a resin material having a coefficient of linear expansion similar to that of the plastic lenses <NUM> and <NUM>. Reference numeral <NUM> indicates a flexible substrate. The flexible substrate <NUM> includes the electrodes <NUM> for applying a voltage to the liquid lens <NUM>, and the temperature sensor <NUM>. The flexible substrate <NUM> is connected to a control board <NUM> with a connector <NUM>. The control board <NUM> incorporates, for example, the imaging device <NUM>, a processor <NUM>, and a memory <NUM>. The processor <NUM> and the memory <NUM> in the example form the controller <NUM> in <FIG>.

The temperature sensor <NUM> measures the temperature around the liquid lens <NUM> and may be, for example, a thermistor. The temperature sensor <NUM> measures the temperature, which is then received by the processor <NUM> through the flexible substrate <NUM>.

The imaging device <NUM> generates and outputs image data by photoelectric conversion, and may include, for example, a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS). For example, the processor <NUM> performs image processing (e.g., preprocessing and feature extraction) on image data, performs various processes (e.g., inspection, character recognition, and individual identification) based on the results of the image processing, transmits and receives data to and from an external device, generates data to be output to the external device, processes data received from the external device, and controls the liquid lens <NUM> and the imaging device <NUM>. For example, the processor <NUM> may be a general-purpose processor such as a central processing unit (CPU) or microprocessor unit (MPU), or may be a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). The memory <NUM> is a nonvolatile storage device, such as an electrically erasable programmable read-only memory (EEPROM). The memory <NUM> stores programs and data to be used by the processor <NUM>.

<FIG> shows example data stored in the memory <NUM> for determining the application voltage for the liquid lens <NUM>. The data represents a table defining the correspondence between the focus position (the distance from the image sensor to the object to be in focus), the temperature measured by the temperature sensor <NUM>, and the voltage to be applied to the liquid lens <NUM>. In the table, the values v11, v12,. of the application voltage are set to achieve both intended focus positions and temperature compensation at the corresponding temperatures. Specific values for the application voltage may be determined by experiments or simulation. In some embodiments, the temperature properties may be measured for individual products to determine the values of the application voltage, for example, before shipment, to accommodate non-negligible variations in the individual optical systems <NUM>.

The processor <NUM> constantly monitors the measured temperature received from the temperature sensor <NUM> during the operation of the image sensor <NUM>. The processor <NUM> determines, as appropriate, the value of the voltage to be applied and controls the voltage value to be output to the electrodes <NUM> based on the measured temperature, the intended focus position, and the table in <FIG>. The control allows accurate adjustment to the intended focus position independently of temperature, thus increasing the reliability and stability of the image sensor <NUM>.

The table in <FIG> may not include a value corresponding to the measured temperature and the intended focus position. In this case, the value corresponding to the closest conditions may be selected, or an appropriate voltage value may be calculated by interpolation. The example table in <FIG> may be modified to have finer or coarser increments of the conditions. In some embodiments, the data may be in the form of a function, instead of a table, that defines the relationship among the focus position, the temperature, and the application voltage.

<FIG> shows an image sensor according to a second example, not falling within the scope of the claims. In the image sensor according to the second example, the lens barrel <NUM> includes a passage <NUM> and a passage <NUM>. The passage <NUM> connects a space <NUM> on the object side of the liquid lens <NUM> with a space <NUM> on the image side of the liquid lens <NUM>. The passage <NUM> connects the space <NUM> on the object side of the second plastic lens <NUM> with a space <NUM> on the image side of the second plastic lens <NUM>.

The passages <NUM> and <NUM> allow warmer air in the spaces nearer the image to be replaced with air in the spaces nearer the object. This reduces the temperature gradient as shown in <FIG>, and reduces the temperature differences between the first plastic lens <NUM>, the liquid lens <NUM>, and the second plastic lens <NUM>. The temperatures at both ends of the temperature gradient (the positions P1 and P2 of the first and second plastic lenses) may be estimated from the measured temperature at an intermediate position (the position P3 of the liquid lens) along the temperature gradient. The estimation is likely to be more accurate with a system having a small temperature gradient than with a system having a large temperature gradient, as can be seen by comparing <FIG> with <FIG>. The passages <NUM> and <NUM> reduce the temperature gradient, thus increasing the accuracy of temperature compensation.

<FIG> shows an image sensor according to a third example, not falling within the scope of the claims. The image sensor according to the third example includes a heat insulator <NUM> between the optical unit <NUM> and the controller <NUM>.

The heat insulator <NUM> may be a plate of transparent resin or glass. The heat insulator <NUM> reduces heat transfer from the controller <NUM> to the optical unit <NUM>. This reduces the temperature rise on the image side of the optical unit <NUM>, and thus reduces the temperature gradient. The image sensor thus has the effects similar to those of the second example.

<FIG> shows an image sensor according to a fourth example, according to a preferred embodiment of the present invention. The image sensor according to the fourth example includes two temperature sensors 117a and 117b arranged along the optical axis. The image sensor with this structure provides measured temperatures at two positions along the temperature gradient. This allows accurate estimation of the temperature gradient and the lens temperatures. The image sensor may thus allow more accurate temperature compensation than in the first to third examples described above. Although <FIG> shows the two temperature sensors, three or more temperature sensors may be included.

Claim 1:
An image sensor (<NUM>), comprising:
an imaging device (<NUM>);
an optical unit (<NUM>) configured to guide light to the imaging device (<NUM>); and
a controller (<NUM>) configured to control the imaging device (<NUM>) and the optical unit (<NUM>), wherein
the optical unit (<NUM>) includes:
an optical system (<NUM>) including a single liquid lens (<NUM>), a first plastic lens (<NUM>) located on an object side of the single liquid lens (<NUM>), and a second plastic lens (<NUM>) located on an image side of the single liquid lens (<NUM>); and
an electrode (<NUM>) to apply a voltage to the single liquid lens (<NUM>); wherein
a plurality of temperature sensors (117a, 117b) arranged along an optical axis are located near the single liquid lens (<NUM>) in the optical unit (<NUM>), and
the controller (<NUM>) controls the voltage to be applied from the electrode (<NUM>) to the single liquid lens (<NUM>) in accordance with a temperature measured by the plurality of temperature sensors (117a, 117b).