Patent Description:
Known small sensor systems capture images of inspection targets and analyze the captured images to inspect and measure the inspection targets. Such sensor systems (hereafter, image sensors) typically change the positions of selected lenses along the optical axis mechanically to adjust the focus. Some image sensors include liquid lenses to adjust the focus (refer to, for example, Patent Literature <NUM>). A liquid lens is an optical component containing a conductive water solution and a nonconductive oil sealed in a lens holder, with the oil-water interface deforming to change the refractive power in response to a voltage applied.

<CIT> describes a camera system comprising an optical unit, a camera body and an adaptor for detachably connecting the optical unit to the camera body. The optical unit may comprise a liquid lens able to adjust its refractive index by the application of a voltage. <CIT> describes a photographical system comprising a photographical lens and a camera body. A focus adjustment unit is able to control a focus of the photographical system by moving the photographical lens forward and backward with respect to the camera body by a motor drive section.

The use of a liquid lens eliminates a lens moving unit and allows the optical system to be compact. Image sensors are expected to have parameters (e.g., the angle of view and the number of pixels of captured images) that may differ depending on users. For an image sensor including an optical system with a liquid lens, the optical system and the imaging device in the image sensor may be modularized to allow a user to select an optical system and an imaging device as appropriate.

For an image sensor including a liquid lens, the voltage applied to the liquid lens is adjusted to change the refractive power. Although the liquid lens may have the refractive power that allows light collected by the optical system with the liquid lens to have a blur circle diameter smaller than or equal to the pixel pitch for an image in focus, a certain period of time is taken from a change in the application voltage to the liquid lens to a change in the refractive power that causes light collected by the optical system with the liquid lens to have a blur circle diameter smaller than or equal to the pixel pitch. In an image sensor including an optical system with a liquid lens, image capturing is to be delayed by a predetermined period after the application voltage to the liquid lens is changed. The blur circle diameter changes differently in response to a change in the application voltage to the liquid lens depending on the lens module selected. A different pixel pitch is set for a different imaging module. Thus, the period taken from a change in the application voltage to the liquid lens to achieving the focus of the image differs depending on the combination of the lens module and the imaging module. Using the same wait period uniformly for any combination of a lens module and an imaging module can produce a blurred image through image capturing performed before the focus is achieved, or may cause an excess wait period after the focus is achieved.

In response to this, a modularized image sensor may use, for any combinations of modules, a specific wait period (hereafter, a predefined wait period) defined as appropriate for a combination of a specific lens module (hereafter, a reference lens module) and a specific imaging device (hereafter, a reference imaging device). The image sensor with this structure may use a selected combination of a lens module less responsive to a change in the application voltage than the reference lens module and an imaging device having a smaller pixel pitch than the reference imaging device. This image sensor may take a longer period from the voltage application to achieving the focus of the image than the predefined wait period, thus producing a blurred and less accurate captured image. The image sensor may use a selected combination of a lens module more responsive to a change in the application voltage than the reference lens module and an imaging device having a larger pixel pitch than the reference imaging device. This image sensor may take a shorter period from the voltage application to achieving the focus of the image than the predefined wait period, thus causing an excess wait period from the voltage application to the image capturing.

In response to the above issue, one or more aspects of the present invention are directed to an image sensor that adjusts the wait period after a change in the application voltage to a liquid lens as appropriate for a combination of a lens module and an imaging module attached to a body module.

An image sensor according to one aspect of the present invention is provided by claims <NUM> to <NUM>.

The image sensor according to the above aspects of the present invention adjusts the wait period after a change in the application voltage to a liquid lens as appropriate for a combination of a lens module and an imaging module attached to a body module.

<FIG> is a diagram of an image sensor <NUM> according to a first embodiment of the present invention.

As shown in the figure, the image sensor <NUM> according to the present embodiment includes a body module <NUM> to which a lens module selected from multiple lens modules (LMs) <NUM> and an imaging module selected from multiple imaging modules (CMs) <NUM> are attachable. The image sensor <NUM> is connected to an information processor <NUM> when in use. The information processor <NUM> is a computer with a program for using the image sensor <NUM> installed. The information processor <NUM> is typically connected to multiple image sensors <NUM>.

Each imaging module <NUM> includes an imaging device <NUM> (monochrome image sensing device in the present embodiment), such as a complementary metal-oxide-semiconductor (CMOS) image sensing device or a charge-coupled device (CCD) image sensing device. An imaging module <NUM> to be attached to the body module <NUM> can be selected from multiple imaging modules <NUM> each including an imaging device <NUM> with different specifications (e.g., the pixel pitch and the number of pixels). Each imaging module <NUM> includes a nonvolatile memory <NUM>, such as a serial electrically erasable programmable read-only memory (EEPROM), storing imaging module format information (hereafter, CM format information) indicating the format of the imaging module <NUM>.

Each lens module <NUM> includes an optical system <NUM> (a combination of lenses) including a liquid lens <NUM>. A lens module <NUM> can be selected from multiple lens modules <NUM> each including an optical system <NUM> with different specifications (e.g., the type of the liquid lens <NUM> in the optical system <NUM>). Each lens module <NUM> includes a nonvolatile memory <NUM>, such as a serial EEPROM, storing lens module format information (hereafter, LM format information) indicating the format of the lens module <NUM>.

The body module <NUM>, to which a lens module <NUM> and an imaging module <NUM> are attachable, includes a controller <NUM> and a storage <NUM>.

The controller <NUM> performs a refractive power control process, an elapsed-period monitoring process, a recognition process, and an adjustment process (described in detail later). The controller <NUM> includes, for example, a driver integrated circuit (IC) for generating an application voltage to the liquid lens <NUM> and a microcontroller.

The storage <NUM> stores various items of information. The storage <NUM> includes a random-access memory (RAM) to be used as a work area by the controller <NUM> and a nonvolatile memory such as a flash memory. The nonvolatile memory in the storage <NUM> stores a wait period table <NUM> (described in detail later).

The refractive power control process, the elapsed-period monitoring process, the recognition process, and the adjustment process to be performed by the controller <NUM> will now be described one after another.

The refractive power control process refers to adjusting the application voltage to the liquid lens <NUM> to control the refractive power of the liquid lens <NUM>. The controller <NUM> performs the refractive power control process in response to an instruction from the information processor <NUM> for changing the installation distance. The installation distance refers to the distance from the front end of the optical system <NUM> to the inspection target. In the refractive power control process, the controller <NUM> determines the voltage to be applied to the liquid lens <NUM> based on an installation distance (an installation distance after a change) designated by the instruction for changing the distance. The controller <NUM> then applies the determined voltage to the liquid lens <NUM>.

The elapsed-period monitoring process is started by the controller <NUM> at the start of the refractive power control process (or in other words, in response to a change in the application voltage to the liquid lens <NUM>). In the elapsed-period monitoring process, the controller <NUM> first sets a waiting completion flag (its use will be described later) to <NUM>. The controller <NUM> then starts measuring the elapsed period after the change in the application voltage. In response to the elapsed period reaching a preset wait period, the controller <NUM> changes the waiting completion flag to <NUM> and ends the elapsed-period monitoring process. The preset wait period is a parameter determined by the adjustment process to be used in the elapsed-period monitoring process and the recognition process. The preset wait period is described in detail later with the adjustment process.

The recognition process shown in <FIG> is performed by the controller <NUM> in response to a predetermined instruction from the information processor <NUM>.

More specifically, the controller <NUM> starting the recognition process in response to the predetermined instruction first determines whether the waiting completion flag has been set to <NUM> (step S101). In response to the waiting completion flag set to <NUM> (Yes in step S101), the controller <NUM> controls the imaging device <NUM> to output image data (step S102). The controller <NUM> then analyzes the image data from the imaging device <NUM> and transmits the analysis result to the information processor <NUM> (step S103). The analysis performed by the controller <NUM> in step S103 is intended for, for example, the recognition of a character string printed on an inspection target or the detection of the position of a label on an inspection target. Upon completing step S103, the controller <NUM> ends the recognition process (<FIG>).

In response to the waiting completion flag not set to <NUM> (No in step S101), the controller <NUM> waits in step S101 until the waiting completion flag is set to <NUM>. In response to the waiting completion flag set to <NUM> (Yes in step S101), the controller <NUM> performs step S102 and the subsequent step.

As in the elapsed-period monitoring process detailed above, the waiting completion flag indicates a value other than <NUM> (specifically, <NUM>) simply in the period from when the application voltage to the liquid lens <NUM> is changed to when the preset wait period has elapsed. In the recognition process (<FIG>), image data is analyzed (step S103) after at least the preset wait period elapses from the change in the application voltage to the liquid lens <NUM>.

The adjustment process is performed by the controller <NUM> when the image sensor <NUM> is turned on for adjusting the preset wait period as appropriate for the combination of the lens module <NUM> and the imaging module <NUM> attached to the body module <NUM>.

The adjustment process will now be described with reference to <FIG> and <FIG>. <FIG> is a flowchart of the adjustment process. <FIG> is an example of the wait period table <NUM>. The lens module <NUM> having LM format information LM-X (X = A, B) is hereafter referred to as an LM 20X. The imaging module <NUM> having CM format information CM-Y (Y = A to D) is hereafter referred to as a CM 30Y.

In the adjustment process, the controller <NUM> first reads LM format information from the nonvolatile memory <NUM> and CM format information from the nonvolatile memory <NUM> in the modules attached to the body module <NUM> (step S201), as shown in <FIG>. The controller <NUM> then reads, from the wait period table <NUM>, the wait period associated with the read LM format information and the read CM format information (step S202). The controller <NUM> then stores the read wait period as a preset wait period (step S203) and ends the adjustment process.

The wait period table <NUM> (<FIG>) stores a wait period (e.g., <NUM>) for each combination of LM format information (e.g., LM-A) and CM format information (e.g., CM-A).

For the image sensor <NUM> including the LM 20X and the CM 30Y, the wait period table <NUM> stores a wait period in association with LM-X and CM-Y. This wait period refers to the period taken for the optical system <NUM> to have a blur circle diameter matching the pixel pitch of the imaging device <NUM> in the CM 30Y after a change in the application voltage to the liquid lens <NUM>. As described above, the adjustment process includes reading, from the wait period table <NUM>, the wait period associated with the LM format information and the CM format information about the modules attached to the body module <NUM>, and storing the read wait period as a preset wait period. The image sensor <NUM> thus adjusts the preset wait period as appropriate for the combination of the lens module <NUM> and the imaging module <NUM> attached to the body module <NUM>.

The above procedure can adjust the preset wait period as appropriate for the combination of the lens module <NUM> and the imaging module <NUM> attached to the body module <NUM> for the reasons described below.

In response to a change in the application voltage to the liquid lens <NUM>, the refractive power of the liquid lens <NUM> gradually approaches a target refractive power (the refractive power corresponding to the application voltage after the change). The blur circle diameter of the optical system <NUM> including the liquid lens <NUM> thus decreases with time after voltage application, as indicated by a curve <NUM> in <FIG>. The curve <NUM> is associated with the optical system <NUM> in the LM 20A. In <FIG>, PY (Y = A to D) is the pixel pitch of the imaging device <NUM> in the CM 30Y.

As indicated by the curve <NUM>, the liquid lens <NUM> in the optical system <NUM> in the LM 20A has the refractive power that stabilizes (or reaches the target refractive power) in about <NUM>. Some liquid lenses <NUM> may use a longer period before the refractive power stabilizes. The preset wait period may be the period taken to stabilize the refractive power of the liquid lens <NUM> in the optical system <NUM> in every LM <NUM> attachable to the body module <NUM> (hereafter, the maximum response period). This allows the production of image data in focus with any combination of an LM <NUM> and a CM <NUM> attached to the body module <NUM>. Setting the maximum response period to the preset wait period can prevent the controller <NUM> from analyzing image data out of focus and producing an erroneous analysis result. For the blur circle diameter being smaller than or equal to the pixel pitch of the imaging device <NUM>, image data in focus can be produced before the maximum response period elapses.

More specifically, the LM 20A and the CM 30A including the imaging device <NUM> with a pixel pitch PA may be attached to the body module <NUM>. The optical system <NUM> in the LM 20A may have a blur circle diameter changing with time as indicated by the curve <NUM> in response to a change in the application voltage to the liquid lens <NUM>. As shown in <FIG>, the optical system <NUM> uses <NUM> before the blur circle diameter matches PA. Thus, the wait period of <NUM> after a change in the application voltage allows the image data to be as accurate as with a longer wait period. A shorter wait period allows a single measurement with a change in the application voltage to be performed in a shorter period. The above procedure can thus adjust the preset wait period as appropriate for the combination of the lens module <NUM> and the imaging module <NUM> attached to the body module <NUM>.

<FIG> is a diagram of an image sensor <NUM> according to a second embodiment of the present invention.

The image sensor <NUM> will be described focusing on the structure and the operation different from those of the image sensor <NUM> (<FIG>) according to the first embodiment.

Similarly to the image sensor <NUM>, the image sensor <NUM> includes the body module <NUM> to which a lens module <NUM> and an imaging module <NUM> are attachable. The lens module <NUM> and the imaging module <NUM> for the image sensor <NUM> have the same hardware configurations as the lens module <NUM> and the imaging module <NUM> for the image sensor <NUM>. However, the nonvolatile memory <NUM> in the lens module <NUM> for the image sensor <NUM> stores blur circle diameter information instead of LM format information. The nonvolatile memory <NUM> in the imaging module <NUM> for the image sensor <NUM> stores pixel pitch information instead of CM format information.

The blur circle diameter information stored in the nonvolatile memory <NUM> in a specific lens module <NUM> indicates the change in the blur circle diameter with time after the application voltage is applied to the liquid lens <NUM> in the optical system <NUM> in the lens module <NUM>. The blur circle diameter information may include coefficients in an approximation function that approximates a curve indicating the relationship between the blur circle diameter and the elapsed period (refer to the curve <NUM> in <FIG>), or may include the coordinates (the blur circle diameter and the elapsed period) of multiple points on the curve.

The pixel pitch information stored in the nonvolatile memory <NUM> in a specific imaging module <NUM> indicates the pixel pitch of the imaging device <NUM> in the imaging module <NUM>. The pixel pitch information may indicate the pixel pitch of the imaging device <NUM> or a value associated with the pixel pitch.

The body module <NUM> for the image sensor <NUM> also has the same hardware configuration as the body module <NUM> for the image sensor <NUM>. However, the storage <NUM> in the body module <NUM> for the image sensor <NUM> includes no wait period table <NUM>. The controller <NUM> in the body module <NUM> for the image sensor <NUM> (hereafter, a second controller <NUM>) is a modification of the controller <NUM> in the body module <NUM> for the image sensor <NUM> to perform an adjustment process shown in <FIG> instead of the adjustment process (<FIG>) described above.

More specifically, the second controller <NUM> starts the adjustment process (<FIG>) when the image sensor <NUM> is turned on. The second controller <NUM> starting the adjustment process first reads blur circle diameter information from the nonvolatile memory <NUM> and pixel pitch information from the nonvolatile memory <NUM> in the modules attached to the body module <NUM> (step S301). The second controller <NUM> then calculates, using the read two items of information, the elapsed period that allows the blur circle diameter to match the pixel pitch (hereafter, a target pixel pitch) indicated by the pixel pitch information (step S302). The process in step S302 is performed in accordance with the structure (data structure) of the blur circle diameter information. More specifically, the blur circle diameter information may include coefficients in an approximation function that approximates a curve indicating the relationship between the blur circle diameter and the elapsed period. In this case, step S302 includes calculating the elapsed period by substituting the target pixel pitch into the approximation function using values included in the blur circle diameter information as coefficients. The blur circle diameter information may include the coordinates (the blur circle diameter and the elapsed period) of multiple points on a curve indicating the relationship between the blur circle diameter and the elapsed period. In this case, the elapsed period is calculated by interpolation.

After step S302, the second controller <NUM> stores the calculated elapsed period as a preset wait period (step S303) and ends the adjustment process.

As described above, the image sensor <NUM> according to the present embodiment can also adjust the preset wait period to the period taken for the optical system <NUM> to have a blur circle diameter matching the pixel pitch of the imaging device <NUM> after a change in the application voltage to the liquid lens <NUM>. The period is the shortest to produce image data in focus. The image sensor <NUM> according to the present embodiment can thus adjust the preset wait period as appropriate for the combination of the lens module <NUM> and the imaging module <NUM> attached to the body module <NUM>.

The image sensor <NUM> is to update the wait period table <NUM> in the body module <NUM> for incorporating one or both of a lens module <NUM> and an imaging module <NUM> in a new format. The image sensor <NUM> according to the present embodiment can incorporate one or both of a lens module <NUM> and an imaging module <NUM> in a new format without changing the body module <NUM>.

The image sensor according to the above embodiments may be modified variously. For example, although the image sensor <NUM> or <NUM> according to the above embodiments includes the imaging device <NUM> as a monochrome image sensing device, the imaging device <NUM> may be a color image sensing device. For the imaging device <NUM> being a single-chip color image sensing device with a Bayer array (RGB), the pixel pitch may be determined by multiplying the actual pixel pitch by a predetermined coefficient A (A > <NUM>). The image sensor <NUM> may be modified to allow the user to designate the accuracy of image data. The image sensor <NUM> may be modified to such an image sensor by, for example, modifying step S302 to calculate, using the read two items of information, the elapsed period that allows the blur circle diameter to match the dimension corresponding to the accuracy designated by the user.

Each lens module <NUM> may incorporate a range sensor for detecting the distance from the inspection target. In the refractive power control process, the controller <NUM> in the image sensor <NUM> or <NUM> may be modified to determine the application voltage to the liquid lens <NUM> based on the distance detected by the range sensor and apply the determined voltage to the liquid lens <NUM>. In the refractive power control process, the controller <NUM> in the image sensor <NUM> or <NUM> may be modified to determine the application voltage to the liquid lens <NUM> based on the distance detected by a range sensor, which is located near the production line for the inspection target separately from the image sensor <NUM> or <NUM>, and apply the determined voltage to the liquid lens <NUM>.

The wait period table <NUM> (<FIG>) may be stored in an external device, such as the information processor <NUM>. The controller <NUM> in the image sensor <NUM> may be modified to transmit, to the external device, a predetermined request including LM format information read from the nonvolatile memory <NUM> and CM format information read from the nonvolatile memory <NUM>. The controller <NUM> may then obtain the wait period from the external device. The controller <NUM> in the image sensor <NUM> may be modified to transmit, to an external device, a predetermined request including information read from the nonvolatile memories <NUM> and <NUM> and format information to request the external device to calculate the wait period (elapsed period).

The refractive power control process may be modified to apply the voltage designated by the information processor <NUM> (or another device) to the liquid lens <NUM>. The refractive power control process may be modified to determine the voltage to be applied to the liquid lens <NUM> based on the refractive power designated by the information processor <NUM> (or another device) and apply the determined voltage to the liquid lens <NUM>. The controller <NUM> in the image sensor <NUM> or <NUM> may be modified to receive a measurement instruction that designates the installation distance, the application voltage, or the refractive power. In response to the instruction, the controller <NUM> may determine whether the application voltage is to be changed. When the application voltage is to be changed, the controller <NUM> may perform the refractive power control process and wait for the preset wait period to elapse before performing steps S102 and S103. The controller <NUM> in the image sensor <NUM> or <NUM> may be modified to, in response to a measurement instruction that designates the installation distance, the application voltage, or the refractive power, perform the refractive power control process and wait for the preset wait period to elapse before performing steps S102 and S103. In response to a measurement instruction that does not designate the installation distance, the application voltage, or the refractive power, the controller <NUM> may immediately perform steps S102 and S103.

The controller <NUM> in the image sensor <NUM> or <NUM> may be modified to constantly capture image data and transmit the captured image data to the information processor <NUM>. Each lens module <NUM> may be modified to include a light source (e.g., a light-emitting diode or an LED) for illuminating a subject. The body module <NUM> may be modified to additionally receive an illumination module <NUM> including a light source for illuminating a subject as schematically shown in <FIG>.

Claim 1:
An image sensor (<NUM>; <NUM>), comprising:
a body module (<NUM>) to which a lens module (<NUM>) and an imaging module (<NUM>) are attachable, the body module (<NUM>) including
a refractive power controller (<NUM>) configured to adjust an application voltage applicable to a liquid lens (<NUM>) included in the lens module (<NUM>) to control a refractive power of the liquid lens (<NUM>), and
a recognition processor (<NUM>) configured to analyze image data from an imaging device (<NUM>) included in the imaging module (<NUM>) to recognize predetermined information about a subject,
characterized in that
the recognition processor (<NUM>) is configured to analyze the image data after a preset period elapses from when the refractive power controller (<NUM>) changes the application voltage to the liquid lens (<NUM>), and
the body module (<NUM>) further includes an adjuster (<NUM>) configured to adjust the preset period.
wherein the adjuster (<NUM>) is configured to read blur circle diameter information indicating a change in a blur circle diameter with time after the application voltage is applied to the liquid lens (<NUM>) from a memory (<NUM>) included in the lens module (<NUM>) and pixel pitch information indicating a pixel pitch (P) of the imaging device (<NUM>) from a memory (<NUM>) included in the imaging module (<NUM>) and adjust, based on the read blue circle diameter information and the read pixel pitch information, the preset period to a period corresponding to a combination of the blur circle diameter information of the lens module (<NUM>) and the pixel pitch information of the imaging device (<NUM>).