Patent Description:
In the production line of a factory, a system called image sensor is frequently used for automating inspection and management of products and saving labors. <CIT> discloses a system having a configuration where a camera and an image processing device are connected with each other with a cable. However, recently, a processing-integrated image sensor has also been launched and the image sensor integrates the camera and the image processing device together so as to capture images and process the captured images with a single device. Such a processing-integrated image sensor is also called a "smart camera," which may also be integrated with an integrating illumination device and lenses.

In order to carry out stable inspection with the image sensor, it is desirable to optimize the model (type), specification and performance of the illuminations, lenses, and imaging elements according to an imaging environment, an object to be inspected, the purposes of the inspection, etc. Therefore, manufactures who provide smart cameras have lined up a large number of products such as illuminations, lenses, imaging elements, etc., with the different models (types), specifications and performances changed little by little, for the user to select the one with the optimum specification.

However, as the IoT development of factories is accelerating, the range of application of smart cameras has been expanded, and it is becoming difficult to provide product variations that cover the diverse user needs. Further, in order to be distinguished from the competitors in product competition, more mass customization and seasonal products are offered to meet the preference of each customer and the life cycle of the products of digital devices typified by smart phones is shortening, and there is an increasing need for partially changing the illuminations and lenses, etc. to be optimized for the inspection as the change of the object to be inspected enters a short cycle. Thus, in the recent years, the so-called a modular structure smart camera has been introduced, which respectively modularizes the illumination elements, lenses, and image capturing elements, and, from the user side, the user may freely combine the illumination elements, lenses, and image capturing elements as desired. Such a smart camera includes an illumination module, a lens module, an imaging element module, etc. The illumination module has a light emitting element such as an LED (Light Emitting Diode) and illuminates an object such as a product with use of the light emission of the light emitting element. The light quantity of the illumination module deteriorates with the number of years of use. Therefore, if the illumination module is continued to be used, the brightness of the illumination module may change even if the illumination module is used with the same setting value as that at the start of use.

In the conventional smart camera, it is possible to monitor the deterioration of the whole system including the illumination module, the lens module, the imaging element module, etc., but it is not possible to monitor the deterioration of each module individually. In addition, although it is conceivable to monitor the light quantity of the illumination module with use of a PD (Photodiode), since it is necessary to install the PD on the substrate of the illumination module, problems such as cost increase and expansion of the substrate area occur.

<CIT> discloses an image sensor system embodied as a lighting system for sports that is illuminated by a plurality of high-power LED-light sources. The flood-lighting can be operated in two different modes, namely a TV-broadcasting mode (with much higher operating power) and a "training" mode used during standard sports training that can cope with much lower lighting levels. Performance data about the LED light sources is evaluated relative to whether or not the light sources will have a predicted life expectancy on the order of the entire lighting system life expectancy. Determining the broadcast power level is obtained by reading manufacturers specifications or by testing, etc. to estimate hours of operation versus power levels.

<CIT> discloses a light source device for an endoscope and a method for using an endoscope system having a light source device.

<CIT> discloses a photoelectric sensor device for predicting the remaining life of a light emitting element, based on correlation data. <CIT> discloses method and devices for health monitoring of lights.

It is an object of the present invention to provide an efficient technique for monitoring deterioration of an illumination module individually in an image sensor including an illumination module and a processing part.

This problem is solved by an image sensor system as claimed by claim <NUM>. Further advantageous embodiments are the subject-matter of the dependent claims.

An image sensor system of the present invention includes an image sensor and a prediction part. The image sensor includes an illumination module and a processing part. The illumination module includes a light emitting part for emitting light and a nonvolatile memory. The processing part stores a first data and a second data in the memory. The first data includes an electric parameter value when the light emitting part is applied with a power and a cumulative power-on time obtained by accumulating a time when the light emitting part is applied with the power. The prediction part obtains the first data or the second data from the nonvolatile memory and predicts a lifetime of the light emitting part based on the first data or the second data. According to the present invention the illumination module is configured for illuminating an object within a field of view of the image sensor with the light emitted from the light emitting part, wherein the second data included in the illumination module includes a value obtained by multiplying the electric parameter value when the light emitting part is applied with the power by the cumulative power-on time, and wherein the prediction part obtains the first data or the second data from the nonvolatile memory and predicts the lifetime of the light emitting part based on the first data or the second data in response to a request received from an external device.

According to this configuration, the electric parameter value when the light emitting part is applied with the power and the cumulative power-on time by accumulating the time when the light emitting part is applied with the power can be obtained from the nonvolatile memory of the illumination module and the lifetime of the light emitting part can be predicted based on the electric parameter value and the cumulative power-on time.

According to this configuration, the value obtained by multiplying the electric parameter value when the light emitting part is applied with the power by the cumulative power-on time can be obtained from the nonvolatile memory of the illumination module, the lifetime of the light emitting part can be predicted based on the value obtained by multiplying the electric parameter value when the light emitting part is applied with the power by the cumulative power-on time.

In this way, the user can recognize the lifetime of the light emitting part and consider the inspection and replacement time of the illumination module based on the prediction result of the lifetime of the light emitting part. Since the user can consider the inspection and replacement time of the illumination module, the user can perform preventive maintenance to prevent beforehand occurrence of a malfunction of the illumination module. Accordingly, the image sensor system capable of monitoring the deterioration of the illumination module is provided.

In the image sensor system, the light emitting part is operable with a plurality of current values, and the processing part may store a third data and a fourth data in the nonvolatile memory according to the current values. The third data includes the plurality of electric parameter values when the light emitting part is operated with the current values and applied with the power and the cumulative power-on time for each of the plurality of electric parameter values. The fourth data includes values obtained by respectively multiplying the plurality of electric parameter values when the light emitting part is operated with the current values and applied with the power by the cumulative power-on time for each of the plurality of electric parameter values. The prediction part may obtain the third data or the fourth data from the nonvolatile memory and predict the lifetime of the light emitting part based on the third data or the fourth data.

According to this configuration, the prediction part can predict the lifetime of the light emitting part based on the plurality of electric parameter values when the light emitting part is operated with the current values and applied with the power and the cumulative power-on time for each of the plurality of electric parameter values.

According to this configuration, the prediction part can predict the lifetime of the light emitting part based on the values obtained by respectively multiplying the plurality of electric parameter values when the light emitting part is operated with the current values and applied with the power by the cumulative power-on time for each of the plurality of electric parameter values. In this way, the prediction accuracy of the lifetime of the light emitting part is improved, and the user can recognize the lifetime of the light emitting part more accurately.

In the image sensor system, the processing part may store a fifth data and a sixth data in the nonvolatile memory. The fifth data includes the electric parameter value greater than or equal to a first threshold value and the cumulative power-on time for the electric parameter value greater than or equal to the first threshold value. The sixth data includes a value obtained by multiplying the electric parameter value greater than or equal to a second threshold value by the cumulative power-on time for the electric parameter value greater than or equal to a second threshold value. The prediction part may obtain the fifth data or the sixth data from the nonvolatile memory and predict the lifetime of the light emitting part based on the fifth data or the sixth data.

According to this configuration, the electric parameter value greater than or equal to the first threshold value and the cumulative power-on time for the electric parameter value greater than or equal to the first threshold value are stored in the nonvolatile memory. According to this configuration, the value obtained by multiplying the electric parameter value greater than or equal to the second threshold value by the cumulative power-on time for the electric parameter value greater than or equal to the second threshold value is stored in the nonvolatile memory. In this way, the number of times of writing to the nonvolatile memory can be reduced, and the period before reaching the limit of the number of times of writing to the nonvolatile memory can be extended.

In the image sensor system, the processing part may store the first data or the second data in the nonvolatile memory at a predetermined time interval. In the image sensor system, the processing part may store the third data or the fourth data in the nonvolatile memory at a predetermined time interval. In the image sensor system, the processing part may store the fifth data or the sixth data in the nonvolatile memory at a predetermined time interval. According to this configuration, since the data is stored in the nonvolatile memory at a predetermined time interval, the number of times of writing to the nonvolatile memory can be reduced, and the period before reaching the limit of the number of times of writing to the nonvolatile memory can be extended.

According to the present invention, a technique for monitoring the deterioration of the illumination module with use of a nonvolatile memory included in the illumination module can be provided.

First, an example of the scene to which the disclosure is applied will be described. <FIG> is a block diagram schematically showing a configuration of an image sensor system. The image sensor system includes an image sensor <NUM> and an external device <NUM>. The image sensor <NUM> includes an illumination part <NUM>, a lens part <NUM> and an imaging part <NUM> as an imaging system. The illumination part <NUM> has a light emitting part <NUM> and a nonvolatile memory <NUM>. The light emitting part <NUM> has at least one light emitting element that emits light upon receiving a power from a power supply (not shown). The light emitting element is, for example, an LED. The illumination part <NUM> is a device for illuminating a subject (an object to be inspected, etc.) within a field of view of the image sensor <NUM> with use of the light emitted from the light emitting part <NUM>. Further, the image sensor <NUM> includes a main body memory <NUM>, a processing part <NUM> and an input/output I/F <NUM> as a processing system. The processing part <NUM> is an example of a control part. The processing part <NUM> controls an imaging system and performs data transmission/reception and the like with an external device <NUM> via the input/output I/F <NUM>. The processing part <NUM> stores a first data in the main body memory <NUM>. The first data includes an electric parameter value when the light emitting part <NUM> is applied with the power and a cumulative power-on time by accumulating the time when the light emitting part <NUM> is applied with the power. The electric parameter value includes at least one of a current value, a voltage value, a VF (forward voltage) value, and a resistance value. The processing part <NUM> stores a second data in the main body memory <NUM>. The second data includes a value obtained by multiplying the electric parameter value when the light emitting part <NUM> is applied with the power by the cumulative power-on time. The processing part <NUM> stores the first data or the second data in the memory <NUM> at any timing or at a predetermined time interval. The processing part <NUM> obtains the first data or the second data from the memory <NUM> at any timing. The processing part <NUM> may obtain the first data or the second data from the memory <NUM> according to a request from the external device <NUM>.

The processing part <NUM> transmits the first data or the second data obtained from the memory <NUM> to the external device <NUM> at any timing. The external device <NUM> receives the first data or the second data and predicts the lifetime of the light emitting part <NUM> based on the received first data or the received second data. Further, the processing part <NUM> may predict the lifetime of the light emitting part <NUM> based on the first data or the second data obtained from the memory <NUM>. The prediction result of the lifetime of the light emitting part <NUM> is displayed on a display device of the external device <NUM>. Therefore, the electric parameter value and the cumulative power-on time are stored in the memory <NUM> of the illumination part <NUM>, and the lifetime of the light emitting part <NUM> is predicted based on the electric parameter value and the cumulative power-on time, and in this way, a user can consider the inspection and replacement time of the illumination part <NUM>. Further, the value obtained by multiplying the electric parameter value by the cumulative power-on time is stored in the memory <NUM> of the illumination part <NUM>, and the lifetime of the light emitting part <NUM> is predicted based on the value obtained by multiplying the electric parameter value by the cumulative power-on time, and in this way, the user can consider the inspection and replacement time of the illumination part <NUM>. In this way, it is possible to monitor the deterioration of the illumination module (the illumination part <NUM>) which is one of a plurality of modules included in the image sensor <NUM>.

The image sensor system according to the first embodiment of the disclosure will be described with reference to <FIG>, <FIG>. <FIG> is a block diagram schematically showing the configuration of the image sensor system. <FIG> is a perspective view schematically showing the appearance of the image sensor, and <FIG> is a perspective view schematically showing a state where the image sensor is disassembled.

The image sensor <NUM> is a device installed in a production line of a factory, for example, and used for various processes using images. The image sensor <NUM> is also called a vision sensor or a vision system. The image sensor <NUM> of the first embodiment is a processing-integrated image sensor (the so-called smart camera) integrally formed by integrating an imaging system and a processing system.

The image sensor <NUM> includes the illumination part <NUM>, the lens part <NUM>, and the imaging part <NUM> as the imaging system. The illumination part <NUM> is a device for illuminating a subject (an object to be inspected, etc.) within a field of view of the image sensor <NUM> and is composed of a plurality of light emitting elements (LEDs, etc.) arranged around the lens part <NUM>, for example. The lens part <NUM> is an optical system for forming an optical image of the subject on the imaging part <NUM>. For example, an optical system having functions such as focus adjustment, aperture, zoom, etc. is used. The imaging part <NUM> is a device for generating and outputting image data through photoelectric conversion and is composed of imaging elements such as CCD and CMOS, for example.

The image sensor <NUM> includes the main body memory <NUM>, the processing part <NUM> and the input/output I/F <NUM> as the processing system. The processing part <NUM> is a device for performing image processing (for example, preprocessing, feature amount extraction, etc.) on the image data taken from the imaging system, various processes based on the result of the image processing (for example, inspection, character recognition, individual identification, etc.), data transmission/reception with the external device <NUM> via the input/output I/F <NUM>, generation of data to be outputted to the external device <NUM>, processing of data received from the external device <NUM>, control of the imaging system and the input/output I/F <NUM>, etc. The processing part <NUM> stores current values when the light emitting part <NUM> is applied with the power and the cumulative power-on time in the main body memory <NUM> and the memory <NUM>. In the following, an example in which the processing part <NUM> stores the current values (an example of the electric parameter value) in the main body memory <NUM> and the memory <NUM> will be described, but the disclosure is not limited to this example, and the processing part <NUM> may store a voltage value, a VF value, and a resistance value in the main body memory <NUM> and the memory <NUM>. The current values when the light emitting part <NUM> is applied with the power (hereinafter simply referred to as "the current values") and the cumulative power-on time are temporarily stored in the main body memory <NUM>. The main body memory <NUM> may be a volatile memory or a nonvolatile memory. The main body memory <NUM> may have any data capacity.

The processing part <NUM> is composed of a processor and a memory, for example, and the processor reads and executes programs stored in the memory so as to realize the various processes described above. A part or all of the functions of the processing part <NUM> may be realized by an ASIC, an FPGA, etc. or may be provided by an external device. The input/output I/F <NUM> is a communication interface for transmitting/receiving data to/from the external device <NUM>. For example, the input/output I/F <NUM> includes a network interface for connecting to a PLC (Programmable Logic Controller, PLC) or a management terminal (computer), a parallel interface for connecting to another sensor or controller, etc..

The image sensor <NUM> of the first embodiment has a modular structure in which, as shown in <FIG>, the three modules, i.e., the illumination part <NUM>, the lens part <NUM>, and the imaging part <NUM>, are selected and attached to a sensor body <NUM>. Each of the modules is fixed to the sensor body <NUM> by screw fastening, for example, and the user can attach/detach the module freely.

Multiple types of modules are prepared to serve as the illumination part (illumination module) <NUM>, such as modules with illumination lights of different wavelengths like white illumination/red illumination/infrared light illumination, and modules with different light emitting element arrangements, light quantities, or light emitting patterns. Further, multiple types of light sources (LEDs, etc.) such as red, blue, green, and infrared light sources are provided in one module, and an illumination module that is capable of irradiating lights of wavelengths (for example, white, violet, pink, etc.) other than red, blue, green, and infrared through control of light emission of each light source may also be used. This type of illumination is called multi-color illumination, etc. Multiple types of modules are prepared to serve as the lens part (lens module) <NUM>, such as a module having a function of adjusting the focus manually or automatically with use of an actuator, etc., a module with different fields of view such as a narrow field of view/a wide field of view, and a module having a zoom function. Multiple types of modules are prepared to serve as the imaging part (imaging module) <NUM>, such as modules having different numbers of pixels, frame rates, and shutter types (rolling shutter/global shutter). The user can combine proper modules as appropriate according to the purpose or the required specification of the image sensor <NUM>.

The illumination part <NUM> has the light emitting part <NUM> for emitting light. The light emitting part <NUM> has at least one light emitting element that emits light upon receiving power supply from the power supply (not shown). The light emitting element is, for example, an LED. The current values applied to the light emitting part <NUM> are variable, and the light emitting part <NUM> is operable with a plurality of current values. The brightness of the illumination part <NUM> varies according to the current values flowing through the light emitting part <NUM>. Power supply control (current value switching, power on/off, etc.) for the light emitting part <NUM> is performed by the processing part <NUM>. The processing part <NUM> stores the cumulative power-on time for each current value applied to the light emitting part <NUM> in the main body memory <NUM>. That is, the processing part <NUM> stores in the main body memory <NUM> the current values before change, the current values after change, the cumulative power-on time for the current values before change and the cumulative power-on time for the current values after change. For example, the processing part <NUM> may store in the main body memory <NUM> a table <NUM> as shown in <FIG> is a diagram showing an example of the table <NUM> stored in the main body memory <NUM>. The plurality of current values is stored in the left field of the table <NUM>. As shown in <FIG>, each of the plurality of current values may be a value within a certain range. Further, each of the plurality of current values may be a certain value. The cumulative power-on time for each of the plurality of current values is stored in the right field of the table <NUM>. For example, the cumulative power-on time at factory shipment is set to <NUM>, and the plurality of current values and the cumulative power-on time for each of the plurality of current values are counted up after power is supplied to the illumination part <NUM>.

The processing part <NUM> stores the data including the plurality of current values and the cumulative power-on time for each of the plurality of current values (hereinafter simply referred to as "the plurality periods of cumulative power-on time") in the memory <NUM>. In this case, the processing part <NUM> writes the data temporarily stored in the main body memory <NUM> to the memory <NUM>. For example, when the table <NUM> of <FIG> is stored in the main body memory <NUM>, the processing part <NUM> writes the table <NUM> of <FIG> to the memory <NUM>. The processing part <NUM> may erase the data of the main body memory <NUM> when the writing to the memory <NUM> is completed. For example, EPROM (Electrically Programmable Read Only Memory, EPROM), EEPROM (Electrically Erasable Programmable Read Only Memory, EEPROM), FeRAM (Ferroelectric Random Access Memory , FeRAM), MRAM (Magnetoresistive Random-Access Memory, MRAM), etc. can be used as the memory <NUM>, and the memory <NUM> may have any data capacity.

The processing part <NUM> may store the plurality of current values and the plurality periods of cumulative power-on time in the memory <NUM> at a predetermined time interval. Specifically, the processing part <NUM> may store the plurality of current values per unit time and the plurality periods of cumulative power-on time per unit time in the memory <NUM> at a predetermined time interval. For example, the processing part <NUM> may store the plurality of current values per hour and the plurality periods of cumulative power-on time per hour in the memory <NUM> hourly. Since the memory <NUM> has a limit of the number of times of writing (lifetime), by storing the data in the memory <NUM> at a predetermined time interval, the number of times of writing to the memory <NUM> can be reduced. In this way, the period before reaching the limit of the number of times of writing to the memory <NUM> can be extended. For example, the limit of the number of times of writing to the EEPROM is about <NUM>,<NUM> times. When data is stored in the EEPROM hourly, the time before reaching the limit of the number of times of writing to the EEPROM is about seven years. Further, the processing part <NUM> may store the plurality of current values and the plurality periods of cumulative power-on time in the memory <NUM> at any timing.

The processing part <NUM> may store the current values greater than or equal to a threshold value and the cumulative power-on time for the current values greater than or equal to the threshold value in the memory <NUM>. Depending on the value of the current values flowing through the light emitting part <NUM>, the current values in some cases do not affect the deterioration of the light emitting part <NUM>. By not storing in the memory <NUM> the current values that do not affect the deterioration of the light emitting part <NUM> and the cumulative power-on time of the current values that do not affect the deterioration of the light emitting part <NUM>, the number of times of writing to the memory <NUM> can be reduced. Further, the amount of data written in the memory <NUM> can be reduced. When the plurality of current values and the plurality periods of cumulative power-on time are stored in the main body memory <NUM>, the processing part <NUM> may extract from the main body memory <NUM> the current values greater than or equal to the threshold value from the plurality of current values and the cumulative power-on time for the current values greater than or equal to the threshold value and store the extracted data in the memory <NUM>. The processing part <NUM> may store in the memory <NUM> the average value of the plurality of current values applied to the light emitting part <NUM> and the total value of the cumulative power-on time for each of the plurality of current values. In this way, the amount of data written in the memory <NUM> can be reduced.

The processing part <NUM> obtains the data stored in the memory <NUM> from the memory <NUM> at any timing. For example, the processing part <NUM> obtains the data including the plurality of current values and the plurality periods of cumulative power-on time from the memory <NUM> at any timing. The processing part <NUM> may obtain the data stored in the memory <NUM> from the memory <NUM> according to a request from the external device <NUM>. For example, the processing part <NUM> may obtain the data including the plurality of current values and the plurality periods of cumulative power-on time from the memory <NUM> according to a request from the external device <NUM>. The user can operate the external device <NUM> at a desired timing to retrieve the data including the plurality of current values and the plurality periods of cumulative power-on time from the memory <NUM>. The processing part <NUM> transmits the data obtained from the memory <NUM> to the external device <NUM>. For example, the processing part <NUM> transmits the data including the plurality of current values and the plurality periods of cumulative power-on time to the external device <NUM>. Further, the processing part <NUM> may transmit the data stored in the main body memory <NUM> to the external device <NUM> according to a request from the external device <NUM>.

The external device <NUM> includes a processing device such as a personal computer (PC), an external storage device such as a database (DB), a communication interface, a display device, etc. The processing device is composed of a processor such as a CPU (Central Processing Unit), a memory, etc. The external device <NUM> receives the data stored in the memory <NUM> from the image sensor <NUM>. For example, the external device <NUM> receives the data including the plurality of current values and the plurality periods of cumulative power-on time from the image sensor <NUM>. The processing device of the external device <NUM> predicts the lifetime of the light emitting part <NUM> based on the data stored in the memory <NUM>. For example, the processing device of the external device <NUM> predicts the lifetime of the light emitting part <NUM> based on the data including the plurality of current values and the plurality periods of cumulative power-on time. The processing device of the external device <NUM> may predict the lifetime of the light emitting part <NUM> by calculating the lifetime of the light emitting part <NUM> with use of a known lifetime calculation formula. The processing device of the external device <NUM> is an example of a prediction part. The processing device of the external device <NUM> may obtain other data used for predicting the lifetime of the light emitting part <NUM> from the database (DB). With the prediction result of the lifetime of the light emitting part <NUM> displayed on the display device of the external device <NUM>, the user can recognize the lifetime of the light emitting part <NUM>. Based on the prediction result of the lifetime of the light emitting part <NUM>, the user can consider the inspection and replacement time of the illumination part <NUM>. In this way, according to the first embodiment, the illumination part <NUM> can be monitored.

For example, after a certain period of time has elapsed since the start of the power supply to the illumination part <NUM>, the illumination part <NUM> may be detached from the image sensor <NUM>, and the detached illumination part <NUM> may be attached to another image sensor <NUM>. The plurality of current values and the plurality periods of cumulative power-on time counted after the start of the power supply to the illumination part <NUM> are stored in the memory <NUM>. Therefore, the external device <NUM> can obtain from the memory <NUM> the plurality of current values and the plurality periods of cumulative power-on time during the period from the start of the power supply to the illumination part <NUM> to the present. In this way, in the image sensor <NUM> of the first embodiment, even when the module is reassembled after the start of the power supply to the illumination part <NUM>, the external device <NUM> can predict the lifetime of the light emitting part <NUM>.

The processing device of the external device <NUM> may determine the state of the light emitting part <NUM> based on the data stored in the memory <NUM>. For example, the processing device of the external device <NUM> may determine the state of the light emitting part <NUM> based on the data including the plurality of current values and the plurality periods of cumulative power-on time. The state of the light emitting part <NUM> includes, for example, decrease of light quantity from a predetermined state of the light emitting part <NUM>. The processing device of the external device <NUM> may determine the state of the light emitting part <NUM> by calculating the light output of the light emitting part <NUM> with use of a known deterioration calculation formula. The processing device of the external device <NUM> may obtain other data used for determining the state of the light emitting part <NUM> from the database (DB). With the determination result of the state of the light emitting part <NUM> displayed on the display device of the external device <NUM>, the user can recognize the state of the light emitting part <NUM>. Based on the determination result of the state of the light emitting part <NUM>, the user can consider the inspection and replacement time of the illumination part <NUM>. In this way, according to the first embodiment, the illumination part <NUM> can be monitored.

The above shows an example in which the processing device of the external device <NUM> predicts the lifetime of the light emitting part <NUM> based on the current values and the cumulative power-on time. The disclosure is not limited to this example, and the processing device of the external device <NUM> may calculate the current values from voltage values, VF values and resistance values and predict the lifetime of the light emitting part <NUM> based on the current values and the cumulative power-on time. In other words, the processing device of the external device <NUM> predicts the lifetime of the light emitting part <NUM> based on the electric parameter value and the cumulative power-on time. The processing device of the external device <NUM> may predict the lifetime of the light emitting part <NUM> based on a plurality of electric parameter values and the plurality periods of cumulative power-on time. Further, the processing device of the external device <NUM> may determine the state of the light emitting part <NUM> based on the plurality of electric parameter values and the plurality periods of cumulative power-on time.

The prediction of the lifetime of the light emitting part <NUM> is performed by the following method. The light output (P) of an LED decreases exponentially with respect to the cumulative power-on time (operation time) and is calculated by the following [Expression <NUM>]. [Expression <NUM>] <MAT>.

The deterioration rate (β) of [Expression <NUM>] varies with the material, structure, use conditions, etc. of the element and is calculated by the following [Expression <NUM>]. [Expression <NUM>] <MAT>.

The degradation constant (β<NUM>) depends on the LED, and the activation energy (Ea) depends on the material of the LED. The temperature of the light emitting layer (Tj) can be calculated from the thermal resistance, power consumption, and ambient temperature depending on the LED.

The processing device of the external device <NUM> calculates the light output (P) of the light emitting part <NUM> based on the current value and the cumulative power-on time with use of [Expression <NUM>] and [Expression <NUM>]. The processing device of the external device <NUM> may calculate the light output (P) of the light emitting part <NUM> based on the plurality of current values and the plurality periods of cumulative power-on time with use of [Expression <NUM>] and [Expression <NUM>]. The processing device of the external device <NUM> determines that the light emitting part <NUM> has reached the end of its lifetime when the light output (P) of the light emitting part <NUM> is less than or equal to a reference value. The processing device of the external device <NUM> determines that the state of the light emitting part <NUM> is bad when the light emitting part <NUM> has reached the end of its lifetime. The processing device of the external device <NUM> determines that the light emitting part <NUM> has not reached the end of its lifetime when the light output (P) of the light emitting part <NUM> is greater than the reference value. The processing device of the external device <NUM> determines that the state of the light emitting part <NUM> is good when the light emitting part <NUM> has not reached the end of its lifetime. The processing device of the external device <NUM> calculates the cumulative power-on time (t) when the light output (P) of the light emitting part <NUM> becomes less than or equal to the reference value with use of [Expression <NUM>]. The processing device of the external device <NUM> may set the calculated cumulative power-on time (t) as the lifetime of the light emitting part <NUM>.

The above is an example in which the processing device of the external device <NUM> predicts the lifetime of the light emitting part <NUM> based on the data including the plurality of current values and the plurality periods of cumulative power-on time obtained from the memory <NUM>, but the disclosure is not limited to this example. For example, the processing part <NUM> may predict the lifetime of the light emitting part <NUM> based on the data including the plurality of current values and the plurality periods of cumulative power-on time obtained from the memory <NUM>. In this case, the processing part <NUM> may receive other data used for predicting the lifetime of the light emitting part <NUM> from the external device <NUM>. The processing part <NUM> is an example of the prediction part. The processing part <NUM> transmits the prediction result of the lifetime the light emitting part <NUM> to the external device <NUM>. With the prediction result of the lifetime of the light emitting part <NUM> displayed on the display device of the external device <NUM>, the user can recognize the lifetime of the light emitting part <NUM>. Based on the prediction result of the lifetime of the light emitting part <NUM>, the user can consider the inspection and replacement time of the illumination part <NUM>. In this way, according to the modified example of the first embodiment, the illumination part <NUM> can be monitored. The processing part <NUM> may perform prediction of the lifetime of the light emitting part <NUM> with the above-described method of <Lifetime Prediction>.

Further, the processing part <NUM> may determine the state of the light emitting part <NUM> based on the data including the plurality of current values and the plurality periods of cumulative power-on time obtained from the memory <NUM>. The state of the light emitting part <NUM> includes, for example, decrease of light quantity from a predetermined state of the light emitting part <NUM>. In this case, the processing part <NUM> may receive other data used for determining the state of the light emitting part <NUM> from the external device <NUM>. The processing part <NUM> transmits the determination result of the state the light emitting part <NUM> to the external device <NUM>. With the determination result of the state of the light emitting part <NUM> displayed on the display device of the external device <NUM>, the user can recognize the state of the light emitting part <NUM>. Based on the determination result of the state of the light emitting part <NUM>, the user can consider the inspection and replacement time of the illumination part <NUM>. In this way, according to the modified example of the first embodiment, the illumination part <NUM> can be monitored.

The second embodiment will be described. In the second embodiment, the same components as those in the first embodiment are denoted by the same reference numerals as those in the first embodiment, and description thereof is omitted. In the second embodiment, the processing part <NUM> stores in the main body memory <NUM> the value obtained by multiplying the electric parameter value when the light emitting part <NUM> is applied with the power by the cumulative power-on time. The processing part <NUM> may store in the main body memory <NUM> a value obtained by multiplying the current value when the light emitting part <NUM> is applied with the power by the cumulative power-on time. The value obtained by multiplying the electric parameter value by the cumulative power-on time is temporarily stored in the main body memory <NUM>. The processing part <NUM> stores the value obtained by multiplying the electric parameter value by the cumulative power-on time in the memory <NUM>. Further, the processing part <NUM> stores the values obtained by respectively multiplying the plurality of electric parameter values by the cumulative power-on time for each of the plurality of electric parameter values in the memory <NUM>. In this case, the processing part <NUM> writes the data temporarily stored in the main body memory <NUM> to the memory <NUM>.

The processing part <NUM> may store the electric parameter value greater than or equal to the threshold value and the cumulative power-on time for the electric parameter value greater than or equal to the threshold value in the memory <NUM>. The processing part <NUM> may store the value obtained by multiplying the electric parameter value when the light emitting part <NUM> is applied with the power by the cumulative power-on time in the memory <NUM> at a predetermined time interval. The processing part <NUM> may store the value obtained by multiplying the electric parameter value greater than or equal to the threshold value by the cumulative power-on time for the electric parameter value greater than or equal to the threshold value in the memory <NUM> at a predetermined time interval. As in the first embodiment, the number of times of writing to the memory <NUM> can be reduced. Further, the amount of data written in the memory <NUM> can be reduced. The processing part <NUM> may store in the memory <NUM> a value obtained by multiplying the average value of the plurality of electric parameter values when the light emitting part <NUM> is applied with the power by the total value of the cumulative power-on time. In this way, the amount of data written in the memory <NUM> can be reduced.

The processing part <NUM> obtains the data stored in the memory <NUM> from the memory <NUM> at any timing. For example, the processing part <NUM> obtains from the memory <NUM> the data including the value obtained by multiplying the electric parameter value by the cumulative power-on time at any timing. The processing part <NUM> may obtain the data stored in the memory <NUM> from the memory <NUM> according to a request from the external device <NUM>. For example, the processing part <NUM> may obtain the data including the value obtained by multiplying the electric parameter value by the cumulative power-on time from the memory <NUM> according to a request from the external device <NUM>. The user can operate the external device <NUM> at a desired timing to retrieve the data stored in the memory <NUM> from the memory <NUM>. The processing part <NUM> transmits the data obtained from the memory <NUM> to the external device <NUM>. For example, the processing part <NUM> transmits the data including the value obtained by multiplying the electric parameter value by the cumulative power-on time to the external device <NUM>. Further, the processing part <NUM> may transmit the data stored in the main body memory <NUM> to the external device <NUM> according to a request from the external device <NUM>.

The external device <NUM> receives the data stored in the memory <NUM> from the image sensor <NUM>. For example, the external device <NUM> receives the data including the value obtained by multiplying the electric parameter value by the cumulative power-on time from the image sensor <NUM>. The processing device of the external device <NUM> predicts the lifetime of the light emitting part <NUM> based on the data stored in the memory <NUM>. For example, the processing device of the external device <NUM> predicts the lifetime of the light emitting part <NUM> based on the data including the value obtained by multiplying the electric parameter value by the cumulative power-on time. The processing device of the external device <NUM> may predict the lifetime of the light emitting part <NUM> based on the data including the value obtained by multiplying the current value by the cumulative power-on time. The processing device of the external device <NUM> may predict the lifetime of the light emitting part <NUM> by calculating the lifetime of the light emitting part <NUM> with use of a known lifetime calculation formula. The processing device of the external device <NUM> may obtain other data used for predicting the lifetime of the light emitting part <NUM> from the DB. With the prediction result of the lifetime of the light emitting part <NUM> displayed on the display device of the external device <NUM>, the user can recognize the lifetime of the light emitting part <NUM>. Based on the prediction result of the lifetime of the light emitting part <NUM>, the user can consider the inspection and replacement time of the illumination part <NUM>. In this way, according to the second embodiment, the illumination part <NUM> can be monitored.

As in the first embodiment, in the image sensor <NUM> of the second embodiment, even when the module is reassembled after the start of the power supply to the illumination part <NUM>, the external device <NUM> can predict the lifetime of the light emitting part <NUM>. Further, the processing device of the external device <NUM> may determine the state of the light emitting part <NUM> based on the data stored in the memory <NUM>. For example, the processing device of the external device <NUM> may determine the state of the light emitting part <NUM> based on the data including the value obtained by multiplying the electric parameter value by the cumulative power-on time.

The above is an example in which the processing device of the external device <NUM> predicts the lifetime of the light emitting part <NUM> based on the data obtained from the memory <NUM>, but the disclosure is not limited to this example. The processing part <NUM> may predict the lifetime of the light emitting part <NUM> based on the data obtained from the memory <NUM>. For example, the processing part <NUM> may predict the lifetime of the light emitting part <NUM> based on the data including the value obtained by multiplying the electric parameter value by the cumulative power-on time. Further, the processing part <NUM> may predict the lifetime of the light emitting part <NUM> based on the data including the value obtained by multiplying the current value by the cumulative power-on time. In this case, the processing part <NUM> may receive other data used for predicting the lifetime of the light emitting part <NUM> from the external device <NUM>. The processing part <NUM> transmits the prediction result of the lifetime the light emitting part <NUM> to the external device <NUM>. With the prediction result of the lifetime of the light emitting part <NUM> displayed on the display device of the external device <NUM>, the user can recognize the lifetime of the light emitting part <NUM>. Based on the prediction result of the lifetime of the light emitting part <NUM>, the user can consider the inspection and replacement time of the illumination part <NUM>. In this way, according to the modified example of the second embodiment, the illumination part <NUM> can be monitored. Further, the processing part <NUM> may determine the state of the light emitting part <NUM> based on the data obtained from the memory <NUM>. For example, the processing part <NUM> may determine the state of the light emitting part <NUM> based on the data including the value obtained by multiplying the electric parameter value by the cumulative power-on time.

Claim 1:
An image sensor system, comprising:
an image sensor (<NUM>) comprising:
an illumination module (<NUM>), comprising:
a light emitting part (<NUM>) for emitting light; and
a nonvolatile memory (<NUM>); and
a processing part (<NUM>) configured to:
store a first data and a second data in the nonvolatile memory (<NUM>), wherein the first data comprises an electric parameter value when the light emitting part (<NUM>) is applied with a power and a cumulative power-on time obtained by accumulating a time when the light emitting part (<NUM>) is applied with the power; and
a prediction part (<NUM>, <NUM>) configured to:
obtain the first data or the second data from the nonvolatile memory (<NUM>); and
predict a lifetime of the light emitting part (<NUM>) based on the first data or the second data;
characterized in that the second data stored in the nonvolatile memory (<NUM>) included in the illumination module (<NUM>) comprises a value obtained by multiplying the electric parameter values when the light emitting part (<NUM>) is applied with the power by the cumulative power-on time, wherein
the prediction part (<NUM>, <NUM>) is configured to:
obtain the first data or the second data from the nonvolatile memory (<NUM>); and
predict the lifetime of the light emitting part (<NUM>) based on the first data or the second data in response to a request received from an external device (<NUM>),
wherein the prediction part (<NUM>, <NUM>) is the processing part (<NUM>) or the external device (<NUM>),
wherein the processing part (<NUM>) further configured to:
compute a light output of the light emitting part based on the cumulative power-on time which is a part of the first data; and
determine that the light emitting part has reached the end of the lifetime when the light output of the light emitting part is less than or equal to a reference value,
wherein the light output of the light emitting part is obtained using an Expression <NUM> and an Expression <NUM> as follows: <MAT>
wherein P represents the light output, P0 represents an initial light output, β represents a degradation rate, and t represents the cumulative power-on time from the first data; <MAT>
wherein β0 represents a degradation constant, IF represents a current value that is the electric parameter value in the first data, Ea represents an activation energy [eV], k represents a Boltzmann constant (<NUM>×<NUM>-<NUM>) [ev/K], and Tj represents a temperature of light emitting layer [K].