Patent Description:
An image forming apparatus that draws an image on a screen or the like by irradiating a light scanning device configured with micro electro mechanical systems (MEMS) with light has been known. The light scanning device comprises a movable mirror that swings about two orthogonal axes. For example, the light scanning device performs Lissajous scanning of light of which an intensity is modulated in accordance with an input image. The movable mirror is resonantly driven by two sine waves having different frequencies. In the Lissajous scanning, the movable mirror is resonantly driven about the two axes. Thus, there is an advantage that a deflection angle is large, and a scanning range of the light can be increased.

In drawing the image by the light scanning device, it is necessary to accurately estimate the deflection angle in a case where the movable mirror is swinging, and irradiate the movable mirror with the light having a predetermined intensity in accordance with the estimated deflection angle. In a case where estimation accuracy of the deflection angle of the movable mirror is low, the image drawn by the light scanning device is distorted. For example, the image drawn by the light scanning device is distorted due to a shift in irradiation timing between an advancing path and a returning path of the swing of the movable mirror.

<CIT> discloses a configuration of providing a stress sensor in a light scanning device and estimating a deflection angle of a movable mirror based on a sensor signal of the stress sensor.

In addition, <CIT> discloses a method of measuring a projection distance by projecting and capturing a measurement image and correcting a position of an image to be subjected to calibration by projecting a calibration image based on the measured projection distance. <CIT> discloses a controller for a tiltable MEMS reflector configured to oscillate the reflector about X axis or about X and Y axes, and the obtain information about current and past tilt angles. <CIT> discloses a micromechanical component with a light window; a mirror element which can be adjusted relative to the light window from a first position about at least one axis of rotation into at least a second position; and an optical sensor with a detection surface which is designed to determine a light intensity on the detection surface and to provide a corresponding sensor signal. <CIT> discloses a two-dimensional scanner in which a scanning center position and a scanning amplitude or the like in a comparatively slow speed scanning direction are certainly measured. <CIT> discloses an apparatus and method for evaluating driving characteristics of a scanner by precisely measuring the driving angle of a scanning mirror rotating within a large angle range of about +/- <NUM> degrees.

The deflection angle of the movable mirror can be estimated using a driving signal for driving the movable mirror or an angle sensor configured with a piezoelectric element, in addition to the stress sensor disclosed in <CIT>.

However, a time delay occurs in the driving signal or the sensor signal due to a noise or the like. Thus, it is difficult to accurately estimate the deflection angle of the movable mirror.

In addition, while performing the calibration of associating a position on an image displayed on a display surface with a position on a captured image by projecting the calibration image is disclosed in <CIT>, a projection timing of a projection unit is not corrected.

An object of the disclosed technology is to provide an image forming apparatus and an operation method thereof that can suppress distortion of an image.

In order to accomplish the above object, an image forming apparatus according to claim <NUM> and an operation method of an image forming apparatus according to claim <NUM> is provided. The image forming apparatus comprises a light emitting device that emits light, a movable mirror that reflects the light emitted from the light emitting device, a first actuator that causes the movable mirror to swing about a first axis, a first reference signal output portion that outputs a first reference signal by estimating a point in time when a deflection angle of the movable mirror about the first axis becomes equal to a first reference angle, a light emission controller that causes the light emitting device to emit the light based on the first reference signal output from the first reference signal output portion, an imaging apparatus that images the light reflected by the movable mirror, and a correction portion that corrects a timing of the first reference signal output by the first reference signal output portion based on imaging information acquired by the imaging apparatus.

The image forming apparatus further comprises a second actuator that causes the movable mirror to swing about a second axis, and a second reference signal output portion that outputs a second reference signal by estimating a point in time when a deflection angle of the movable mirror about the second axis becomes equal to a second reference angle, in which the correction portion corrects the timing of the first reference signal output by the first reference signal output portion and a timing of the second reference signal output by the second reference signal output portion based on the imaging information captured by the imaging apparatus.

It is preferable that the first reference angle is an angle at which the deflection angle of the movable mirror about the first axis becomes zero, and the second reference angle is an angle at which the deflection angle of the movable mirror about the second axis becomes zero.

It is preferable that the first reference angle is an angle at which the deflection angle of the movable mirror about the first axis is the maximum or minimum, and the second reference angle is an angle at which the deflection angle of the movable mirror about the second axis is the maximum or minimum.

The first reference signal output portion estimates the point in time when the deflection angle of the movable mirror about the first axis becomes equal to the first reference angle, based on a first driving signal provided to the first actuator, and the second reference signal output portion estimates the point in time when the deflection angle of the movable mirror about the second axis becomes equal to the second reference angle, based on a second driving signal provided to the second actuator.

It is preferable that the image forming apparatus further comprises an optical element that guides a part of the light to the imaging apparatus on an optical path of the light reflected by the movable mirror.

An operation method of an image forming apparatus according to another aspect of the present disclosure is an operation method of an image forming apparatus including a light emitting device that emits light, a movable mirror that reflects the light emitted from the light emitting device, a first actuator that causes the movable mirror to swing about a first axis, a first reference signal output portion that outputs a first reference signal by estimating a point in time when a deflection angle of the movable mirror about the first axis becomes equal to a first reference angle, a light emission controller that causes the light emitting device to emit the light based on the first reference signal output from the first reference signal output portion, and an imaging apparatus that images the light reflected by the movable mirror, the operation method comprising correcting a timing of the first reference signal output by the first reference signal output portion based on imaging information acquired by the imaging apparatus.

According to the disclosed technology, an image forming apparatus and an operation method thereof that can suppress distortion of an image can be provided.

Hereinafter, an embodiment according to the disclosed technology will be described in detail with reference to the drawings. As an example, in the following embodiment, a form of applying the disclosed technology to an image forming apparatus that forms an image on a projection surface by scanning laser light using a Lissajous method will be described.

<FIG> illustrates an example of a configuration of an image forming apparatus <NUM> of the present embodiment. As illustrated in <FIG>, the image forming apparatus <NUM> of the present embodiment comprises a control device <NUM>, a MEMS driver <NUM>, a light emitting device <NUM>, a multiplexing optical system <NUM>, a collimator <NUM>, a MEMS mirror <NUM>, and an imaging apparatus <NUM>. The MEMS mirror <NUM> is an example of a "light scanning device" according to the embodiment of the disclosed technology.

The light emitting device <NUM> includes a laser driver <NUM> and a laser light source <NUM>. The laser driver <NUM> of the present embodiment drives the laser light source <NUM> based on an intensity modulation signal supplied from the control device <NUM> and causes laser light for forming an image to be output from the laser light source <NUM>. For example, the laser light source <NUM> outputs the laser light of three colors of red (R), green (G), and blue (B). The laser light is an example of "light" according to the embodiment of the disclosed technology.

The laser light output from the laser light source <NUM> is multiplexed by the multiplexing optical system <NUM>. Then, the MEMS mirror <NUM> is irradiated with the multiplexed laser light through the collimator <NUM>. The laser light condensed in the MEMS mirror <NUM> is reflected toward a projection surface <NUM> by the MEMS mirror <NUM>. For example, the projection surface <NUM> is a screen for projecting the image, or a retina of an eye of a person. That is, the image forming apparatus <NUM> of the present embodiment is used for a projector, augmented reality (AR) glasses, and the like.

In the present embodiment, the projection surface <NUM> is not limited to a surface of an actual object such as the screen and includes an imaginary plane in a space.

The MEMS driver <NUM> drives the MEMS mirror <NUM> under control of the control device <NUM>. In the MEMS mirror <NUM>, a mirror portion <NUM> (refer to <FIG>) that reflects laser light L swings about each of two axes orthogonal to each other as a central axis. In the present embodiment, the laser light L is scanned in a state of drawing a Lissajous curve on the projection surface <NUM> by the swing of the mirror portion <NUM> based on a driving signal. The Lissajous curve is a curve that is decided by a swing frequency about a first axis, a swing frequency about a second axis, and a phase difference therebetween. The mirror portion <NUM> is an example of a "movable mirror" according to the embodiment of the disclosed technology.

The control device <NUM> of the present embodiment includes a field programmable gate array (FPGA) 20A and a memory 20B. For example, the memory 20B is a volatile memory and stores various information such as an image signal representing the image projected to the projection surface <NUM>. For example, the memory 20B stores the image signal input from an outside of the image forming apparatus <NUM>.

The imaging apparatus <NUM> generates a captured image IP by imaging the projection surface <NUM> irradiated with the laser light L by the MEMS mirror <NUM>, and outputs the generated captured image IP to the control device <NUM>. The imaging apparatus <NUM> is configured with an image sensor such as a charge coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor. In addition, the imaging apparatus <NUM> may be configured with a position sensitive detector (PSD). The captured image IP is used for correcting timings of a first reference signal and a second reference signal, described later. The captured image IP is an example of "imaging information" according to the embodiment of the disclosed technology.

<FIG> illustrates an example of a configuration of the MEMS mirror <NUM>. The MEMS mirror <NUM> includes the mirror portion <NUM>, a first support portion <NUM>, a first movable frame <NUM>, a second support portion <NUM>, a second movable frame <NUM>, a connecting portion <NUM>, and a fixed frame <NUM>.

The mirror portion <NUM> has a reflecting surface 40A on which an incidence ray is reflected. For example, the reflecting surface 40A is formed with a thin metal film of gold (Au), aluminum (Al), silver (Ag), or a silver alloy. For example, a shape of the reflecting surface 40A is a circular shape.

The first support portion <NUM> is arranged outside the mirror portion <NUM> at each of positions that face with a second axis a<NUM> interposed therebetween. The first support portions <NUM> are connected to the mirror portion <NUM> on a first axis a<NUM> and support the mirror portion <NUM> in a swingable manner about the first axis a<NUM>.

The first movable frame <NUM> is a rectangular frame surrounding the mirror portion <NUM> and is connected to the mirror portion <NUM> through the first support portions <NUM> on the first axis a<NUM>. A piezoelectric element <NUM> is formed on the first movable frame <NUM> at each of positions that face with the first axis a<NUM> interposed therebetween. In such a manner, a pair of first actuators <NUM> are configured by forming two piezoelectric elements <NUM> on the first movable frame <NUM>.

The pair of first actuators <NUM> are arranged at positions that face with the first axis a<NUM> interposed therebetween. The first actuators <NUM> cause the mirror portion <NUM> to swing about the first axis a<NUM> by applying rotational torque about the first axis a<NUM> to the mirror portion <NUM>.

The second support portion <NUM> is arranged outside the first movable frame <NUM> at each of positions that face with the first axis a<NUM> interposed therebetween. The second support portions <NUM> are connected to the first movable frame <NUM> on the second axis a<NUM> and support the first movable frame <NUM> and the mirror portion <NUM> in a swingable manner about the second axis a<NUM>. In the present embodiment, the second support portions <NUM> are torsion bars that stretch along the second axis a<NUM>.

The second movable frame <NUM> is a rectangular frame surrounding the first movable frame <NUM> and is connected to the first movable frame <NUM> through the second support portions <NUM> on the second axis a<NUM>. The piezoelectric element <NUM> is formed on the second movable frame <NUM> at each of positions that face with the second axis a<NUM> interposed therebetween. In such a manner, a pair of second actuators <NUM> are configured by forming two piezoelectric elements <NUM> on the second movable frame <NUM>.

The pair of second actuators <NUM> are arranged at positions that face with the second axis a<NUM> interposed therebetween. The second actuators <NUM> cause the mirror portion <NUM> to swing about the second axis a<NUM> by applying rotational torque about the second axis a<NUM> to the mirror portion <NUM> and the first movable frame <NUM>.

The connecting portion <NUM> is arranged outside the second movable frame <NUM> at each of positions that face with the first axis a<NUM> interposed therebetween. The connecting portions <NUM> are connected to the second movable frame <NUM> on the second axis a<NUM>.

The fixed frame <NUM> is a rectangular frame surrounding the second movable frame <NUM> and is connected to the second movable frame <NUM> through the connecting portions <NUM> on the second axis a<NUM>.

In the present embodiment, the first axis a<NUM> and the second axis a<NUM> are orthogonal to each other. In the following description, a direction parallel to the first axis a<NUM> will be referred to as an X direction, a direction parallel to the second axis a<NUM> will be referred to as a Y direction, and a direction orthogonal to the first axis a<NUM> and the second axis a<NUM> will be referred to as a Z direction.

<FIG> are diagrams for describing a deflection angle in a case where the mirror portion <NUM> swings. <FIG> illustrates a deflection angle (hereinafter, referred to as a first deflection angle) θ1 of the mirror portion <NUM> about the first axis a<NUM>. <FIG> illustrates a deflection angle (hereinafter, referred to as a second deflection angle) θ2 of the mirror portion <NUM> about the second axis a<NUM>.

The first deflection angle θ1 is an angle at which a line normal to the reflecting surface 40A is inclined with respect to the Z direction in a YZ plane. The second deflection angle θ2 is an angle at which the line normal to the reflecting surface 40A is inclined with respect to the Z direction in an XZ plane.

<FIG> illustrates an example of a functional configuration of the control device <NUM>. As illustrated in <FIG>, the control device <NUM> is configured with an image input portion <NUM>, an information generation portion <NUM>, an information storage portion <NUM>, a first reference signal output portion 63A, a second reference signal output portion 63B, a light emission controller <NUM>, a correction portion <NUM>, a table holding portion <NUM>, and a humidity and temperature sensor <NUM>. The image input portion <NUM>, the information generation portion <NUM>, the information storage portion <NUM>, the first reference signal output portion 63A, the second reference signal output portion 63B, the light emission controller <NUM>, the correction portion <NUM>, and the table holding portion <NUM> are functional portions implemented by causing the FPGA 20A and the memory 20B to operate in cooperation.

Image data DT that represents the image to be formed is input into the image input portion <NUM> from the outside. Hereinafter, the image corresponding to the image data DT input into the image input portion <NUM> may be referred to as an input image. As an example, in the present embodiment, the image data DT of colors represented by RGB signals is input into the image input portion <NUM>. The image data DT input into the image input portion <NUM> is output to the information generation portion <NUM>. The image data DT input into the image input portion <NUM> is not limited to the present embodiment and may be data corresponding to the image to be formed. For example, the image data DT may be binarized data that represents whether or not to output the laser light L. In addition, for example, the image data DT may be data that represents multiple values of an output amount.

The information generation portion <NUM> generates intensity information SI that represents a correspondence relationship between a scanning position of the laser light L by the MEMS mirror <NUM> and a signal intensity of the input image. In a case where the input image is a color image, the signal intensity represents an intensity of each of the RGB signals. The information storage portion <NUM> stores the intensity information SI generated by the information generation portion <NUM>.

The MEMS driver <NUM> outputs first driving signals V1A and V1B illustrated in <FIG> as an example to the pair of first actuators <NUM> of the MEMS mirror <NUM>, and generates a first zero cross signal ZC1 and outputs the first zero cross signal ZC1 to the first reference signal output portion 63A. The first driving signal V1A and the first driving signal V1B are sine waves having a phase difference of <NUM>°.

The first zero cross signal ZC1 is a pulsed signal that represents points at which the first driving signals V1A and V1B become zero. Signals output to the first reference signal output portion 63A from the MEMS driver <NUM> are not limited to zero cross signals and may be signals having the same periods as the first driving signals V1A and V1B.

The MEMS driver <NUM> outputs second driving signals V2A and V2B illustrated in <FIG> as an example to the pair of second actuators <NUM> of the MEMS mirror <NUM>, and generates a second zero cross signal ZC2 and outputs the second zero cross signal ZC2 to the second reference signal output portion 63B. The second driving signal V2A and the second driving signal V2B are sine waves having a phase difference of <NUM>°.

The second zero cross signal ZC2 is a pulsed signal that represents points at which the second driving signals V2A and V2B become zero. Signals output to the second reference signal output portion 63B from the MEMS driver <NUM> are not limited to zero cross signals and may be signals having the same periods as the second driving signals V2A and V2B.

A frequency (hereinafter, referred to as a first driving frequency) of the first driving signals V1A and V1B and a frequency (hereinafter, referred to as a second driving frequency) of the second driving signals V2A and V2B have different frequency ratios. A frequency ratio of the first driving frequency and the second driving frequency is decided based on a shape of a Lissajous curve of light scanning performed by the MEMS mirror <NUM>.

The first reference signal output portion 63A outputs a first reference signal PS1 to the light emission controller <NUM> by estimating a point in time when the first deflection angle θ1 of the mirror portion <NUM> becomes equal to a first reference angle. In the present embodiment, the first reference angle is set to <NUM>°. The first reference signal output portion 63A generates the first reference signal PS1 by estimating the point in time when the first deflection angle θ1 becomes equal to the first reference angle, based on the first driving signals V1A and V1B.

Specifically, as illustrated in <FIG>, the first reference signal output portion 63A outputs a signal obtained by delaying the first zero cross signal ZC1 input from the MEMS driver <NUM> by a delay time period D1 as the first reference signal PS1. In a case where the mirror portion <NUM> resonates about the first axis ai, the delay time period D1 is ideally a time period corresponding to <NUM>/<NUM> of a period of the first zero cross signal ZC1. However, a shift occurs due to an environmental condition (a temperature, a humidity, and the like). Thus, in the present embodiment, the first reference signal output portion 63A acquires the delay time period D1 based on a temperature and a humidity detected by the humidity and temperature sensor <NUM> and a look-up table (hereinafter, referred to as the LUT) 66A held in the table holding portion <NUM>, and generates the first reference signal PS1 based on the acquired delay time period D1.

The second reference signal output portion 63B outputs a second reference signal PS2 to the light emission controller <NUM> by estimating a point in time when the second deflection angle θ2 of the mirror portion <NUM> becomes equal to a second reference angle. In the present embodiment, the second reference angle is set to <NUM>°. The second reference signal output portion 63B generates the second reference signal PS2 by estimating the point in time when the second deflection angle θ2 becomes equal to the second reference angle, based on the second driving signals V2A and V2B.

Specifically, as illustrated in <FIG>, the second reference signal output portion 63B outputs a signal obtained by delaying the second zero cross signal ZC2 input from the MEMS driver <NUM> by a delay time period D2 as the second reference signal PS2. In a case where the mirror portion <NUM> resonates about the second axis a<NUM>, the delay time period D2 is ideally a time period corresponding to <NUM>/<NUM> of a period of the second zero cross signal ZC2. However, a shift occurs due to an environmental condition (a temperature, a humidity, and the like). Thus, in the present embodiment, the second reference signal output portion 63B acquires the delay time period D2 based on the temperature and the humidity detected by the humidity and temperature sensor <NUM> and the LUT 66A held in the table holding portion <NUM>, and generates the second reference signal PS2 based on the acquired delay time period D2.

As illustrated in <FIG> as an example, a relationship among the delay time period D1, the temperature, and the humidity and a relationship among the delay time period D2, the temperature, and the humidity are recorded in advance in the LUT 66A. For example, these relationships are decided based on a past history.

The light emission controller <NUM> causes the light emitting device <NUM> to emit the laser light L based on the first reference signal PS1 and the second reference signal PS2. In a drawing mode in which the image based on the image data DT is drawn on the projection surface <NUM>, the light emission controller <NUM> reads out the intensity information SI from the information storage portion <NUM> and causes the light emitting device <NUM> to emit the laser light L having an intensity decided based on the first reference signal PS1 and the second reference signal PS2 for each constant time period (for example, for each clock period).

In addition, in the present embodiment, a calibration mode for correcting the timings of the first reference signal PS1 and the second reference signal PS2 output by the first reference signal output portion 63A and the second reference signal output portion 63B, respectively, is performed. In the calibration mode, the light emission controller <NUM> causes the light emitting device <NUM> to emit the laser light L in accordance with the timing at which the first reference signal PS1 is output from the first reference signal output portion 63A, and the timing at which the second reference signal PS2 is output from the second reference signal output portion 63B. That is, in the calibration mode, the light emission controller <NUM> causes the light emitting device <NUM> to emit the laser light L at the point in time when the first deflection angle θ1 is estimated to become equal to the first reference angle (in the present embodiment, <NUM>°), and the point in time when the second deflection angle θ2 is estimated to become equal to the second reference angle (in the present embodiment, <NUM>°).

In the calibration mode, the imaging apparatus <NUM> images a pattern that is drawn on the projection surface <NUM> by reflecting the laser light L emitted from the light emitting device <NUM> by the MEMS mirror <NUM>. The imaging apparatus <NUM> outputs the captured image IP generated by imaging the projection surface <NUM> to the correction portion <NUM>.

The correction portion <NUM> corrects the timings of the first reference signal PS1 and the second reference signal PS2 output by the first reference signal output portion 63A and the second reference signal output portion 63B, respectively, based on a shift amount of the pattern captured in the captured image IP from a predetermined shape.

<FIG> illustrates an example of the pattern drawn on the projection surface <NUM> in the calibration mode. In the present example, the frequency ratio of the first driving frequency and the second driving frequency is set to <NUM>:<NUM> for simplification of description. In the present example, a Lissajous curve <NUM> is drawn on the projection surface <NUM> by the light scanning performed by the MEMS mirror <NUM>.

<FIG> illustrates a case where a shift does not occur between the swing of the mirror portion <NUM> and the timings of the first reference signal PS1 and the second reference signal PS2. In such a manner, in a case where a shift does not occur, the light emitting device <NUM> emits the laser light L at the point in time of θ1 = <NUM> and the point in time of θ2 = <NUM>. Thus, a first reference line L1 and a second reference line L2 on the projection surface <NUM> are irradiated with the laser light L. In the present embodiment, the first reference line L1 is a straight line that passes through a center of the Lissajous curve <NUM> and is parallel to the X direction. The second reference line L2 is a straight line that passes through the center of the Lissajous curve <NUM> and is parallel to the Y direction. Reference numeral P denotes a point (that is, a bright point) irradiated with the laser light L on the projection surface <NUM>.

In a case where the frequency ratio of the first driving frequency and the second driving frequency is set to make the Lissajous curve <NUM> more precise, the bright points P draw straight line patterns along the first reference line L1 and the second reference line L2.

<FIG> illustrates a case where a shift occurs between the swing of the mirror portion <NUM> and the timings of the first reference signal PS1 and the second reference signal PS2. In such a manner, in a case where a shift occurs, the light emitting device <NUM> emits the laser light L at points in time shifted from the point in time of θ1 = <NUM> and the point in time of θ2 = <NUM>. Thus, positions shifted from the first reference line L1 and the second reference line L2 on the projection surface <NUM> are irradiated with the laser light L. In the present example, straight lines L1A and L1B shifted from the first reference line L1 and straight lines L2A and L2B shifted from the second reference line L2 are irradiated with the laser light L.

The straight line L1A is a line that is irradiated with the laser light L on an advancing path of Lissajous scanning. The straight line L1B is a line that is irradiated with the laser light L on a returning path of the Lissajous scanning. In addition, the straight line L2A is a line that is irradiated with the laser light L on the advancing path of the Lissajous scanning. The straight line L2B is a line that is irradiated with the laser light L on the returning path of the Lissajous scanning. The advancing path refers to a path along which the first deflection angle θ1 is increased for the Y direction, and a path along which the second deflection angle θ2 is increased for the X direction. The returning path refers to a path along which the first deflection angle θ1 is decreased for the Y direction, and a path along which the second deflection angle θ2 is decreased for the X direction.

In such a manner, in a case where a shift occurs between the swing of the mirror portion <NUM> and the timings of the first reference signal PS1 and the second reference signal PS2, the first reference line L1 is separated into the straight line L1A and the straight line L1B, and the second reference line L2 is separated into the straight line L2A and the straight line L2B.

In a case where the frequency ratio of the first driving frequency and the second driving frequency is set to make the Lissajous curve <NUM> more precise, the bright points P draw straight line patterns along the straight line L1A, the straight line L1B, the straight line L2A, and the straight line L2B.

<FIG> illustrates an example of a functional configuration of the correction portion <NUM>. The correction portion <NUM> includes a correction amount calculation portion <NUM> and a signal correction portion <NUM>. The captured image IP is input into the correction portion <NUM> from the imaging apparatus <NUM>.

The correction amount calculation portion <NUM> derives a shift amount ΔY between the straight line L1A and the straight line L1B in the Y direction and a shift amount ΔX between the straight line L2A and the straight line L2B in the X direction based on the captured image IP. In addition, the correction amount calculation portion <NUM> derives a correction amount δ1 of the timing of the first reference signal PS1 for ΔX = <NUM> and a correction amount δ2 of the timing of the second reference signal PS2 for ΔY = <NUM> and outputs the derived correction amounts δ1 and δ2 to the signal correction portion <NUM>. For example, the correction amount calculation portion <NUM> derives the correction amounts δ1 and δ2 based on a relationship between the shift amount ΔX and the correction amount δ1 and a relationship between the shift amount ΔY and the correction amount δ2 stored in advance. Each of the relationship between the shift amount ΔX and the correction amount δ1 and the relationship between the shift amount ΔY and the correction amount δ2 is an almost proportional relationship.

The correction amount calculation portion <NUM> derives the shift amounts ΔX and ΔY based on the captured image IP obtained in a state where the timings of the first reference signal PS1 and the second reference signal PS2 are not corrected (that is, a state of <NUM> = <NUM> and δ2 = <NUM>), and derives the correction amounts δ1 and δ2 based on the derived shift amounts ΔX and ΔY.

The signal correction portion <NUM> corrects the timing of the first reference signal PS1 output from the first reference signal output portion 63A based on the correction amount δ1 input from the correction amount calculation portion <NUM>, and corrects the timing of the second reference signal PS2 output from the second reference signal output portion 63B based on the correction amount δ2 input from the correction amount calculation portion <NUM>.

<FIG> illustrate an example of timing correction performed by the signal correction portion <NUM>. <FIG> illustrates an example of correcting the timing of the first reference signal PS1 based on the correction amount <NUM>. <FIG> illustrates an example of correcting the timing of the second reference signal PS2 based on the correction amount δ2. Correcting the timings of the first reference signal PS1 and the second reference signal PS2 results in ΔX = <NUM> and ΔY = <NUM>. Accordingly, the straight line L1A matches the straight line L1B, and the straight line L2A matches the straight line L2B.

The signal correction portion <NUM> may be provided inside each of the first reference signal output portion 63A and the second reference signal output portion 63B. In this case, the first reference signal output portion 63A corrects the timing of the first reference signal PS1 based on the correction amount δ1 input from the correction amount calculation portion <NUM>. Similarly, the second reference signal output portion 63B corrects the timing of the second reference signal PS2 based on the correction amount δ2 input from the correction amount calculation portion <NUM>.

For example, the calibration mode is executed for a predetermined period when the image forming apparatus <NUM> is started. After the calibration mode is finished, the first reference signal output portion 63A and the second reference signal output portion 63B continue the timing correction based on the correction amounts δ1 and δ2 in the drawing mode. The calibration mode may be periodically executed during execution of the drawing mode.

As described above, the image forming apparatus <NUM> can suppress distortion of the image drawn on the projection surface <NUM> by correcting the timings of the first reference signal PS1 and the second reference signal PS2 based on the imaging information acquired by the imaging apparatus <NUM>.

Hereinafter, various modification examples of the embodiment will be described.

In the embodiment, while the first reference line L1 and the second reference line L2 are set to pass through the center of the Lissajous curve <NUM> as illustrated in <FIG>, setting positions of the first reference line L1 and the second reference line L2 are not limited thereto and can be appropriately changed.

As illustrated in <FIG> as an example, the positions of the first reference line L1 and the second reference line L2 may be set in an end part of a drawing region of the laser light L. In the present example, in a case where a shift does not occur between the swing of the mirror portion <NUM> and the timings of the first reference signal PS1 and the second reference signal PS2, the first reference line L1 and the second reference line L2 are irradiated with the laser light L. On the other hand, in a case where a shift occurs, the straight lines L1A and L1B shifted from the first reference line L1 and the straight lines L2A and L2B shifted from the second reference line L2 are irradiated with the laser light L. The correction portion <NUM>, in the same manner as the embodiment, corrects the timings of the first reference signal PS1 and the second reference signal PS2 so that the straight line L1A matches the straight line L1B, and the straight line L2A matches the straight line L2B.

In such a manner, in a case where the positions of the first reference line L1 and the second reference line L2 are set in the end part of the drawing region, a user is unlikely to recognize the first reference line L1 and the second reference line L2. Thus, it is possible to execute calibration without causing the user to feel awkward while displaying the image on the projection surface <NUM>.

In addition, in the embodiment, in the calibration mode, the irradiation with the laser light L is performed on the advancing path and the returning path of the Lissajous scanning. Accordingly, in a case where a shift occurs as described above, the first reference line L1 is separated into the straight line L1A and the straight line L1B, and the second reference line L2 is separated into the straight line L2A and the straight line L2B. Instead, the irradiation with the laser light L may be performed on only one of the advancing path and the returning path of the Lissaj ous scanning.

<FIG> illustrates an example of performing the irradiation with the laser light L on only the advancing path in a case where the positions of the first reference line L1 and the second reference line L2 are set in the end part of the drawing region of the laser light L. In the present example, in a case where a shift occurs between the swing of the mirror portion <NUM> and the timings of the first reference signal PS1 and the second reference signal PS2, the straight line L1A shifted from the first reference line L1 and the straight line L2A shifted from the second reference line L2 are irradiated with the laser light L. In the present example, the correction portion <NUM> may correct the timings of the first reference signal PS1 and the second reference signal PS2 so that the straight line L1A matches the first reference line L1, and the straight line L2A matches the second reference line L2.

In addition, in the embodiment, in the calibration mode, the irradiation with the laser light L is performed a plurality of times in one scanning period of the Lissajous scanning. One scanning period of the Lissajous scanning corresponds to a frame period of the image projected to the projection surface <NUM>. In the calibration mode, the number of times the irradiation with the laser light L is performed in one scanning period of the Lissajous scanning may be once (that is, one pulse).

<FIG> illustrates an example of performing the irradiation with the laser light L only once in one scanning period of the Lissajous scanning. In the present example, in a case where a shift does not occur as described above, a reference point P0 is irradiated with the laser light L. On the other hand, in a case where a shift occurs as described above, a point P1 shifted from the reference point P0 is irradiated with the laser light L. In the present example, the correction portion <NUM> may correct the timings of the first reference signal PS1 and the second reference signal PS2 so that the point P1 matches the reference point P0.

In the example illustrated in <FIG>, while the reference point P0 is set in the end part of the drawing region, the reference point P0 may be at a center position (that is, a position of θ1 = <NUM> and θ2 = <NUM>) of the drawing region. In addition, the reference point P0 may be set as a point at which the Lissajous curve <NUM> intersects. In this case, it is also preferable that the irradiation with the laser light L is performed each time the Lissajous scanning passes through the reference point P0 (that is, the irradiation with the laser light L is performed twice in one scanning period of the Lissajous scanning). In this case, in a case where a shift occurs as described above, the reference point P0 is separated into two points. Thus, the correction portion <NUM> may correct the timings of the first reference signal PS1 and the second reference signal PS2 so that the two separated points match.

In the embodiment, while the projection surface <NUM> such as the screen is irradiated with the laser light L from the MEMS mirror <NUM>, an eyeball or the like of a person may be irradiated with the laser light L from the MEMS mirror <NUM>. In this case, as illustrated in <FIG> as an example, a first optical element <NUM> having a light diffusion function is provided on an optical path of the laser light L reflected by the MEMS mirror <NUM>. The first optical element <NUM> has a function of expanding a so-called eye-box (a range in which a video can be viewed by moving eyes).

The laser light L emitted from the light emitting device <NUM> is incident on the MEMS mirror <NUM>. The laser light L incident on the MEMS mirror <NUM> is emitted toward the first optical element <NUM> by modulating a reflection direction by the swinging mirror portion <NUM>.

The first optical element <NUM> transmits and diffuses the laser light L incident from the MEMS mirror <NUM>. For example, the first optical element <NUM> is a microlens array. The first optical element <NUM> is not limited to a microlens array and may be an optical element such as a frosted glass, a grating, or a hologram or may be a combination thereof.

A second optical element <NUM> for guiding the laser light L to an eyeball E is provided on the optical path of the laser light L transmitted through the first optical element <NUM>.

In addition, a third optical element <NUM> for guiding a part of the laser light L to the imaging apparatus <NUM> is provided on the optical path of the laser light L transmitted through the first optical element <NUM>. The third optical element <NUM> condenses the part of the laser light L transmitted through the first optical element <NUM> and forms an image of the condensed part of the laser light L on an imaging surface of the imaging apparatus <NUM>. In the present example, since the laser light L is diffused by the first optical element <NUM>, it is preferable to provide the third optical element <NUM> that condenses the laser light L. The third optical element <NUM> is an example of an "optical element that guides a part of light to an imaging apparatus" according to the embodiment of the disclosed technology.

For example, the second optical element <NUM> and the third optical element <NUM> are lenses formed of glass or resin. The second optical element <NUM> and the third optical element <NUM> are not limited to lenses and may be optical elements such as holograms or concave mirrors or may be a combination thereof. It is preferable to configure the third optical element <NUM> with a hologram from a viewpoint of size reduction compared to a case of configuring the third optical element <NUM> with a liquid crystal or the like.

In addition, for example, the MEMS mirror <NUM> is accommodated in a package <NUM>. An inside of the package <NUM> is depressurized or is in a vacuum in order to decrease air resistance of the swinging mirror portion <NUM>. For example, the package <NUM> is formed of glass. At least a part of the package <NUM> has light transmittance. The laser light L emitted from the light emitting device <NUM> is incident on the MEMS mirror <NUM> through a light transmission portion <NUM> of the package <NUM>, and the laser light L reflected by the mirror portion <NUM> is emitted to the outside through the light transmission portion <NUM>.

In addition, as illustrated in <FIG> as an example, the imaging apparatus <NUM> may be provided inside the package <NUM>. In <FIG>, the imaging apparatus <NUM> is provided on the optical path of the laser light L reflected on a surface of the light transmission portion <NUM> out of the laser light L reflected by the mirror portion <NUM>. In this case, the imaging apparatus <NUM> receives the laser light L before the laser light L is diffused. Thus, it is not necessary to provide an optical element (the third optical element <NUM> illustrated in <FIG>) for condensing the laser light L. The captured image IP generated by performing an imaging operation by the imaging apparatus <NUM> is output to the outside through a lead terminal <NUM>. In such a manner, size reduction of the image forming apparatus <NUM> can be achieved by accommodating the imaging apparatus <NUM> inside the package <NUM>. In the present example, the light transmission portion <NUM> is an example of the "optical element that guides the part of the light to the imaging apparatus" according to the embodiment of the disclosed technology.

In addition, in the embodiment, while both of the first reference angle and the second reference angle are set to <NUM>°, the first reference angle and the second reference angle may be angles other than <NUM>°. The first reference angle may be set as an angle at which the first deflection angle θ1 is the maximum or minimum, and the second reference angle may be set as an angle at which the second deflection angle θ2 is the maximum or minimum. In this case, as illustrated in <FIG> as an example, four reference lines L1 to L4 are irradiated with the laser light L on the projection surface <NUM>. The reference lines L1 to L4 are straight lines tangential to an outer shape of the Lissajous curve <NUM>. In a case where a shift occurs between the swing of the mirror portion <NUM> and the timings of the first reference signal PS1 and the second reference signal PS2, the bright points P of the laser light L are displaced inward from the reference lines L1 to L4. The correction portion <NUM> may correct the timings of the first reference signal PS1 and the second reference signal PS2 based on displacement amounts of the bright points P from the reference lines L1 to L4 that are derived using the captured image IP.

In addition, in the embodiment, the first reference signal output portion 63A outputs the first reference signal PS1 by estimating the point in time when the first deflection angle θ1 becomes equal to the first reference angle, using the temperature and the humidity detected by the humidity and temperature sensor <NUM> and the LUT 66A. Instead, in implementations not according to the claimed subject matter, the first reference signal output portion 63A may output the first reference signal PS1 by estimating the point in time when the first deflection angle θ1 becomes equal to the first reference angle, based on an angle sensor (not illustrated) that detects the first deflection angle θ1. Similarly, in implementations not according to the claimed subject matter, the second reference signal output portion 63B may output the second reference signal PS2 by estimating the point in time when the second deflection angle θ2 becomes equal to the second reference angle, based on an angle sensor (not illustrated) that detects the second deflection angle θ2.

In addition, while in the claimed subject-matter the MEMS mirror <NUM> of two axes is used as the light scanning device in implementations not according to the claimed subject matter, a MEMS mirror of one axis may be used as the light scanning device. Accordingly, the disclosed technology can also be applied to an image forming apparatus comprising a light scanning device of one axis in which a movable mirror swings about a first axis.

The embodiment and each modification example can be appropriately combined without contradiction.

In the embodiment, for example, the following various processors can be used as a hardware structure of a processing unit that executes various processing of the image input portion <NUM>, the information generation portion <NUM>, the information storage portion <NUM>, the first reference signal output portion 63A, the second reference signal output portion 63B, the light emission controller <NUM>, the correction portion <NUM>, and the table holding portion <NUM>. The various processors include, in addition to a central processing unit (CPU) that is a general-purpose processor functioning as various processing units by executing software (program), a programmable logic device (PLD) such as the FPGA that is a processor having a circuit configuration changeable after manufacturing, a dedicated electric circuit such as an application specific integrated circuit (ASIC) that is a processor having a circuit configuration dedicatedly designed to execute specific processing, and the like.

One processing unit may be configured with one of the various processors or may be configured with a combination of two or more processors of the same type or different types (for example, a combination of a plurality of FPGAs and/or a combination of a CPU and an FPGA). In addition, a plurality of processing units may be configured with one processor.

Examples of the plurality of processing units configured with one processor include, first, as represented by a computer such as a client and a server, a form in which one processor is configured with a combination of one or more CPUs and software, and the processor functions as the plurality of processing units. Second, as represented by a system on chip (SoC) or the like, a form of using a processor that implements functions of the entire system including the plurality of processing units by one integrated circuit (IC) chip is included. In such a manner, various processing units are configured using one or more of the various processors as a hardware structure.

Claim 1:
An image forming apparatus (<NUM>) comprising:
a light emitting device (<NUM>) that emits light;
a movable mirror (<NUM>) that reflects the light emitted from the light emitting device (<NUM>);
a first actuator (<NUM>) that causes the movable mirror (<NUM>) to swing about a first axis (ai);
a second actuator (<NUM>) that causes the movable mirror (<NUM>) to swing about a second axis (a<NUM>);
a first reference signal output portion (63A) that outputs a first reference signal by estimating a point in time when a deflection angle of the movable mirror (<NUM>) about the first axis (a<NUM>) becomes equal to a first reference angle, wherein the first reference signal output portion (63A) estimates the point in time when the deflection angle of the movable mirror (<NUM>) about the first axis (a<NUM>) becomes equal to the first reference angle, based on a first driving signal provided to the first actuator (<NUM>);
a second reference signal output portion (63B) that outputs a second reference signal by estimating a point in time when a deflection angle of the movable mirror (<NUM>) about the second axis (a<NUM>) becomes equal to a second reference angle, the second reference signal output portion (63B) estimates the point in time when the deflection angle of the movable mirror (<NUM>) about the second axis (a<NUM>) becomes equal to the second reference angle, based on a second driving signal provided to the second actuator (<NUM>);
a light emission controller (<NUM>) that causes the light emitting device (<NUM>) to emit the light based on the first reference signal output from the first reference signal output portion (63A);
an imaging apparatus (<NUM>) that images the light reflected by the movable mirror (<NUM>); and
a correction portion (<NUM>) that corrects a timing of the first reference signal output by the first reference signal output portion (63A) and a timing of the second reference signal output by the second reference signal output portion (63B) based on imaging information acquired by the imaging apparatus (<NUM>).