Projection type display device, control method of projection type display device, control program of projection type display device

Provided are a projection type display device, a control method of a projection type display device, and a control program of a projection type display device, capable of being continuously used even in a case where a vehicle vibrates, and performing optimal display depending on the vibration. An HUD 100 includes a projection display section 50 that projects image light obtained by spatially modulating light emitted from a light source onto a combiner 12 to display a virtual image, a first vibration detector 61 that detects a first vibration of the combiner 12, a second vibration detector 62 that detects a second vibration of the projection display section 50, a third vibration detector 63 that detects a third vibration of the projection display section 50 with respect to the combiner 12 on the basis of the first vibration and the second vibration, and a display controller 64 that controls the image to be displayed by the projection display section 50. The display controller 64 changes a display format of content to be displayed by the projection display section 50 on the basis of the third vibration.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a projection type display device, a control method of the projection type display device, and a computer readable medium storing a control program of the projection type display device.

2. Description of the Related Art

A vehicle head-up display (HUD) that projects, using a windshield in a variety of vehicles such as an automobile, a train, a ship, a construction machine, an aircraft, or an agricultural machine, or a combiner disposed in the vicinity of the windshield as a screen, light onto the screen to display an image is known (see JP2015-009677A and JP2016-193723A). According to the HUD, it is possible to cause a driver to visually recognize an image based on light projected from the HUD as a real image on a screen or a virtual image in front of the screen.

JP2015-009677A discloses an HUD that projects image light onto a windshield of a vehicle to perform display. The HUD increases the size of display content in a case where vibration of the vehicle is large, to enhance visibility during the vibration.

JP2016-193723A discloses an HUD that projects image light onto a combiner disposed in a vehicle to perform display. The HUD sets the combiner to a closed state to stop display of an image in a case where vibration of the vehicle is large, to secure forward visibility.

SUMMARY OF THE INVENTION

In an HUD for displaying an image using a combiner provided in a vehicle, an installation position of the combiner in the vehicle and an installation position of a unit that projects image light of the HUD are different from each other. Further, the combiner is formed in a thin plate shape, and vibrates more easily than in the unit that projects the image light. Accordingly, a possibility that a combiner vibration and a unit vibration due to the vibration of the vehicle are not the same is high. In a case of a working machine such as a heavy machine, a construction machine, or an agricultural machine, since a vehicle frequently vibrates, a difference between the combiner vibration and the unit vibration becomes extremely large.

The HUD disclosed in JP2015-009677A controls the size of display content on the basis of a relative vibration of the vehicle with respect to a different vehicle, but in a case where image light is projected to the combiner, since the combiner and the unit separately vibrate, there is a case where it is not necessary to change the size of the display content. For example, even though the vehicle vibrates with respect to the different vehicle, there is a case where shaking of the unit for the combiner is small with respect to the different vehicle. In such a case, it is not necessary to change the display.

In the HUD disclosed in JP2016-193723A, in a case where the vibration of the vehicle is large, display of an image is stopped, for example, in a vehicle in which it is presumed that a vehicle body is severely shaking as in the working machine, the HUD cannot be nearly used.

In consideration of the above-mentioned problems, an object of the invention is to provide a projection type display device, a control method of the projection type display device, and a computer readable medium storing a control program of the projection type display device, capable of continuously using a vehicle in a case where the vehicle vibrates, and capable of performing optimal display depending on the vibration.

According to an aspect of the invention, there is provided a projection type display device comprising: a projection display section that includes a light modulator that spatially modulates light emitted from a light source on the basis of input image information, and projects image light obtained through spatial modulation using the light modulator onto a combiner that is provided in a vehicle to display an image based on the image light; a first vibration detector that detects a first vibration of the combiner; a second vibration detector that detects a second vibration of the projection display section; a third vibration detector that detects a third vibration of the projection display section with respect to the combiner on the basis of the first vibration and the second vibration; and a display controller that controls the image information input to the light modulator to control the image to be displayed by the projection display section, in which the display controller changes a display format of content to be displayed by the projection display section on the basis of the third vibration.

According to another aspect of the invention, there is provided a control method of a projection type display device that includes a projection display section that includes a light modulator that spatially modulates light emitted from a light source on the basis of input image information, and projects image light obtained through spatial modulation using the light modulator onto a combiner that is provided in a vehicle to display an image based on the image light, the method comprising: a first vibration detection step of detecting a first vibration of the combiner; a second vibration detection step of detecting a second vibration of the projection display section; a third vibration detection step of detecting a third vibration of the projection display section with respect to the combiner on the basis of the first vibration and the second vibration; and a display control step of controlling the image information input to the light modulator to control the image to be displayed by the projection display section, in which in the display control step, a display format of content to be displayed by the projection display section is changed on the basis of the third vibration.

According to still another aspect of the invention, there is provided a non-transitory computer readable medium storing a control program of a projection type display device that includes a projection display section that includes a light modulator that spatially modulates light emitted from a light source on the basis of input image information, and projects image light obtained through spatial modulation using the light modulator onto a combiner that is provided in a vehicle to display an image based on the image light, the control program causing a computer to execute: a first vibration detection step of detecting a first vibration of the combiner; a second vibration detection step of detecting a second vibration of the projection display section; a third vibration detection step of detecting a third vibration of the projection display section with respect to the combiner on the basis of the first vibration and the second vibration; and a display control step of controlling the image information input to the light modulator to control the image to be displayed by the projection display section, in which in the display control step, a display format of content to be displayed by the projection display section is changed on the basis of the third vibration.

According to the invention, it is possible to provide a projection type display device, a control method of the projection type display device, and a computer readable medium storing a control program of the projection type display device, capable of continuously using a vehicle in a case where the vehicle vibrates, and capable of performing optimal display depending on the vibration.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1is a schematic diagram showing a configuration of a construction machine1in which an HUD100that is a projection type display device according to a first embodiment of the invention is mounted.

The construction machine1is a hydraulic excavator, which includes respective parts of a lower traveling body2, an upper revolving body3that is supported by the lower traveling body2to be revolved, a front working part4that is supported by the upper revolving body3, and the like. The lower traveling body2and the upper revolving body3form a main part of the construction machine1.

The lower traveling body2includes a metallic or rubber crawler for traveling a public road and a workplace.

The upper revolving body3includes an operator cab5in which an operating device for operating the front working part4and a cab seat6on which an operator is seated are provided.

In the operator cab5, a front windshield11is provided in front of the cab seat6where an operator7is seated, and a combiner12is provided between the front windshield11and the cab seat6.

A projection unit10that forms a projection type display device is provided in the operator cab5, and causes the operator7who sits on the cab seat6to visually recognize a virtual image in front of the combiner12using image light projected onto the combiner12.

The front working part4includes an arm4C that is movably supported by the upper revolving body3in a gravity direction and a direction (a vertical direction in the figure) perpendicular to the gravity direction, a boom4B that is supported by the arm4C to be rotatable with respect to the arm4C, and a bucket4A that is supported by the boom4B to be rotatable with respect to the boom4B. The bucket4A is a part that is directly in contact with a working target such as a ground surface or a discharge, and forms a working tool.

A configuration in which a different working tool such as a steel cutter, a concrete crusher, a gripper, or a percussion type crushing tool, instead of the bucket4A, is mounted in the boom4B may be used.

The bucket4A may be moved in the vertical direction in the figure with respect to the operator cab5through the arm4C and the boom4B. Further, the bucket4A is rotatable using a direction (a direction vertical to a sheet plane in the figure) perpendicular to a sight direction of the operator who sits on the cab seat6and the gravity direction as an axis. Further, the boom4B is rotatable using the direction vertical to the sheet plane in the figure as an axis.

FIG. 2is a schematic diagram showing an example of an internal configuration of the operator cab5in the construction machine1shown inFIG. 1.

As shown inFIG. 2, the HUD100includes a projection unit10, a combiner mounting part15, and the combiner12. The combiner mounting part15is fixed to a right side pillar13of the operator cab5. The combiner12is attachably and detachably provided in the combiner mounting part15.

The projection unit10is provided on an upper and rear side of the operator7in a state where the operator7sits on the cab seat6, and projects image light onto the combiner12mounted in the combiner mounting part15.

The operator7of the construction machine1can view image light that is projected onto the combiner12that is mounted in the combiner mounting part15and is reflected therefrom to visually recognize information on an icon, a text, or the like for supporting an operation of the construction machine1as a virtual image. Further, the combiner12has a function of reflecting the image light projected from the projection unit10and transmitting light from the outside (outside world). Accordingly, the operator can visually recognize the virtual image based on the image light projected from the projection unit10and an outside scene in an overlapping manner.

In the example shown inFIG. 1, and the HUD100is mounted in a hydraulic excavator, but the HUD100may be similarly mounted in a vehicle such as an automobile, an aircraft, a train, or a ship, as well as a working machine (for example, a wheel loader, a bulldozer, a motor grader, a forklift, a tracker, or the like) in which a working tool capable of being operated by an operator is mounted in front of the cab seat6.

As shown inFIG. 2, a front direction (a movement direction in a case where the lower traveling body2moves forward) of the cab seat6in the operator cab5is referred to as a second direction Y, a direction vertical to the seat of the cab seat6is referred to as a third direction Z, and a direction that is vertical to both of the second direction Y and the third direction Z is referred to as a first direction X.

FIG. 3is a schematic diagram showing a state where the front windshield11is seen from the cab seat6of the operator cab5in the construction machine1shown inFIG. 1.

The operator cab5is surrounded by the front windshield11, a right side windshield21, and a left side windshield22. The operator cab5includes a left operation lever23for operating bending and stretching of the front working part4and revolution of the upper revolving body3, a right operation lever24for operating drilling and opening of the bucket4A of the front working part4, and the like, around the cab seat6.

Allocation of operational functions to the left operation lever23and the right operation lever24is an example, and the invention is not limited thereto. The left operation lever23and the right operation lever24form an operation member for performing operations of the bucket4A such as movement of the bucket4A, drilling using the bucket4A, and opening of the bucket4A.

The right side pillar13is provided between the front windshield11and the right side windshield21, and the combiner mounting part15is fixed to the right side pillar13. InFIG. 3, a projection range12A in which image light projected from the projection unit10is incident is shown in the combiner12.

Further, a triaxial acceleration sensor12B is fixed to the combiner12. The triaxial acceleration sensor12B detects an acceleration of the combiner12in the first direction X, an acceleration of the combiner12in the second direction Y, and an acceleration of the combiner12in the third direction Z, respectively.

FIG. 4is a schematic diagram showing an internal configuration of the projection unit10shown inFIGS. 1 and 2.

The projection unit10includes a light source unit40, a light modulation element44, a drive section45that drives the light modulation element44, a projection optical system46, a diffuser47, a reflecting mirror48, a magnifier49, a system controller60that controls the light source unit40and the drive section45, a storage70configured of a storage medium such as a flash memory, a housing40that contains the above-mentioned components, and a triaxial acceleration sensor80fixed to the housing40.

The light source unit40includes a light source controller40A, an R light source41rthat is a red light source that emits red light, a G light source41gthat is a green light source that emits green light, a B light source41bthat is a blue light source that emits blue light, a dichroic prism43, a collimator lens42rthat is provided between the R light source41rand the dichroic prism43, a collimator lens42gthat is provided between the G light source41gand the dichroic prism43, and a collimator lens42bthat is provided between the B light source41band the dichroic prism43.

The dichroic prism43is an optical member for guiding light emitted from each of the R light source41r, the G light source41g, and the B light source41bto the same optical path. That is, the dichroic prism43transmits red light that is collimated by the collimator lens42rto be output to the light modulation element44. Further, the dichroic prism43reflects green light that is collimated by the collimator lens42gto be output to the light modulation element44. In addition, the dichroic prism43reflects blue light that is collimated by the collimator lens42bto be output to the light modulation element44. An optical member having such a function is not limited to a dichroic prism. For example, a cross dichroic mirror may be used.

The R light source41r, the G light source41g, and the B light source41bmay respectively employ a light emitting element such as a laser or a light emitting diode (LED). The R light source41r, the G light source41g, and the B light source41bform light sources of the HUD100. In this embodiment, the light sources of the HUD100are configured to include three light sources of the R light source41r, the G light source41g, and the B light source41b, but the number of light sources may be 1, 2, or 4 or more.

The light source controller40A sets the intensity of light emitted from each of the R light source41r, the G light source41g, and the B light source41bto a predetermined light intensity pattern, and performs a control for sequentially emitting light from the R light source41r, the G light source41g, and the B light source41baccording to the light intensity patterns.

The light modulation element44spatially modulates light that is emitted from the dichroic prism43on the basis of image information, and emits the spatially modulated light (red color image light, blue color image light, and green color image light) to the projection optical system46.

The light modulation element44may employ, for example, a liquid crystal on silicon (LCOS), a digital micromirror device (DMD), a micro electro mechanical systems (MEMS) element, a liquid crystal display device, or the like.

The drive section45drives the light modulation element44on the basis of image information input from the system controller60, and causes light (red color image light, blue color image light, and green color image light) based on the image information to be output to the projection optical system46.

The light modulation element44and the drive section45form a light modulator of the HUD100.

The projection optical system46is an optical system that projects light emitted from the light modulation element44of the light source unit40to the diffuser47. The optical system is not limited to a lens, and may employ a scanner. For example, light emitted from a scan type scanner may be diffused by the diffuser47to become a plane light source.

The reflecting mirror48reflects the light diffused by the diffuser47toward the magnifier49.

The magnifier49magnifies an image based on the light reflected from the reflecting mirror48to be projected in the projection range12A.

The light source unit40, the light modulation element44, the drive section45, the projection optical system46, the diffuser47, the reflecting mirror48, and the magnifier49form a projection display section50that spatially modulates light emitted from the R light source41r, the G light source41g, and the B light source41bon the basis of image information input from the system controller60and projects image light obtained through the spatial modulation onto the combiner12mounted in the combiner mounting part15to display a virtual image based on the image light.

The projection display section50, the system controller60, and the storage70are contained in the housing30, and an opening30A for emitting the image light from the magnifier49is provided in a part of the housing30.

The triaxial acceleration sensor80is fixed in the vicinity of the opening30A of the housing30, and detects an acceleration in the first direction X, an acceleration in the second direction Y, and an acceleration in the third direction Z, respectively. The projection display section50is supported by the housing30. Accordingly, it is possible to respectively detect the acceleration in the first direction X, the acceleration in the second direction Y, and the acceleration in the third direction Z of the projection display section50using the triaxial acceleration sensor80.

The system controller60controls the light source controller40A and the drive section45so that image light based on image information is emitted to the diffuser47through the projection optical system46.

The diffuser47, the reflecting mirror48, and the magnifier49shown inFIG. 4are optically designed so that an image based on image light projected onto the combiner12can be visually recognized as a virtual image at a position in front of the combiner12.

The system controller60is configured of a variety of processors as main components, and includes a read only memory (ROM) in which a program for executing the processors is stored, a random access memory (RAM) that is a work memory, and the like.

The variety of processors includes a central processing unit (CPU) that is a general-purpose processor that performs a variety of processes by executing a program, a programmable logic device (PLD) that is a processor capable of changing a circuit configuration after manufacturing, such as a field programmable gate array (FPGA), an exclusive electric circuit that is a processor that has a circuit configuration that is exclusively designed for executing a specific process, such as an application specific integrated circuit (ASIC), or the like.

A structure of the variety of processors is, more specifically, an electric circuit in which circuit elements such as semiconductor devices are combined.

A processor of the system controller60may be configured as one processor among a variety of processors, or may be configured as a combination of the same kind or different kinds of two or more processors (for example, a combination of a plurality of FPGAs or a combination of a CPU and an FPGA).

The storage70stores work support information and different relevant information. The work support information refers to information that is displayed in the vicinity of the bucket4A that is frequently watched by an operator during work and supports efficient progress of the work. The work support information includes a text or an arrow indicating a drilling direction of the bucket4A, a text or a scale indicating the amount of drilling (OO m), warning information such as a text or an icon for calling operator's attention, or the like. The relevant information refers to information that is not directly relevant to work, such as a date, a time, fuel, and the like.

A vehicle body controller81shown inFIG. 4generally controls the construction machine1. The construction machine1is provided with a display section82such as a liquid crystal display device or an organic electro luminescence (EL) display device (not shown inFIGS. 1 to 3).

FIG. 5is a functional block diagram of the system controller60shown inFIG. 4.

The system controller60includes a first vibration detector61, a second vibration detector62, a third vibration detector63, and a display controller64. The first vibration detector61, the second vibration detector62, the third vibration detector63, and the display controller64are functional blocks formed as the processor of the system controller60executes a program including a control program of the projection type display device stored in the ROM.

The first vibration detector61detects a vibration in the first direction X applied to the combiner12, on the basis of information on the acceleration in the first direction X detected by the triaxial acceleration sensor12B.

In this specification, information detected as a vibration of an object includes a frequency and an amplitude of the vibration. The frequency of the vibration may be calculated by a change in the size of acceleration information output in a time series manner. The amplitude of the vibration is calculated through computation using the frequency of the vibration and the acceleration information.

Further, the first vibration detector61detects a vibration in the second direction Y applied to the combiner12, on the basis of information on the acceleration in the second direction Y detected by the triaxial acceleration sensor12B.

In addition, the first vibration detector61detects a vibration in the third direction Z applied to the combiner12, on the basis of information on the acceleration in the third direction Z detected by the triaxial acceleration sensor12B.

The second vibration detector62detects a vibration in the first direction X applied to the projection display section50, on the basis of information on the acceleration in the first direction X detected by the triaxial acceleration sensor80.

The second vibration detector62detects a vibration in the second direction Y applied to the projection display section50, on the basis of information on the acceleration in the second direction Y detected by the triaxial acceleration sensor80.

In addition, the second vibration detector62detects a vibration in the third direction Z applied to the projection display section50, on the basis of information on the acceleration in the third direction Z detected by the triaxial acceleration sensor80.

The third vibration detector63detects a third vibration (relative vibration) of the projection display section50with respect to the combiner12, on the basis of the first vibration detected by the first vibration detector61and the second vibration detected by the second vibration detector62.

The third vibration detector63detects a third vibration in the first direction X on the basis of the first vibration and the second vibration in the first direction X, detects a third vibration in the second direction Y on the basis of the first vibration and the second vibration in the second direction Y, and detects a third vibration in the third direction Z on the basis of the first vibration and the second vibration in the third direction.

The display controller64controls image information input to the drive section45, to control an image to be displayed by the projection display section50. Further, the display controller64changes a display format of content to be displayed by the projection display section50on the basis of the third vibration detected by the third vibration detector63.

A plurality of display formats may be provided as the display format of the content displayed by the projection display section50. Hereinafter, an example in which a first display format and a second display format are exchangeable will be described.

The second display format is a display format with a visibility in vibration higher than that of the first display format. The display format with a high visibility refers to a display format in which in a case where displayed content vibrates, an observer who observes the content does not easily feel fatigue or discomfort or can easily recognize details of the content, which is determined on the basis of ergonomics, for example.

FIG. 6is a diagram showing an example in which content is displayed in the first display format by the projection display section50.

As shown inFIG. 6, the bucket4A and an obstacle111such as a human are observed from an operator through the projection range12A of the combiner12. Further, pieces of content121to125are displayed in the projection range12A.

The content121is formed of an arrow image indicating a movement direction (downward direction) of the bucket4A and a text image indicating the amount of drilling (A m).

The content122is a text image for warning the operator.

The content123is a frame image that surrounds the obstacle111for notifying the operator of the position of the obstacle111.

The content124is a text image indicating a current time.

The content125is an icon image indicating a state of fuel of the construction machine1. “E” in the icon image is a character indicating a state where there is no fuel, and “F” is a character indicating a state where fuel is full.

FIG. 7is a diagram showing an example in which content is displayed in the second display format by the projection display section50.

As shown inFIG. 7, the content121shown inFIG. 6is changed to content121A in the second display format. The content121A is formed of only an arrow image, without a text image indicating the amount of drilling. A text image is not easily readable in vibration, and thus, its visibility deteriorates. In the content121A, since the text image is removed, the visibility is enhanced.

The content122shown inFIG. 6is changed to content122A in the second display format. The content122A is formed of an icon image indicating a warning. A text image is not easily readable in vibration, and thus, its visibility deteriorates. In the content122A, since the text image is changed to the icon image, the visibility is enhanced.

The content123shown inFIG. 6is changed to content123A in the second display format. The content123A is formed so that the size of a frame image with respect to the obstacle111is larger than that in the first display format. As the frame image becomes large, a gap between the frame image and the obstacle111becomes large. Thus, in a case where the content123A vibrates, visibility of the content123A is enhanced.

The content124shown inFIG. 6is changed to content124A in the second display format. The content124A is formed so that a text font size is enlarged more than that in the first display format. In a case where a text has a small font size, the text is not easily readable due to vibration, but in a case where a text has a large font size, the text is easily readable even in a case where vibration occurs, and thus, visibility is enhanced.

The content125shown inFIG. 6is changed to content125A in the second display format. The content125A is formed so that its size is enlarged more than that in the first display format and the texts of “E” and “F” are removed. In this way, as the icon image is enlarged and texts are removed, visibility is enhanced.

Although not shown inFIG. 7, for example, in the content121A, the content122A, or the content123A, by setting the thickness of a contour of each image to be larger than that in the first display format, it is also possible to enhance the visibility.

Further, in the second display format, the content121A or the content122A may be displayed to be distantly separated from the bucket4A compared with the case of the first display format. Thus, even in a case where the content121A or the content122A vibrates up and down, for example, it is possible to prevent the content121A or the content122A from overlapping with and the bucket4A, thereby enhancing the visibility of the content.

FIG. 8is a flowchart for illustrating an operation of the HUD100.

In a case where the HUD100is started, the first vibration detector61acquires acceleration information from the triaxial acceleration sensor12B, and detects a first vibration (including a vibration in the first direction X, a vibration in the second direction Y, and a vibration in the third direction Z) of the combiner12on the basis of the acquired acceleration information (step S1).

Further, the second vibration detector62acquires acceleration information from the triaxial acceleration sensor80, and detects a second vibration (including a vibration in the first direction X, a vibration in the second direction Y, and a vibration in the third direction Z) of the projection display section50on the basis of the acquired acceleration information (step S2).

In addition, the third vibration detector63detects a third vibration (including a vibration in the first direction X, a vibration in the second direction Y, and a vibration in the third direction Z) of the projection display section50with respect to the combiner12on the basis of the first vibration detected in step S1and the second vibration detected in step S2(step S3).

Then, the display controller64determines whether the third vibration in the first direction X detected in step S3satisfies a first condition (step S4). The first condition is a condition in which content is visually recognized so that a sense of resolution of the content deteriorates more considerably than that in a case where there is no vibration, and specifically, is a condition in which a frequency of vibration is equal to or greater than a frequency threshold and an amplitude of the vibration is equal to or greater than an amplitude threshold.

In a case where the third vibration applied in the first direction X does not satisfy the first condition (NO in step S4), the display controller64determines whether the third vibration in the second direction Y detected in step S3satisfies a second condition (step S5). The second condition is the same as the first condition.

In a case where the third vibration applied in the second direction Y does not satisfy the second condition (NO in step S5), the display controller64determines whether the third vibration in the third direction Z detected in step S3satisfies a third condition (step S6). The third condition is the same as the first condition. The first condition, the second condition, and the third condition form predetermined conditions.

In a case where the third vibration applied in the third direction Z does not satisfy the third condition (NO in step S6), the display controller64controls the display format of the content to be displayed by the projection display section50to the first display format (step S8).

On the other hand, in a case where the third vibration applied in the first direction X satisfies the first condition (YES in step S4), in a case where the third vibration applied in the second direction Y satisfies the second condition (YES in step S5), or in a case where the third vibration applied in the third direction Z satisfies the third condition (YES in step S6), the display controller64controls the display format of the content to be displayed by the projection display section50to the second display format (step S7).

After step S7and step S8, the procedure returns to step S1, and during the operation of the HUD100, the above-mentioned processes are repeated.

As described above, according to the HUD100, it is possible to display content by performing exchange between the first display format in which the amount of information is prioritized and the second display format in which visibility is prioritized, on the basis of information (frequency and amplitude) on vibration of the projection display section50with respect to the combiner12. Thus, even in a situation where the construction machine1considerably vibrates, it is possible to enhance visibility of content.

In this embodiment, the first condition, the second condition, and the third condition are the same conditions, but may be different conditions, or two among the three conditions and the remaining one may be different from each other.

Humans are sensitive to a sense of resolution of a display image in a horizontal direction more than that in a vertical direction. Accordingly, by setting the first condition relating to the third vibration in the first direction X as a condition that is stricter than the second condition and the third condition (for example, by setting the amplitude threshold as a lower value), it is possible to enhance visibility of content.

Further, in a case where the amplitude of the third vibration applied in the second direction Y is large, a distance from an operator to content to be displayed is changed. With respect to a vibration in a depth direction of the content to be displayed, it is considered that an influence on the sense of resolution of the content is small. Accordingly, by setting the second condition relating to the third vibration in the second direction Y as a condition that is smoother than the first condition and the third condition (for example, by setting the amplitude threshold as a higher value), it is possible to reduce a switching frequency of the display formats, and as a result, it is possible to enhance visibility of content.

Further, an optical member disposed at a rear stage with reference to the light modulation element44among the optical members that form the projection display section50is supported by the housing30, but a direction in which the optical member easily vibrates is determined according to which portion the optical member is supported by.

For example, a configuration in which the magnifier49is supported at both end portions in the first direction X is assumed. In this configuration, the magnifier49does not easily vibrate with respect to the vibration in the first direction X, but vibrates in the second direction Y more easily than in the first direction X. Accordingly, by setting the condition relating to the third vibration in the second direction Y as a condition that is stricter than the first condition (for example, by setting the amplitude threshold as a lower value), it is possible to enhance visibility of content.

Further, inFIG. 8, in a case where the determination in step S4is YES, the procedure may not directly proceed to step S7, and instead, in a case where the frequency of the YES determination in step S4or a period of time during which the determination of step S4continuously is YES exceeds a threshold, the procedure may proceed to step S7. Thus, it is possible to prevent a display format of content from being frequently changed due to a slight change of vibration, to thereby enhance visibility of the content.

For the same reason, in a case where the determination of step S5is YES, the procedure may not directly proceed to step S7, and instead, in a case where the frequency of the YES determination in step S5or a period of time during which the determination of step S5continuously is YES exceeds a threshold, the procedure may proceed to step S7.

For the same reason, in a case where the determination of step S6is YES, the procedure may not directly proceed to step S7, and instead, in a case where the frequency of the YES determination in step S6or a period of time during which the determination of step S6continuously is YES exceeds a threshold, the procedure may proceed to step S7.

Further, in a case where the amplitude of the third vibration in the first direction X and the amplitude of the third vibration in the third direction Z detected in step S3exceed a value that is sufficiently larger than the amplitude threshold, the display controller64may stop projection of image light in the projection display section50to stop display of a virtual image. In this way, it is possible to prevent an operator from suffering from visually induced motion sickness, to thereby enhance work efficiency.

Further, in a case where the display of the virtual image is stopped, it is preferable that the display controller64causes the display section82to display an image based on the image information through the vehicle body controller81. Thus, it is possible to continuously provide work support information or the like to the operator, to thereby prevent decrease in work efficiency.

Further, in a case where the content is displayed in the second display format in step S7, and in a case where the amplitude of the third vibration in the first direction X or the amplitude of the third vibration in the third direction Z detected in step S3exceeds a threshold, it is preferable that the display controller64continuously moves the content in a direction where the third vibration in the first direction X or the third direction Z is negated to display the content.

For example, in the display example shown inFIG. 7, in a case where it is detected that the content121A is moved in a lateral direction by the threshold at a predetermined frequency, by repeating a process of moving the content121A rightward by the threshold with respect to an original display position to display the content121A at a timing when the content121A is moved leftward and moving the content121A leftward by the threshold with respect to the original display position to display the content121A at a timing when the content121A is moved rightward, it is possible to negate the vibration of the content121A.

As described above, by changing the first display format to the second display format and continuously moving content in a direction where vibration is negated to display the content, it is possible to further enhance visibility of the content when the vibration is applied to the construction machine1.

Further, in a case where a certain determination in steps4,5, and6is YES and the amplitude of the third vibration in the first direction X or the amplitude of the third vibration in the third direction Z detected in step S3exceeds a threshold, the display controller64may cause the projection display section50to display information for urging confirmation of an installation state of the combiner12(for example, a text image of “Please, confirm a combiner mounting state.”).

It may be considered that one reason why the amplitude of the third vibration becomes extremely large is because the combiner12is not favorably mounted with respect to the combiner mounting part15. For this reason, in this case, by causing an operator to confirm the installation state of the combiner12, it is possible to remove a cause of vibration of the combiner12, to thereby enhance visibility of content.

In the above description, a configuration in which vibration in three directions of the projection display section50with respect to the combiner12is detected by the triaxial acceleration sensor12B and the triaxial acceleration sensor80is shown, but a configuration in which in detecting the vibration of the projection display section50with respect to the combiner12, a vibration in one direction among three directions is detected may be used. In this case, similarly, it is possible to perform exchange between display formats of content, to thereby enhance visibility in a case where the vibration is large.

Further, a configuration in which an acceleration sensor is used for detection of vibration of the combiner12and vibration of the projection display section50is shown, but a configuration in which vibration is more specifically detected using an acceleration sensor and an angular velocity sensor may be used.

As described above, the following configurations are disclosed in this specification.

(1) There is disclosed a projection type display device includes: a projection display section that includes a light modulator that spatially modulates light emitted from a light source on the basis of input image information, and projects image light obtained through spatial modulation using the light modulator onto a combiner that is provided in a vehicle to display an image based on the image light; a first vibration detector that detects a first vibration of the combiner; a second vibration detector that detects a second vibration of the projection display section; a third vibration detector that detects a third vibration of the projection display section with respect to the combiner on the basis of the first vibration and the second vibration; and a display controller that controls the image information input to the light modulator to control the image to be displayed by the projection display section, in which the display controller changes a display format of content to be displayed by the projection display section on the basis of the third vibration.

(2) In the projection type display device according to (1), in a case where the third vibration satisfies a predetermined condition, the display controller changes the display format of the content to a display format with a visibility higher than a visibility of a display format of the content in a case where the third vibration does not satisfy the condition.

(3) In the projection type display device according to (2), the first vibration detector detects the first vibration in each of a first direction, a second direction, and a third direction, on the basis of acceleration information in each of the first direction, the second direction, and the third direction detected by a triaxial acceleration sensor fixed to the combiner, in which the second vibration detector detects the second vibration in each of the first direction, the second direction, and the third direction, on the basis of acceleration information in each of the first direction, the second direction, and the third direction detected by a triaxial acceleration sensor fixed to a housing including the projection display section, the third vibration detector detects the third vibration in the first direction on the basis of the first vibration and the second vibration in the first direction, detects the third vibration in the second direction on the basis of the first vibration and the second vibration in the second direction, and detects the third vibration in the third direction on the basis of the first vibration and the second vibration in the third direction, the condition includes a first condition relevant to the third vibration in the first direction, a second condition relevant to the third vibration in the second direction, and a third condition relevant to the third vibration in the third direction, at least one condition among the first condition, the second condition, and the third condition is different from other conditions among the first condition, the second condition, and the third condition, and the display controller changes the display format of the content in a case where the third vibration in the first direction satisfies the first condition, in a case where the third vibration in the second direction satisfies the second condition, or in a case where the third vibration in the third direction satisfies the third condition.

(4) In the projection type display device according to any one of (1) to (3), in a case where an amplitude of the third vibration exceeds a threshold, the display controller continuously moves the content to be displayed by the projection display section in a direction where the third vibration is negated to display the content.

(5) In the projection type display device according to any one of (1) to (3), in a case where an amplitude of the third vibration exceeds a threshold, the projection display section displays information for urging confirmation of a mounting state of the combiner.

(6) There is disclosed a control method of a projection type display device that includes a projection display section that includes a light modulator that spatially modulates light emitted from a light source on the basis of input image information, and projects image light obtained through spatial modulation using the light modulator onto a combiner that is provided in a vehicle to display an image based on the image light, includes: a first vibration detection step of detecting a first vibration of the combiner; a second vibration detection step of detecting a second vibration of the projection display section; a third vibration detection step of detecting a third vibration of the projection display section with respect to the combiner on the basis of the first vibration and the second vibration; and a display control step of controlling the image information input to the light modulator to control the image to be displayed by the projection display section, in which in the display control step, a display format of content to be displayed by the projection display section is changed on the basis of the third vibration.

(7) In the control method of the projection type display device according to (6), in the display control step, in a case where the third vibration satisfies a predetermined condition, the display format of the content is changed to a display format with a visibility higher than a visibility of a display format of the content in a case where the third vibration does not satisfy the condition.

(8) In the control method of the projection type display device according to (7), in the first vibration detection step, the first vibration in each of a first direction, a second direction, and a third direction is detected on the basis of acceleration information in each of the first direction, the second direction, and the third direction detected by a triaxial acceleration sensor fixed to the combiner, in the second vibration detection step, the second vibration in each of the first direction, the second direction, and the third direction is detected on the basis of acceleration information in each of the first direction, the second direction, and the third direction detected by a triaxial acceleration sensor fixed to a housing including the projection display section, in the third vibration detection step, the third vibration in the first direction is detected on the basis of the first vibration and the second vibration in the first direction, the third vibration in the second direction is detected on the basis of the first vibration and the second vibration in the second direction, and the third vibration in the third direction is detected on the basis of the first vibration and the second vibration in the third direction, the condition includes a first condition relevant to the third vibration in the first direction, a second condition relevant to the third vibration in the second direction, and a third condition relevant to the third vibration in the third direction, at least one condition among the first condition, the second condition, and the third condition is different from other conditions among the first condition, the second condition, and the third condition, and in the display control step, the display format of the content is changed in a case where the third vibration in the first direction satisfies the first condition, in a case where the third vibration in the second direction satisfies the second condition, or in a case where the third vibration in the third direction satisfies the third condition.

(9) In the control method of the projection type display device according to any one of (6) to (8), in the display control step, in a case where an amplitude of the third vibration exceeds a threshold, the content to be displayed by the projection display section is continuously moved in a direction where the third vibration is negated to be displayed.

(10) In the control method of the projection type display device according to any one of (6) to (8), in the display control step, in a case where an amplitude of the third vibration exceeds a threshold, the projection display section displays information for urging confirmation of a mounting state of the combiner.

(11) There is disclosed a non-transitory computer readable medium storing a control program of a projection type display device that includes a projection display section that includes a light modulator that spatially modulates light emitted from a light source on the basis of input image information, and projects image light obtained through spatial modulation using the light modulator onto a combiner that is provided in a vehicle to display an image based on the image light, causes a computer to execute: a first vibration detection step of detecting a first vibration of the combiner; a second vibration detection step of detecting a second vibration of the projection display section; a third vibration detection step of detecting a third vibration of the projection display section with respect to the combiner on the basis of the first vibration and the second vibration; and a display control step of controlling the image information input to the light modulator to control the image to be displayed by the projection display section, in which in the display control step, a display format of content to be displayed by the projection display section is changed on the basis of the third vibration.

According to the invention, it is possible to provide a projection type display device, a control method of the projection type display device, and a computer readable medium storing a control program of the projection type display device, capable of continuously using a vehicle in a case where the vehicle vibrates, and capable of performing optimal display depending on the vibration.

As described above, the invention has been described with reference to specific embodiments, but the invention is not limited to the embodiments, and a variety of modifications may be made in a range without departing from the technical concept of the invention.

This application is based on Japanese Patent Application No. 2016-248112, filed on Dec. 21, 2016, the disclosure of which is incorporated herein by reference.

EXPLANATION OF REFERENCES