Patent ID: 12207066

DETAILED DESCRIPTION

In order to make the technical problems, technical solutions, and advantages of the present disclosure clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

In order to realize the integration of sound and picture of display devices, an embodiment of the present disclosure provides a display device, as shown inFIG.1, wherein the display device includes:a display screen100, including a first display region110, a middle display region130, and a second display region120arranged sequentially along a first direction a;a plurality of sound-emitting units, which are arranged on a side facing away from the display screen100,wherein the plurality of the sound-emitting units include: a plurality of first sound-emitting units210, a plurality of second sound-emitting units220and a plurality of third sound-emitting units230, the orthographic projection of the first sound-emitting units210on the plane of the display screen100is located in the first display region110, the orthographic projection of the second sound-emitting units220on the plane of the display screen100is located in the second display region120, and the orthographic projection of the third sound-emitting units230on the plane of the display screen100is located in the third display region130;wherein the plurality of the first sound-emitting units210and the plurality of the second sound-emitting units220respectively include at least a sound-emitting unit emitting sound at a first frequency band, a sound-emitting unit emitting sound at a second frequency band, and a sound-emitting unit emitting sound at a third frequency band; wherein the first frequency band, the second frequency band, and the third frequency band increase in turn; and all of the plurality of the third sound-emitting units230are the sound-emitting units emitting sound at the second frequency band.

In the embodiments of the present disclosure, optionally, the first direction a is a horizontal direction, and can be a horizontal direction to the right; since the settings of the plurality of first sound-emitting units210, the plurality of second sound-emitting units220, and the plurality of third sound-emitting units230respectively correspond to the first display region110, the second display region120, and the middle display region130arranged sequentially along the first direction a, the first display region110, the middle display region130, and the second display region120respectively form a left channel play region, a middle channel play region, and a right channel play region of the display device; so as to form the whole display screen as a sound-emitting screen, that is to say, form the effect that the sound-emitting units are distributed and arranged on the whole display screen, and make sounds are emitted by the whole display screen.

In an embodiment of the present disclosure, optionally, the plurality of first sound-emitting units210correspond to the first display region110, the plurality of second sound units220correspond to the second display region120, and the plurality of third sound units230correspond to the middle display region130, respectively, in a uniform distribution.

In the embodiments of the present disclosure, the plurality of first sound-emitting units210and the plurality of second sound-emitting units220respectively include a sound-emitting unit emitting sound in a first frequency band, a sound-emitting unit emitting sound in a second frequency band, and a sound-emitting unit emitting sound in a third frequency band, wherein the first frequency band, the second frequency band, and the third frequency band increase in turn, and optionally, the first frequency band, the second frequency band, and the third frequency band respectively correspond to sounds in three frequency bands, namely, a high frequency band, a middle frequency band, and a low frequency band. In this way, by arranging sound-emitting units of three frequency bands, namely, a high frequency band, a middle frequency band, and a low frequency band in the left and right channel playing regions corresponding to the display screen respectively, the playing requirements of each frequency band can be met, so as to meet the user's requirements for the sound of each frequency band.

Furthermore, all of the plurality of third sound-emitting units230are sound-emitting units which emit sound in the second frequency band, wherein the second frequency band is the middle frequency band, and the plurality of third sound-emitting units230corresponding to the middle display region130can satisfy the playing requirements of the middle channel.

Therefore, by using the display device according to the embodiments of the present disclosure, the plurality of first sound-emitting units210corresponding to the first display region, the plurality of second sound-emitting units220corresponding to the second display region and the plurality of third sound-emitting units230corresponding to the middle display region are provided, and the sound-emitting units corresponding to different display regions satisfy different frequency band requirements, so that the whole display screen is formed as a sound-emitting screen, and the sound-emitting screen has a left channel playing region, a middle channel playing region, and a right channel playing region. When the display screen is used for image display, the left channel playing region, the middle channel playing region, and the right channel playing region of the whole display screen are used for sound playing, which can satisfy the playing requirements of integration of sound and picture.

In an embodiment of the present disclosure, optionally, the orthographic projections of the plurality of the third sound-emitting units230on the plane, where the display screen100is located, are evenly distributed in the middle display region130, each of the third sound-emitting units230corresponds to a sub-region of the middle display region.

The middle display region130is divided into a plurality of sub-regions, and the third sound-emitting unit230is respectively arranged in each sub-region, so that when the display screen displays an image, the corresponding third sound-emitting unit230can be controlled to emit sound according to the position of the human face image displayed on the image in the middle display region130, so as to realize a sound and picture integrated playing effect.

For example, the middle display region130includes M×N sub-regions, M and N are respectively positive integers, such as M and N are respectively 3, that is to say, the middle display region130includes 3×3 sub-regions, at least one third sound-emitting unit230is provided for each sub-region, and the third sound-emitting units230of each sub-region can emit sound individually so as to satisfy the playing requirements of the middle sound channel.

In the embodiments of the present disclosure, it should be noted that the first frequency band, the second frequency band, and the third frequency band respectively correspond to high, middle, and low frequency band sounds, and specifically correspond to the frequency ranges of the high, middle, and low frequency bands, which can be determined according to specific provisions in the industry, and are not limited herein.

In an embodiment of the present disclosure, optionally, each of the sound-emitting units includes an exciter and a vibration panel, respectively, wherein the exciter is mounted on the vibration panel, and the exciter drives the vibration panel to vibrate so as to generate sound.

In this manner, the vibration sound wave is transmitted to human ears by using the vibration panel as a vibration body. That is, when the sound-emitting unit emits sounds, the sound output can be realized by using the vibration panel as the vibration body without a loudspeaker and an earphone.

In an embodiment of the present disclosure, as shown inFIG.1, the display device includes a display panel300, one surface of the display panel is the display screen, wherein the display panel300includes a plurality of sub-panels, and one of the plurality of sub-panels is reused as the vibration panel.

By using the present embodiment, among the plurality of sub-panels of the display panel300, each of the sub-panels serves as a vibration body and is driven by the exciter to generate sound waves, thereby achieving sound output. Optionally, the plurality of sub-panels are arranged in one-to-one correspondence with a plurality of lasers.

By using the implementation structure for performing sound emitting by vibration of the sub-panel of the display panel300, the display device is capable of transmitting the sounds by vibration of the sub-panel without providing the loudspeaker and the earphone, and is configured as a screen sound-emitting technology. Compared with the display device provided with the earphone and the loudspeaker, the display device adopting the implementation structure can further improve a screen-to-body ratio and ensure a real full-screen effect.

Furthermore, when displaying an image, a display device using a screen to perform sound emitting can judge the sound-emitting position of the sound-emitting object on the image according to the displayed image, and control the sub-panel at a corresponding position to vibrate and emit sounds according to the determined sound-emitting position, so as to truly realize a sound and picture integrated playing effect.

In an embodiment of the present disclosure, optionally, the plurality of the sub-panels are combined to form the display panel.

Specifically, each sub-panel is formed as the sound-emitting unit in combination with the provided exciter, and the plurality of sub-panels are combined to form the display panel including a large-area display screen, and the display panel can achieve a screen sound-emitting effect while displaying the image.

Furthermore, by controlling the vibration frequency of the exciters to which the sub-panels are connected, and/or the sub-panels are made of different material properties, corresponding high, middle or low frequency sounds can be emitted.

In the display device according to the embodiment of the present disclosure, optionally, as shown inFIG.1, the areas of the first display region110and the second display region120are equal, and the area of the middle display region130is at least twice the area of the first display region110.

Optionally, making the area of the middle display region130far greater than the area of the first display region110and the second display region120, such that the first display region110and the second display region120are respectively arranged corresponding to the left and right side edges of the display region to form corresponding left channel playing region and right channel playing region; the middle display region130is used for displaying a main part of an output image, and is able to control a sub-region at a corresponding position to emit sound waves according to the sound emitting position on the output image in the middle display region130, so as to realize the sound and picture integrated playing effect

The display device according to the above-mentioned embodiments of the present disclosure uses a sound-emitting screen formed by splicing an excitation source and the plurality of sub-panels together, wherein the plurality of sound-emitting units formed by the excitation source and the sub-panels correspond to the first display region and the second display region of the display screen, respectively at least include the sound-emitting units emitting the high-frequency sound, a middle-frequency sound, and a low-frequency sound which correspond to a middle display region of the display screen, respectively are sound-emitting units for generating the middle-frequency sound, and can ensure the middle-frequency sound to be played while satisfying the sound playing of the left and right channels, so as to satisfy the user's requirements for sounds of each frequency band. On this basis, when the display screen is used for image display, the left channel playing region, the middle channel playing region, and the right channel playing region of the whole display screen are used for sound playing, which can satisfy the playing requirements of integration of sound and picture.

Embodiments of the present disclosure further provide a sound-emitting control method applied to any of the above-mentioned display devices, as shown inFIG.2, combined withFIG.1, the method includes:S210, acquiring video data and audio data of audio and video data to be output;S220, detecting a sound-emitting position of a sound-emitting object in a target image frame when the target image frame of the video data is displayed on the display screen, and determining the third sound-emitting unit corresponding to the sound-emitting position;S230, extracting a target sound-emitting signal corresponding to the sound-emitting object from an audio signal corresponding to the target image frame of the audio data, and converting the audio signal into a left channel signal, a middle channel signal, and a right channel signal; andS240, outputting the left channel signal to the plurality of the first sound-emitting units, outputting the right channel signal to the plurality of the second sound-emitting units, outputting the middle channel signal to at least one of the third sound-emitting units, and outputting the target sound-emitting signal to the third sound-emitting unit corresponding to the display position.

By using the sound-emitting control method described in the embodiments of the present disclosure, the display screen divided into the left channel playing region, the right channel playing region, and a middle channel playing region are used, when audio and video data are output, the video data and audio data are separated, sound-emitting object are detected and positioned on a target image frame, and an object sound-emitting signal is detected and separated on an audio signal corresponding to the target image frame, so that the left channel signal is output to the plurality of first sound-emitting units corresponding to the first display region, and the right channel signal is output to the plurality of second sound-emitting units corresponding to the second display region; and middle channel signals are output to the plurality of third sound-emitting units corresponding to the middle display region, and the corresponding third sound-emitting units are controlled to output target sound-emitting signals corresponding to the sound-emitting object images according to the position of the located sound-emitting object images, so that the requirements of sound-picture integration can be met.

Specifically, a full-screen sound-emitting display screen can be realized, the plurality of first sound-emitting units and the plurality of second sound-emitting units corresponding to the left and right channel playing regions are respectively used for playing the left and right channel signals of audio data, and the plurality of third sound-emitting units corresponding to the middle channel playing region are used for playing the middle channel signals of audio data and locating corresponding target sound-emitting signals according to the sound-emitting object image.

It should be noted that since the middle sound channel is often used for playing main sound signals in all audio, such as person dialogue, etc. that is to say, most of the sound information about the person in the audio data is the middle frequency signal, the third sound-emitting unit used for emitting the middle frequency sound is used to play the middle sound channel signal, and the left and right sound channels are generally used for playing audio signals such as environment and sound effect enhancement, etc. so as to enhance the sound signal played in the middle channel, and there are signals in each frequency band; therefore, the plurality of first sound-emitting units and the plurality of second sound-emitting units which emit high, middle, and low frequency sounds are used to play the left and right sound channel signals.

It should be noted that, in the embodiments of the present disclosure, the sound-emitting object in the video data includes but is not limited to being able to include only human face images, animal head images, and sound-emitting machines, etc.

In one embodiment, in step S220, detecting the sound-emitting position of the sound-emitting object in the target image frame when the target image frame of the video data is displayed on the display screen, as shown inFIG.3, specifically includes:performing sound-emitting object detection and sound-emitting object positioning on the image in the video data according to the video data and audio data in the audio-video data extracted in step S210;specifically, by performing image recognition analysis on the target image frame of the video data, a sound-emitting object known to have a specific shape in the target image frame can be analyzed, such as a human face, an animal head, a sound-emitting machine and so on, and on this basis, further image recognition can be used to determine the position of the sound-emitting object in the target image frame.

In addition, in the process of performing sound-emitting object detection and sound-emitting object positioning on the image in the video data, by performing channel separation on the audio data and performing sound-emitting signal detection on each separated sub-channel, a sound-emitting signal of the sound-emitting object in the target image frame can be detected; matching the detected sound-emitting signal with the identified sound-emitting object in the video data enables the relationship between the sound-emitting object and the corresponding sound-emitting signal to be determined, thereby determining the sound-emitting position of the sound-emitting object in the target image frame.

It should be noted that, by using the present embodiment, the sound-emitting position of the sound-emitting object in the target image frame is detected, and since image recognition analysis needs to be performed on the target image frame to determine the sound-emitting object in the target image frame, it is limited to be applied to a scene capable of determining the sound-emitting object in the target image frame.

Referring toFIG.3, in the present embodiment, by using the sound-emitting control method according to an embodiment of the present disclosure, the method includes: audio and video are separated, sound-emitting object detection and sound-emitting object positioning are performed by using the separated video data, and channel separation and sound-emitting signal detection are performed according to the separated audio data; the sound-emitting position of the sound-emitting object in the target image frame is determined by performing the sound-emitting object positioning and the sound-emitting signal detection, and the third sound-emitting unit corresponding to the sound-emitting position are determined; after the sound-emitting object is located and the sound signal is detected, channel regeneration is performed, the left channel signal, the middle channel signal, and the right channel signal are separated, and the left channel signal is played in a left channel region, the right channel signal is played in a right channel region, and the middle channel signal is played in a middle channel region respectively.

Optionally, in one embodiment, when channel separation and object sound-emitting detection are performed, the separated audio signals generally have 2.0, 2.1, 5.1 channels, etc. wherein the 2.0 channel is relatively common; when channel separation is performed, the audio signal of the above-mentioned initial channels is separated into various sub-channels; when object sound-emitting object is detected, whether the object sound-emitting signal exists in each sub-channel is detected respectively; optionally, the method for detecting an object sound-emitting signal can use a detection model trained by TensorFlow, such as when human voices is detected; and human voice detection was performed by using a spleeter library in ffmpeg as the human voice detection model.

In an embodiment of the present disclosure, optionally, in another embodiment, in step S220, when the target image frame of the video data is displayed on the display screen, detecting the sound-emitting position of the sound-emitting object of the target image frame in the target image frame, as shown inFIG.4, which includes:separating the audio data into left channel data and right channel data;performing sound image positioning calculation according to the left channel data and the right channel data to determine sound image positioning information; anddetermining the sound-emitting position of the sound-emitting object in the target image frame according to the sound image positioning information and a result of frame difference processing on the target image frame of the video data.

Specifically, as shown inFIG.5, the meaning of the sound image is: when two loudspeakers are used for stereo playing, the listener does not perceive the presence of two sound sources, but perceives as if a spatial point between the two loudspeakers emit sounds, which is regard as the sound image.

When the signals of the left and right channels are the same, the sound image is in the middle position of the left and right channels;when there is a time difference and/or intensity difference between the signals of the left and right channels, the sound image will shift; andthe positioning of the sound image is achieved by the time difference and/or the intensity difference between the signals of the left and right channels.

Based on this, the sound image positioning information includes an audio time difference and an intensity level difference between a left channel signal and a right channel signal, and in an embodiment of the present disclosure, performing sound image positioning calculation according to the left channel data and the right channel data to determine sound image positioning information, which includes:as shown inFIG.6, performing cross-correlation calculation by using the left channel signal and the right channel signal to determine the audio time difference of the target image frame;as shown inFIG.7, using the left channel signal and the right channel signal, respectively performing frame signal average calculation and left and right channel intensity calculation, and determining the intensity level difference between the left channel signal and the right channel signal; anddetermining the sound image location information according to the audio time difference and the intensity level difference.

In the disclosed embodiment, optionally, the separated left and right channel signals in the audio data are Pulse Code Modulation (PCM) signals.

In addition, according toFIG.6, before performing the cross-correlation calculation, it is also necessary to respectively perform signal normalization on the left channel signal and signal grouping on the right channel signal in sequence;performing the cross-correlation calculation includes: the left channel signal and the right channel signal are respectively calculated by ITD and analyzed by a cross-correlation function to determine the audio time difference of the target image frame.

Specifically, from the determined audio time difference, the sound image position in the lateral direction of the display screen can be determined according to a corresponding relationship between the sound image straight line positioning percentage and the audio time difference represented inFIG.8.

In the embodiment of the present disclosure, combined withFIG.7, after the frame signal average calculation and the intensity calculation between the left and right channels are performed and the intensity level difference between the left channel signal and the right channel signal is determined, the sound image position in the longitudinal direction of the display screen can be determined according to the corresponding relationship between the sound image straight line positioning percentage and the intensity level difference between the left and right channel signals as shown inFIG.9.

According to the above-mentioned principle, the audio time difference between the left channel signal and the right channel signal and the intensity level difference calculated in the above-mentioned manner are used to determine the sound image positioning information of the sound-emitting object.

On this basis, the motion state of the current video frame is calculated by using a frame difference method, and the audio time difference and intensity level difference obtained by the above-mentioned process are combined to determine the sound-emitting position of the sound-emitting object in the target image frame.

In the above-mentioned embodiments, specifically, the audio signal is separated into left and right channel signals, and format conversion is performed to obtain a time stamp and signal data of each audio frame; on this basis, the cross-correlation function and the calculation of the relative position between the signals are used to calculate the sound image positioning and obtain the horizontal sound image position on the display screen. On the basis of the calculated horizontal position, the longitudinal sound image position on the screen can be obtained, and then the frame difference processing can be used to determine the sound-emitting position of the sound-emitting object in the object sound-emitting signal. By using the present embodiment, it is not necessary to perform image analysis according to the target image frame obtained by separation to obtain the sound-emitting object, and therefore it is possible to analyze the sound-emitting position of the sound-emitting object on the basis of uncertainty of the sound-emitting object.

In step S230, when the target sound-emitting signal of the audio signal corresponding to the target image frame in the audio data is extracted, the target sound-emitting signal can be determined according to the above-mentioned sound image positioning information.

Optionally, when the sound-emitting object is human, a human voice model can be used to detect each sub-channel signal to determine a target sound-emitting signal.

Furthermore, in the embodiments of the present disclosure, by using the above-mentioned two embodiments, after determining the sound-emitting position of the sound-emitting object of the target image frame in the target image frame, the playing region of the display screen can be determined according to the sound-emitting position, and according to the corresponding relationship between the plurality of third sound-emitting units and the display region, that is to say, the third sound-emitting units corresponding to the sound-emitting position can be determined, that is to say, the target sound-emitting channel is determined.

In the embodiments of the present disclosure, furthermore, by using the above-mentioned two embodiments, after determining the sound-emitting position of the sound-emitting object of the target image frame in the target image frame, the audio data is separated into the left channel, the right channel, the middle channel, and the target sound-emitting channel by performing the channel regeneration, and when playing, the left and right channels are played in the left and right channel playing region, the middle channel is played in the middle channel playing region, and the target sound-emitting channel is used for playing the target sound-emitting signal of the sound-emitting object. Specifically, when the target sound channel coincides with the other channel position channels, the channels are merged and then are used for playing.

In general, the conversion manner between channels can be changed from 2 channel to 3 channel or from 2 channel to multiple channels.

In another embodiment of the present disclosure, by performing the channel regeneration, the audio data is separated into the left channel, the right channel, and the target sound-emitting channel, and furthermore, with regard to each sub-region of the middle channel playing region, each sub-region is respectively provided with at least one third sound-emitting unit; the middle channel signal of the audio data is respectively divided into the plurality of sub-channel signals, and each sub-channel signal corresponds to the channel of one sub-region; for example, when the middle channel playing region includes sub-regions 1 to 9, a total of nine sub-regions, the middle channel signal of the audio data is further divided into nine sub-channels by the channel regeneration; and each sub-channel signal respectively corresponds to the sub-region, and the third sound-emitting unit located in the sub-region is used for playing the corresponding sub-channel signal.

Specifically, when the target sound channel coincides with the other channel position channels, the channels are merged and then are used for playing.

Therefore, in the embodiments of the present disclosure, optionally, in one embodiment, in step S240, the step of outputting the middle channel signal to at least one of the third sound-emitting units includes:converting the middle channel signal into a plurality of sub-channel signals, wherein each sub-channel signal corresponds to one of the third sound-emitting units;sending each sub-channel signal to the corresponding third sound-emitting unit respectively.

Optionally, in one embodiment, in step S240, the step of outputting the middle channel signal to at least one of the third sound-emitting units includes:the orthographic projection of the third sound-emitting units, where the middle channel signal is outputted, on the plane of the display screen is located in the middle position of the middle display region.

In another embodiment, as described above, when the third sound-emitting units which output the middle channel signal and the target sound-emitting signal are the same sound-emitting unit, the middle channel signal and the target sound-emitting signal are combined and output to the corresponding third sound-emitting unit.

It should be noted that, in embodiments of the present disclosure, optionally, the target sound-emitting channels which are used for playing the target sound-emitting signals can include a third sound-emitting unit which is determined based on the sound-emitting position of the sound-emitting object described above.

In another embodiment, the target sound-emitting channels which are used for playing the target sound-emitting signal can include at least two third sound-emitting units, wherein the at least two third sound-emitting units are located in a part region of the corresponding middle display region and include the third sound-emitting unit determined according to the sound-emitting position of the above-mentioned sound-emitting object; in addition, all the third sound-emitting units can also be included, and in the present embodiment, when the target sound-emitting signals corresponding to the sound-emitting object are played by at least two third sound-emitting units, the sound played by the third sound-emitting unit determined by the sound-emitting position of the sound-emitting object can be greater than the sound played by the other third sound-emitting units, so that the requirements of sound-picture integration can be met.

According to another aspect of embodiments of the present disclosure, the sound-emitting control device is also provided, wherein the sound-emitting control device is applied to any of the above-mentioned display devices, as shown inFIG.10, the device includes:a data acquisition module1010, which is used for acquiring the video data and the audio data of audio and video data to be output;a detection module1020, which is used for detecting a sound-emitting position of a sound-emitting object in a target image frame when the target image frame of the video data is displayed on the display screen, and determining the third sound-emitting unit corresponding to the sound-emitting position;a conversion module1030, which is used for extracting a target sound-emitting signal corresponding to the sound-emitting object from an audio signal corresponding to the target image frame of the audio data, and converting the audio signal into a left channel signal, a middle channel signal, and a right channel signal; andan output module1040, which is used for outputting the left channel signal to the plurality of the first sound-emitting units, outputting the right channel signal to the plurality of the second sound-emitting units, outputting the middle channel signal to at least one of the third sound-emitting units, and outputting the target sound-emitting signal to the third sound-emitting unit corresponding to the display position.

Optionally, in the sound-emitting control device, wherein the step that the detection module1020detects the sound-emitting position of the sound-emitting object in a target image frame when the target image frame of the video data is displayed on the display screen includes:separating the audio data into left channel data and right channel data;performing sound image positioning calculation according to the left channel data and the right channel data to determine sound image positioning information; anddetermining the sound-emitting position of the sound-emitting object in the target image frame according to the sound image positioning information and a result of frame difference processing on the target image frame of the video data.

Optionally, in the sound-emitting control device, wherein in the step that the output module440outputs the middle channel signal to at least one of the third sound-emitting units:the orthographic projection of the third sound-emitting units, where the middle channel signal is outputted, on the plane of the display screen is located in the middle position of the middle display region.

Optionally, the sound-emitting control device, wherein the step that the output module440outputs the middle channel signal to at least one of the third sound-emitting units includes:converting the middle channel signal into a plurality of sub-channel signals, wherein each sub-channel signal corresponds to one of the third sound-emitting units;sending each sub-channel signal to the corresponding third sound-emitting unit respectively.

Optionally, the sound-emitting control device, wherein when the third sound-emitting units which output the middle channel signal and the target sound-emitting signal are the same sound-emitting unit, the output module440combines the middle channel signal and the target sound-emitting signal and outputs the combined signal to the corresponding third sound-emitting unit.

While the foregoing is directed to alternative embodiments of the present disclosure, it should be understood by those skilled in the art that various improvements and modifications may be made without departing from the principle of the present disclosure, and theses improvement and modifications shall fall within the scope of protection of the present disclosure.