Information processing apparatus and information processing method for generating and processing a file including speech waveform data and vibration waveform data

Provided is an information processing apparatus including a file generation unit that generates a file including speech waveform data and vibration waveform data. The file generation unit cuts out waveform data in a to-be-synthesized band from first speech data, synthesizes waveform data extracted from a synthesizing band of vibration data with the to-be-synthesized band to generate second speech data, and encodes the second speech data to generate the file.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a U.S. National Phase of International Patent Application No. PCT/JP2017/038118 filed on Oct. 23, 2017, which claims priority benefit of Japanese Patent Application No. JP 2016-235574 filed in the Japan Patent Office on Dec. 5, 2016. Each of the above-referenced applications is hereby incorporated herein by reference in its entirety.

TECHNICAL FIELD

BACKGROUND ART

Various devices for generating vibration are developed nowadays. In addition, a device equipped with the device as described above and using vibration as one of information transmission means becomes widespread. In one example, Patent Literature 1 discloses a technique of presenting internal information of a terminal to a user by vibration.

CITATION LIST

Patent Literature

DISCLOSURE OF INVENTION

Technical Problem

Meanwhile, the device using vibration as described above is also conceived to perform communication regarding vibration information with other devices, in addition to processing that is completed within the device as described in Patent Literature 1. However, the communication relating to vibration information has not been standardized yet, and in a case where vibration information is transmitted and received together with speech information and moving image information, data communication traffic is conceived to increase.

In view of this, the present disclosure provides a novel and improved information processing apparatus, information processing method, and program, capable of implementing unified communication of vibration information while reducing data communication traffic.

Solution to Problem

According to the present disclosure, there is provided an information processing apparatus including: a file generation unit configured to generate a file including speech waveform data and vibration waveform data. The file generation unit cuts out waveform data in a to-be-synthesized band from first speech data, synthesizes waveform data extracted from a synthesizing band of vibration data with the to-be-synthesized band to generate second speech data, and encodes the second speech data to generate the file.

In addition, according to the present disclosure, there is provided an information processing method including: generating, by a processor, a file including speech waveform data and vibration waveform data. The generating the file further includes cutting out waveform data in a to-be-synthesized band from first speech data, synthesizing waveform data extracted from a synthesizing band of vibration data with the to-be-synthesized band to generate second speech data, and encoding the second speech data to generate the file.

In addition, according to the present disclosure, there is provided a program causing a computer to function as an information processing apparatus including: a file generation unit configured to generate a file including speech waveform data and vibration waveform data. The file generation unit cuts out waveform data in a to-be-synthesized band from first speech data, synthesizes waveform data extracted from a synthesizing band of vibration data with the to-be-synthesized band to generate second speech data, and encodes the second speech data to generate the file.

In addition, according to the present disclosure, there is provided an information processing apparatus including: a file analysis unit configured to acquire speech waveform data and vibration waveform data from a file including the speech waveform data and the vibration waveform data. The file analysis unit separates speech data obtained by decoding the file on the basis of a to-be-synthesized band to acquire the speech waveform data and the vibration waveform data.

In addition, according to the present disclosure, there is provided an information processing method including: acquiring, by a processor, speech waveform data and vibration waveform data from a file including the speech waveform data and the vibration waveform data. The acquiring further includes separating speech data obtained by decoding the file on the basis of a to-be-synthesized band to acquire the speech waveform data and the vibration waveform data.

In addition, according to the present disclosure, there is provided a program causing a computer to function as an information processing apparatus including: a file analysis unit configured to acquire speech waveform data and vibration waveform data from a file including the speech waveform data and the vibration waveform data. The file analysis unit separates speech data obtained by decoding the file on the basis of a to-be-synthesized band to acquire the speech waveform data and the vibration waveform data.

Advantageous Effects of Invention

According to the present disclosure as described above, it is possible to implement unified communication of vibration information while reducing data communication traffic.

MODE(S) FOR CARRYING OUT THE INVENTION

Moreover, the description will be given in the following order.1. Embodiment1.1. Overview of embodiment1.2. System configuration example1.3. Functional configuration example of file generation device101.4. Functional configuration example of playback device201.5. Details on file generation and file analysis1.6. Modification of to-be-synthesized band1.7. Operations of file generation device10and playback device202. Hardware configuration example3. Concluding remarks

1.1. Overview of Embodiment

An overview of an embodiment of the present disclosure is now described. As described above, various devices that generate vibration are used nowadays, and many vibration devices intended for mobile terminals are developed. Furthermore, a linear resonant actuator (LRA), a piezoelectric (voltage) actuator, or the like capable of reproducing various vibration patterns is also widespread nowadays, and so it is conceived that communication of vibration information would be actively progressed from now on.

While, the communication of vibration information has not been standardized yet, unlike the communication of speech information and image information. For this reason, a format for allowing vibration information to transmitting and receiving easily is necessary. However, in a case of newly generating a vibration format, it is important to synchronize vibration information with the speech information and moving image information transmitted and received simultaneously with the vibration information.

FIGS. 1A and 1Bare diagrams illustrated to describe synchronization between the vibration information and the speech and moving image information.FIG. 1Aillustrates a moving image obtained by capturing a golf shot and speech information SO1and vibration information VO1outputted accompanying the moving image. Here, in the example illustrated inFIG. 1A, the output timings of moving image, speech information SO1, and vibration information VO1are synchronized with each other, so the user is able to view the moving image without uncomfortable feeling.

On the other hand, the example illustrated inFIG. 1Billustrates a case where the output timings of the moving image and the speech information SO1are synchronized with each other but are not synchronized with that of the vibration information VO1. Referring toFIG. 1B, the vibration information VO1is output after the action of golf shot is completed and the player has left the video. Such output deviation is more likely to give a sense of incompatibility to the user, so it is desirable to provide measures to prevent the output deviation.

Here, in one example, synchronization between moving image, speech, and vibration using a time stamp is conceived as an approach to prevent such output deviation as described above. However, it is necessary for synchronization using time stamps to set a new own vibration file format standard, and the burden such as generation of the time stamps increases. Furthermore, it is necessary to transmit and receive vibration information in addition to moving image information and speech information at the time of communication, which leads to increase in data communication traffic.

The information processing apparatus, the information processing method, and the program according to the present embodiment are conceived focusing on the above points, and it is possible to present synchronized vibration to the user while reducing data communication traffic by embedding vibration information in a general-purpose file format. To this end, the technical idea according to the present embodiment has one feature of cutting out waveform data in a to-be-synthesized band from speech data and synthesizing waveform data extracted from a synthesizing band of vibration data with the to-be-synthesized band. The functions of the information processing apparatus, information processing method, and program according to the present embodiment and the effects achieved from the functions are described below in detail.

1.2. System Configuration Example

A system configuration example according to the present embodiment is now described.FIG. 2is a diagram illustrating a system configuration example according to the present embodiment. Referring toFIG. 2, an information processing system according to the present embodiment includes a file generation device10and a playback device20. In addition, the file generation device10and the playback device20are connected via a network30so that they can communicate with each other.

File Generation Device10

The file generation device10according to the present embodiment is an information processing apparatus that generates a file including speech waveform data and vibration waveform data. As described above, the file generation device10according to the present embodiment has one feature of generating the above-mentioned file by embedding vibration waveform data in a general-purpose speech file format or moving image file format.

The file generation device10according to the present embodiment may have a function of generating the above-mentioned file on the basis of the collected moving image information, speech information, and vibration information and transmitting it to the playback device20. In this case, the file generation device10according to the present embodiment may be, in one example, a smartphone, a tablet computer, a mobile phone, a personal computer (PC), a wearable device, or the like. The file generation device10is capable of encoding the captured moving image and the vibration information collected together with the moving image to generate a file and transmitting the file to the playback device20.

Further, the file generation device10according to the present embodiment may be a content server that generates, saves, and delivers the content including speech information and vibration information. In this case, the file generation device10is capable of generating the file on the basis of the collected moving image information, speech information, and vibration information, and delivering the file in response to a request from the playback device20. In addition, the file generation device10according to the present embodiment may generate the above-mentioned file by embedding vibration waveform data in an existing speech file or moving image file.

The playback device20according to the present embodiment is an information processing apparatus that outputs sound and vibration on the basis of speech waveform data and vibration waveform data obtained by decoding a file received from the file generation device10.

In addition, the playback device20according to the present embodiment may play back a moving image together with sound and vibration. The playback device20according to the present embodiment may be, in one example, a smartphone, a tablet computer, a mobile phone, a PC, a wearable device, or the like.

The network30has a function of connecting the file generation device10and the playback device20. The network30may include the public line network such as the Internet, telephone line networks, and satellite communication networks, the various local area networks (LANs) and wide area networks (WANs) including Ethernet (registered trademark), or the like. In addition, the network30may include a leased line network such as Internet protocol-virtual private network (IP-VPN). In addition, the network30may include a wireless communication network such as Wi-Fi (registered trademark) and Bluetooth (registered trademark).

1.3. Functional Configuration Example of File Generation Device10

A functional configuration example of the file generation device10according to the present embodiment is now described. Referring toFIG. 2, the file generation device10according to the present embodiment includes a collection unit110, a file generation unit120, and a communication unit130.

The collection unit110has a function of collecting speech information, moving image information, vibration information, or the like. For this purpose, the collection unit110according to the present embodiment is provided with a microphone, an imaging device, a vibration sensor, or the like.

File Generation Unit120

The file generation unit120has a function of generating a file including speech waveform data and vibration waveform data. In this event, the file generation unit120according to the present embodiment initially cuts out waveform data in a to-be-synthesized band from the first speech data. Here, the first speech data may be speech data collected by the collection unit110or speech data obtained by decoding an existing file. Then, the file generation unit120synthesizes waveform data extracted from a synthesizing band of the vibration data with the to-be-synthesized band to generate second speech data. Subsequently, the file generation unit120is capable of encoding the generated second speech data to generate a file including the speech waveform data and the vibration waveform data.

In this event, the file generation unit120according to the present embodiment may generate the above-mentioned file in a format, in one example, such as a widely used MPEG-1 Audio Layer-3 (MP3) format, MP4 format, or the like. In other words, the file generation unit120according to the present embodiment is capable of embedding the vibration waveform data in a predetermined band of the speech data of the existing file format.

The function of the file generation unit120according to the present embodiment as described above makes it possible to easily transmit and receive vibration information synchronized with speech and a moving image without preparing its own format. In addition, the use of the existing file format makes it possible to reduce data communication traffic. Moreover, details on the function of the file generation unit120according to the present embodiment are described later separately.

The communication unit130has a function of performing information communication with the playback device20. Specifically, the communication unit130according to the present embodiment transmits the file generated by the file generation unit120to the playback device20. In addition, in a case where the file generation device10according to the present embodiment is a content server or the like, the communication unit130receives a file download request or the like from the playback device20.

The functional configuration example of the file generation device10according to the present embodiment is described above. Moreover, the configuration described above with reference toFIG. 2is merely an example, and the functional configuration of the file generation device10according to the present embodiment is not limited to this example. The file generation device10may be further provided with other components than those mentioned above. In one example, the file generation device10may further have an input unit that accepts an input operation by a user, and an output unit that outputs visual information and auditory information. The functional configuration of the file generation device10according to the present embodiment is flexibly modifiable.

1.4. Functional Configuration Example of Playback Device20

A functional configuration example of the playback device20according to the present embodiment is now described. Referring toFIG. 2, the playback device20according to the present embodiment includes an output unit210, a file analysis unit220, and a communication unit230.

The output unit210has a function of outputting speech, moving images, vibration, or the like. The output unit210according to the present embodiment may perform output based on speech waveform data and vibration waveform data acquired by the file analysis unit220described later. For this purpose, the output unit210according to the present embodiment includes various devices that output visual information, auditory information, and tactile information.

The output unit210includes, in one example, a cathode ray tube (CRT) display device, a liquid crystal display (LCD) device, an organic light emitting diode (OLED) device, a touch panel, or the like.

Further, the output unit210includes, in one example, an audio output amplifier, a loudspeaker, or the like. Moreover, the output unit210according to the present embodiment may include a plurality of loudspeakers that support stereo reproduction.

Further, the output unit210includes a vibration device such as LRA, piezoelectric actuator, or eccentric motor, and an integrated circuit (IC) that drives the vibration device. Moreover, the output unit210according to the present embodiment may be provided with a plurality of vibration devices corresponding to the plurality of loudspeakers.

File Analysis Unit220

The file analysis unit220has a function of acquiring speech waveform data and vibration waveform data from the file generated by the file generation device10. In this event, the file analysis unit220according to the present embodiment has one feature of separating speech data obtained by decoding the file on the basis of the to-be-synthesized band to acquire the speech waveform data and the vibration waveform data. Moreover, details on the function of the file analysis unit220according to the present embodiment are described later separately.

The communication unit230has a function of performing information communication with the file generation device10. Specifically, the communication unit230according to the present embodiment receives the file including the speech waveform data and the vibration waveform data from the file generation device10. In addition, in the case where the file generation device10is a content server or the like, the communication unit230may transmit a content download request to the file generation device10.

The functional configuration example of the playback device20according to the present embodiment is described above. Moreover, the above-described functional configuration described with reference toFIG. 2is merely an example, and the functional configuration of the playback device20according to the present embodiment is not limited to this example. The playback device20may be further provided with other components than those listed above. In one example, the playback device20may further include an input unit or the like that accepts an input operation by the user. The functional configuration of the playback device20according to the present embodiment is flexibly modifiable.

1.5. Details on File Generation and File Analysis

Details on file generation and file analysis according to the present embodiment are now described. As described above, the file generation device10according to the present embodiment is capable of generating the second speech data by cutting out waveform data in the to-be-synthesized band from the first speech data and by synthesizing waveform data extracted from the synthesizing band of the vibration waveform data with the to-be-synthesized band.

Generation and Analysis of Monophonic Speech File

The generation and analysis of a monophonic speech file according to the present embodiment are first described. The file generation device10according to the present embodiment is capable of generating a file corresponding to monophonic speech, and the playback device20according to the present embodiment is capable of acquiring monophonic speech data and vibration data by analyzing the file.

FIG. 3is a diagram illustrated to describe generation of a monophonic speech file according to the present embodiment. As illustrated inFIG. 3, the file generation unit120of the file generation device10initially cuts out waveform data in the to-be-synthesized band from first speech data SD0and acquires speech waveform data Sb1. In addition, the file generation unit120extracts vibration waveform data Vb1in the synthesizing band from vibration data VD0. Moreover, the first speech data SD0may be speech data collected by the collection unit110or may be speech data acquired by decoding the existing file.

Here, the to-be-synthesized band and the synthesizing band according to the present embodiment may be a bass frequency band of 0 Hz to a predetermined frequency or less. In one example, as illustrated inFIG. 3, the file generation unit120may cut out the band of 0 Hz to 200 Hz from the first speech data SD0and may extract the vibration waveform data Vb1of 0 Hz to 200 Hz from the waveform data VD0. In other words, the file generation unit120according to the present embodiment is capable of cutting out speech waveform data of a bass frequency band that is generally difficult to perceive it from the first speech data SD0, and is capable of extracting vibration waveform data of a band that is easy to perceive it as vibration from the vibration data VD0.

Then, the file generation unit120according to the present embodiment synthesizes the synthesizing band extracted from the vibration data with the first speech data obtained by cutting off the to-be-synthesized band to generate second speech data SD1. In other words, the file generation unit120synthesizes waveform data Sb1in the band of 200 Hz or more that is extracted from the first speech data SD0and the synthesizing band of 0 Hz to 200 Hz that is extracted from the vibration data VD0to generate the second speech data SD1.

Further, the file generation unit120encodes the generated second speech data SD1to generate a file FM1. In this event, the file generation unit120according to the present embodiment is capable of generating the file FM1using a general-purpose format such as MP3 or MP4.

The analysis of the monophonic speech file by the playback device20is now described.FIG. 4is a diagram illustrated to describe analysis of a monophonic speech file by the playback device20according to the present embodiment. As illustrated inFIG. 4, the communication unit230of the playback device20initially receives the file FM1generated by the file generation device10. In this event, the communication unit230may receive the file FM1actively transmitted by, in one example, a user's operation from the file generation device10, or may receive the file FM1by making a download request to the file generation device10.

Then, the file analysis unit220according to the present embodiment decodes the received file FM1to acquire the speech data SD1. Furthermore, the file analysis unit220separates the speech data SD1on the basis of the to-be-synthesized band and acquires speech data SD2for playback and vibration data VD2for playback. In other words, the file analysis unit220according to the present embodiment is capable of acquiring the speech data SD2for playback and the vibration data VD2for playback by separating the speech data SD1extracted from the file FM1into the band of 0 Hz to 200 Hz and the band of 200 Hz or more.

In this event, as illustrated inFIG. 4, the speech data SD2for playback includes the speech waveform data Sb1of 200 Hz or more, and the vibration data VD2for playback includes vibration waveform data Vb1of 0 Hz to 200 Hz. The speech data SD2for playback and the vibration data VD2for playback, which are acquired by the file analysis unit220, are transferred to the output unit210and are output as auditory information and tactile information, respectively.

Generation and Analysis of Stereophonic Speech File

The generation and analysis of a stereophonic speech file according to the present embodiment are now described. The file generation device10according to the present embodiment is capable of generating a file corresponding to stereophonic speech, and the playback device20according to the present embodiment is capable of acquiring stereophonic speech data and vibration data by analyzing the file.

In this event, the file generation unit120according to the present embodiment may generate third speech data by adding waveform data in the to-be-synthesized band being cut out from two pieces of the first speech data and synthesizing the added waveform data with the to-be-synthesized band in one of two pieces of the first speech data. In addition, the file generation unit120may generate the second speech data by synthesizing the waveform data in the synthesizing band that is extracted from the vibration data with the to-be-synthesized band in the other one of two pieces of the first speech data. The file generation unit120is capable of encoding the second speech data and the third speech data to generate a stereophonic speech file.

FIG. 5is a diagram illustrated to describe generation of a stereophonic speech file according to the present embodiment. The file generation unit120initially separates two first speech data SDL0and SDR0on the basis of the to-be-synthesized band. Here, the first speech data SDL0and SDR0may be speech data corresponding to the left and right loudspeakers, respectively.

Further, the file generation unit120extracts the vibration waveform data Vb1in the synthesizing band from the vibration data VD0. Here, the to-be-synthesized band and the synthesizing band each may be the band of 0 Hz to 200 Hz.

Then, the file generation unit120adds speech waveform data SbL2and SbR2in the to-be-synthesized band that are respectively extracted from the first speech data SDL0and SDR0to obtain added speech waveform data SbM1. In this event, the file generation unit120synthesizes the added speech waveform data SbM1with speech waveform data SbR1in the band of 200 Hz or more that is extracted from the first speech data SDR0to generate third speech waveform data SDR1.

On the other hand, the file generation unit120synthesizes speech waveform data SbL1in the band of 200 Hz or more that is extracted from the first speech data SDL0and the vibration waveform data Vb1in the band of 0 Hz to 200 Hz that is extracted from the vibration data VD0, and so synthesizes speech waveform data SDL1.

In other words, the file generation unit120according to the present embodiment is capable of generating two pieces of speech data on the basis of two pieces of the first speech data and one piece of the vibration data. In this event, one of the generated two pieces of speech data includes the vibration waveform data extracted from the waveform data, and the other speech data includes added waveform data in the bass frequency band that is extracted from the two pieces of first speech data. The function of the file generation unit120according to the present embodiment as described above makes it possible to add information of the speech waveform data in the bass frequency band and includes it to one of the speech data, thereby preventing reduction in sound quality.

Further, the file generation unit120encodes the generated third speech data SDR1and second speech data SDL1to generate the file FM1, and transmits the generated file FM1to the playback device20.

The analysis of a stereophonic speech file by the playback device20is now described.FIG. 6is a diagram illustrated to describe analysis of a stereophonic speech file by the playback device20according to the present embodiment. The file analysis unit220according to the present embodiment initially decodes the file FM1received by the communication unit230to acquire the speech data SDR1and SDL1.

Then, the file analysis unit220according to the present embodiment acquires two pieces of speech data SDR2and SDR1for play back and one piece of vibration data VD2for playback on the basis of the acquired two pieces of speech data SDR1and SDL1. In this event, the file analysis unit220extracts a to-be-synthesized band of one of the two pieces of speech data obtained by decoding the file to acquire vibration waveform data. In addition, the file analysis unit220acquires speech waveform data obtained by synthesizing waveform data of the to-be-synthesized band duplicated from the other speech data with the to-be-synthesized band of the one.

In other words, the file analysis unit220according to the present embodiment separates the speech data SDL1obtained by decoding the file FM1on the basis of the to-be-synthesized band to acquire the vibration data VD2for playback including the vibration waveform data Vb1. In addition, the file analysis unit220synthesizes the added speech waveform data SbM1in the to-be-synthesized band of the speech data SDR1obtained by decoding the file FM1with the speech waveform data SbL1in the band of 200 Hz or more of the speech data SDRL1to acquire speech data SDL2for left playback. In addition, the file analysis unit220divertingly uses the speech data SDR1obtained by decoding the file FM1as the speech data SDR2for right playback.

The function of the file analysis unit220according to the present embodiment as described above makes it possible to acquire speech data for left and right playback with sound quality secured and vibration data from the two pieces of speech data extracted by decoding the file FM1.

Generation and Analysis of Stereophonic Speech Multi-Vibration File

The generation and analysis of a stereophonic speech multi-vibration file according to the present embodiment are now described. The file generation device10according to the present embodiment generates a file corresponding to stereophonic speech and multi-vibration, and the playback device20according to the present embodiment is capable of analyzing the file to generate stereophonic speech data and a plurality of pieces of vibration data corresponding to the stereophonic speech data.

FIG. 7is a diagram illustrated to describe generation of the stereophonic speech multi-vibration file according to the present embodiment. The file generation unit120initially extracts the speech waveform data SbL1and SbR1in the band of 200 Hz or more respectively from the two first speech data SDL0and SDR0. In addition, the file generation unit120extracts the vibration waveform data VbL1and VbR1in the band of 0 Hz to 200 Hz respectively from the two vibration waveform data VDL0and VDR0.

Subsequently, the file generation unit120synthesizes the extracted speech waveform data SbR1with vibration waveform data VbR1to generate the second speech data SDR1. Similarly, the file generation unit120synthesizes the speech waveform data SbL1with the vibration waveform data VbL1to generate the second speech data SDL1. In addition, the file generation unit120encodes the two generated second speech data SDL1and SDR1to generate the file FM1. As described above, the file generation unit120according to the present embodiment makes it possible to generate the file corresponding to stereophonic speech and multi-vibration.

The analysis of the stereophonic speech multi-vibration file by the playback device20is now described. The file analysis unit220according to the present embodiment is capable of extracting two sets of speech waveform data and vibration waveform data from two pieces of speech data obtained by decoding the file received by the communication unit230.

FIG. 8is a diagram illustrated to describe analysis of the stereophonic speech file by the playback device20according to the present embodiment. The file analysis unit220according to the present embodiment initially decodes the file FM1received by the communication unit230to acquire the speech data SDR1and SDL1.

Then, the file analysis unit220according to the present embodiment separates the acquired two pieces of speech data SDR1and SDL1on the basis of the to-be-synthesized band. In other words, the file analysis unit220according to the present embodiment separates the speech data SDR1on the basis of the to-be-synthesized band, so the file analysis unit220is capable of acquiring the speech data SDR2for right playback including the speech waveform data SbR1in the band of 200 Hz or more and the vibration data VDR2for right playback including the vibration waveform data VbR1in the band of 0 Hz to 200 Hz. Similarly, the file analysis unit220separates the speech data SDL1on the basis of the to-be-synthesized band, so acquires the speech data SDL2for left playback including the speech waveform data SbL1in the band of 200 Hz or more and the vibration data VDL2for left playback including the vibration waveform data VbL1in the band of 0 Hz to 200 Hz.

The function of the file analysis unit220according to the present embodiment as described above makes it possible to acquire two pieces of vibration data corresponding to the left and right vibration devices from the two pieces of speech data extracted by decoding the file FM1.

1.6. Modification of to-be-Synthesized Band

The functions of the file generation device10and the playback device20according to the present embodiment are described above. Moreover, the above description is given of the case where the to-be-synthesized band according to the present embodiment is a band of 0 Hz to 200 Hz as an example, but the to-be-synthesized band according to the present embodiment is not limited to this example. The to-be-synthesized band according to the present embodiment may be other bands than 0 Hz to 200 Hz.

Any Band of Predetermined Frequency or Less

In one example, the to-be-synthesized band and the synthesizing band according to the present embodiment may be any band of a predetermined frequency or less.FIG. 9is a diagram illustrated to describe a case where the to-be-synthesized band and the synthesizing band according to the present embodiment are any band equal to or less than a predetermined frequency.

As illustrated inFIG. 9, the to-be-synthesized band and the synthesizing band according to the present embodiment may be, in one example, a band of 80 Hz to 120 Hz. In this case, the file generation unit120of the file generation device10is capable of extracting speech waveform data Sb1-1in the band of 0 Hz to 80 Hz and speech waveform data Sb1-2in the band of 120 Hz or more from the first speech data SD0. In addition, the file generation unit120may extract the vibration waveform data Vb1in the band of 80 Hz to 120 Hz from the vibration data VD0. The file generation unit120is capable of synthesizing the extracted speech waveform data Sb1-1and Sb1-2and the vibration waveform data Vb1to generate the second speech data SD1.

The function of the file generation unit120according to the present embodiment as described above makes it possible to include the vibration waveform data in the band of 80 Hz to 120 Hz, which is easy to perceive it as vibration, in the second speech data and, at the same time, reduce the amount of speech waveform data to be cut, thereby keeping sound quality high.

Ultra-High Frequency Band Equal to or Higher than Predetermined Frequency

Further, in one example, the to-be-synthesized band according to the present embodiment may be an ultra-high frequency band equal to or higher than a predetermined frequency. Here, the ultra-high frequency band according to the present embodiment may be, in one example, a human inaudible range. More specifically, in a case where the file format corresponds to a high-resolution digital speech signal or the like, the to-be-synthesized band according to the present embodiment may be a band of 20 kHz or more.

FIG. 10is a diagram illustrated to describe a case where the to-be-synthesized band according to the present embodiment is an ultra-high frequency band equal to or higher than a predetermined frequency.FIG. 10illustrates an example where the to-be-synthesized band is 20 kHz or more. In this case, the file generation unit120according to the present embodiment cuts out the speech waveform data in the band of 20 KHz or more from the first speech data SD0to acquire the speech waveform data Sb1in the band of 0 Hz to 20 kHz. In addition, the file generation unit120acquires the vibration waveform data Vb1in any band from the vibration data VD0. In this event, the file generation unit120may acquire, in one example, vibration waveform data in the band of 200 Hz or less, or may acquire vibration waveform data in the band of 80 Hz to 120 Hz. Subsequently, the file generation unit120is capable of synthesizing the acquired speech waveform data Sb1with vibration waveform data Vb1to generate the second speech data SD1.

The function of the file generation unit120according to the present embodiment as described above makes it possible to include the vibration waveform data in any band, which is sufficient to perceive it as vibration, in the second speech data and, at the same time, completely save the speech waveform data corresponding to the human audible range, thereby generating a file with high sound quality.

Moreover, in the example ofFIG. 10, the case where the to-be-synthesized band is the band of 20 kHz or more is described as an example, but the ultra-high frequency band according to the present embodiment is not limited to this example. The to-be-synthesized band according to the present embodiment may be, in one example, a band of 15 kHz or more, or 18 kHz or more. The human inaudible range varies among individuals, but in general, it is known that the frequency band of 15 kHz or more becomes difficult to hear with age. Thus, the file generation unit120according to the present embodiment is capable of generating a file in which the sound quality relating to the audible range of more users is secured by including the vibration waveform data in the bands as described above.

Further, in the case where the to-be-synthesized band according to the present embodiment is the ultra-high frequency band, the file generation unit120is capable of synthesizing the waveform data in a plurality of synthesizing bands extracted from the vibration data with the to-be-synthesized band. In this event, in one example, the file generation unit120may generate a file including a plurality of pieces of vibration waveform data depending on the performance or characteristics of the playback device20. The playback device20according to the present embodiment is capable of extracting vibration waveform data corresponding to its own performance or characteristics from the above-mentioned file and performing vibration output based on the extracted vibration waveform data.

FIG. 11is a diagram illustrated to describe a file including a plurality of pieces of vibration waveform data and extraction of vibration waveform data from the file according to the present embodiment. The file analysis unit220according to the present embodiment initially decodes the file FM1received by the communication unit230to acquire the speech data SD1.

Further, in this event, the file analysis unit220acquires header information HI1attached to the file FM1. Here, the header information HI1may include information used to specify the to-be-synthesized band. In one example, the header information HI1may include information indicating a frequency of the to-be-synthesized band corresponding to each performance or model type.

In other words, the file analysis unit220according to the present embodiment is capable of specifying the to-be-synthesized band on the basis of the header information HI1attached to the file FM1or its own device information. Moreover, in the case of the example illustrated inFIG. 11, the file analysis unit220acquires the speech data SD2for playback including the speech waveform data Sb1and the vibration data VD2for playback including the vibration waveform data Vb1, from the speech data SD1, on the basis of the header information HI1.

The functions of the file generation device10and the playback device20according to the present embodiment as described above make it possible to transmit and receive a file including a plurality of pieces of vibration waveform data depending on performance or characteristics of the playback device20and to output appropriate vibration corresponding to performance or characteristics of the playback device20.

Moreover, the above description is given of the case where a plurality of pieces of vibration waveform data is included in the file to be transmitted or received, but the file generation device10according to the present embodiment may generate a file including vibration waveform data suitable for the performance or characteristics of the playback device20in advance on the basis of on the device information of the transmission destination playback device20. In addition, the playback device20according to the present embodiment is also capable of processing the vibration waveform data depending on the performance or characteristics after extracting the common vibration waveform data included in the file.

1.7. Operations of File Generation Device10and Playback Device20

The operations of the file generation device10and the playback device20according to the present embodiment are now described in detail.

Operation of File Generation Device10that Transmits File Generated on Basis of Collected Information

The operation of the file generation device10that transmits the file generated on the basis of collected information is now described. As described above, the file generation device10according to the present embodiment may generate a file on the basis of the speech information and vibration information collected by the collection unit110, and may transmit the file to the playback device20. In this case, the file generation device10according to the present embodiment may be, in one example, a smartphone, a tablet computer, a mobile phone, a PC, a wearable device, or the like.FIG. 12is a flowchart illustrating the operation of the file generation device10that transmits the file generated on the basis of the collected information.

Referring toFIG. 12, the collection unit110of the file generation device10initially collects speech data and vibration data (S1101). In addition, in this event, the collection unit110may collect moving image data at the same time.

Then, the file generation unit120according to the present embodiment generates the second speech data including the speech waveform data and the vibration waveform data (S1102).

Then, the file generation unit120according to the present embodiment encodes the second speech data generated in step S1102to generate a file including speech waveform data and vibration waveform data (S1103).

Then, the communication unit130according to the present embodiment transmits the file generated in step S1103to the playback device20(S1104).

Operation of File Generation Device10that Delivers Content Including Vibration Waveform Data

The operation of the file generation device10that delivers content including vibration waveform data is now described. As described above, the file generation device10according to the present embodiment may generate a new file by embedding the vibration waveform data in the speech data extracted from the existing file, and the file generation device10may be a content server that delivers the file as content.FIG. 13is a flowchart illustrating the operation of the file generation device10that delivers content including vibration waveform data.

Referring toFIG. 13, initially, the file generation unit120according to the present embodiment extracts speech waveform data and vibration waveform data respectively from a speech file and a vibration file (S1201).

Then, the file generation unit120according to the present embodiment generates the second speech data including the speech waveform data and the vibration waveform data (S1202).

Then, the file generation unit120according to the present embodiment encodes the second speech data generated in step S1202to generate a file including the speech waveform data and the vibration waveform data (S1203).

Subsequently, the file generation unit120according to the present embodiment registers the file generated in step S1203as content (S1204).

Then, the communication unit230according to the present embodiment transmits the content registered in step S1204to the playback device20on the basis of a download request from the playback device20(S1205).

Operation of Playback Device20

The operation of the playback device20according to the present embodiment is now described.FIG. 14is a flowchart illustrating the operation of the playback device20according to the present embodiment.

Referring toFIG. 14, initially, the communication unit230of the playback device20receives the file transmitted from the file generation device10(S1301).

Then, the file analysis unit220according to the present embodiment acquires header information from the file received in step S1301(S1302).

Subsequently, the file analysis unit220according to the present embodiment specifies a to-be-synthesized band on the basis of the header information acquired in step S1302and the device information to acquire the speech data and the vibration data (S1303).

Then, the output unit210according to the present embodiment outputs sound and vibration on the basis of the speech data and the vibration data acquired in step S1303(S1304).

2. Hardware Configuration Example

An example of a hardware configuration common to the file generation device10and the playback device20according to the present disclosure is now described.FIG. 15is a block diagram illustrating a hardware configuration example of the file generation device10and the playback device20according to the present disclosure. Referring toFIG. 15, the file generation device10and the playback device20each includes, in one example, a CPU871, a ROM872, a RAM873, a host bus874, a bridge875, an external bus876, an interface877, an input device878, an output device879, a storage880, a drive881, a connection port882, and a communication device883. Moreover, the hardware configuration shown here is illustrative, and some of components can be omitted. In addition, other components than the components shown here can be further included.

The CPU871functions as, in one example, an arithmetic processing unit or a control device, and controls some or all of the operations of each component on the basis of various programs recorded in the ROM872, the RAM873, the storage880, or a removable recording medium901.

The ROM872is a means for storing programs loaded into the CPU871, data used for operation, or the like. The RAM873temporarily or permanently stores, in one example, a program to be loaded into the CPU871, various parameters appropriately changing in executing the program, or the like.

The CPU871, the ROM872, and the RAM873are mutually connected via, in one example, the host bus874capable of high-speed data transmission. On the other hand, the host bus874is connected to the external bus876having a relatively low data transmission rate, in one example, via the bridge875. In addition, the external bus876is connected to various components via the interface877.

Examples of the input device878include a mouse, a keyboard, a touch panel, buttons, a switch, a lever, or the like. Furthermore, examples of the input device878include a remote controller capable of transmitting a control signal using infrared rays or other radio waves (hereinafter referred to as a remote controller). In addition, the input device878includes an audio input device such as a microphone.

The output device879is a device capable of visually or audibly notifying the user of the acquired information, which includes a display device such as a cathode ray tube (CRT), an LCD, or an organic EL, an audio output device such as a loudspeaker or a headphone, a printer, a mobile phone, a facsimile, or the like. In addition, the output device879according to the present disclosure includes various vibration devices or the like that outputs tactile stimuli.

The storage880is a device used to store various types of data. Examples of the storage880include a magnetic storage device such as hard disk drives (HDDs), a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like.

The drive881is a device that reads information recorded on the removable recording medium901such as a magnetic disk, an optical disk, a magneto-optical disk, or semiconductor memory or writes information to the removable recording medium901.

Examples of the removable recording medium901include a DVD medium, a Blu-ray (registered trademark) medium, an HD DVD medium, various kinds of semiconductor storage media, or the like. Of course, the removable recording medium901is preferably, in one example, an IC card or an electronic device mounted with a contactless IC chip.

The connection port882is a port used for connection with an external connection device902, such as a universal serial bus (USB) port, an IEEE 1394 port, a small computer system interface (SCSI), an RS-232C port, or an optical speech terminal.

External Connection Device902

Examples of the external connection device902include a printer, a portable music player, a digital camera, a digital moving image camera, an IC recorder, or the like.

The communication device883is a communication device used for connection with a network, and examples thereof include a communication card for wired or wireless LAN, Bluetooth (registered trademark), or wireless USB (WUSB), a router for optical communication, a router for asymmetric digital subscriber line (ADSL), or a modem for various communications.

As described above, the file generation device10according to the present disclosure synthesizes speech data including speech waveform data and vibration waveform data, encodes the speech data using a general-purpose format, and so is capable of generating a file including the speech waveform data and the vibration waveform data. In addition, the playback device20according to the present disclosure separates the file on the basis of the to-be-synthesized band, so is capable of acquiring and outputting speech data and vibration data. Such a configuration makes it possible to implement unified communication of vibration information while reducing data communication traffic.

Further, the respective steps in the processing of the file generation device10and the playback device20herein are not necessarily processed in chronological order in accordance with the sequences listed in the flowcharts. In one example, the respective steps in the processing of the file generation device10and the playback device20may be processed in an order different from the sequences described in the flowchart, or may be processed in parallel.

Further, the effects described in this specification are merely illustrative or exemplified effects, and are not limitative. In other words, with or in the place of the above effects, the technology according to the present disclosure can achieve other effects that are clear to those skilled in the art from the description of this specification.

An information processing apparatus including:

a file generation unit configured to generate a file including speech waveform data and vibration waveform data,

in which the file generation unit cuts out waveform data in a to-be-synthesized band from first speech data,

synthesizes waveform data extracted from a synthesizing band of vibration data with the to-be-synthesized band to generate second speech data, and

encodes the second speech data to generate the file.(2)

The information processing apparatus according to (1),

in which the to-be-synthesized band and the synthesizing band are any band of a predetermined frequency or less.(3)

in which the to-be-synthesized band and the synthesizing band are bass frequency bands of 0 Hz to a predetermined frequency or less.(4)

The information processing apparatus according to (1),

in which the to-be-synthesized band is an ultra-high frequency audible range of a predetermined frequency or more.(5)

The information processing apparatus according to (4),

in which the file generation unit synthesizes waveform data in a plurality of the synthesizing bands with the to-be-synthesized band.(6)

in which the file generation unit generates two pieces of the second speech data and encodes the two pieces of the second speech data to generate the file.(7)

in which the file generation unit generates third speech data by adding waveform data in the to-be-synthesized band cut out from two pieces of the first speech data and synthesizing the added waveform data with the to-be-synthesized band in one of the pieces of the first speech data, synthesizes waveform data in the synthesizing band extracted from the vibration data with the to-be-synthesized band in another one of the pieces of the first speech data to generate the second speech data, and encodes the second speech data and the third speech data to generate the file.(8)

a communication unit configured to transmit the file.(9)

a collection unit configured to collect the vibration waveform data.(10)

An information processing apparatus including:

a file analysis unit configured to acquire speech waveform data and vibration waveform data from a file including the speech waveform data and the vibration waveform data,

in which the file analysis unit separates speech data obtained by decoding the file on the basis of a to-be-synthesized band to acquire the speech waveform data and the vibration waveform data.(11)

The information processing apparatus according to (10),

in which the file analysis unit specifies the to-be-synthesized band on the basis of header information attached to the file.(12)

The information processing apparatus according to (11),

in which the file analysis unit specifies the to-be-synthesized band further on the basis of device information.(13)

The information processing apparatus according to any one of (10) to (12),

in which the file analysis unit extracts two sets of the speech waveform data and the vibration waveform data from two pieces of the speech data obtained by decoding the file.(14)

The information processing apparatus according to any one of (10) to (12),

in which the file analysis unit extracts a to-be-synthesized band in one of two pieces of the speech data obtained by decoding the file to acquire the vibration waveform data, and

synthesizes waveform data of a to-be-synthesized band duplicated from another one of the pieces of the speech data with the to-be-synthesized band of the one to acquire the speech waveform data.(15)

The information processing apparatus according to any one of (10) to (14), further including:

a communication unit configured to receive the file.(16)

The information processing apparatus according to any one of (10) to (15), further including:

an output unit configured to perform output based on the vibration waveform data and the speech waveform data.(17)

An information processing method including:

generating, by a processor, a file including speech waveform data and vibration waveform data,

in which the generating the file further includes

cutting out waveform data in a to-be-synthesized band from first speech data,

synthesizing waveform data extracted from a synthesizing band of vibration data with the to-be-synthesized band to generate second speech data, and

encoding the second speech data to generate the file.(18)

A program causing a computer to function as an information processing apparatus including:

a file generation unit configured to generate a file including speech waveform data and vibration waveform data,

in which the file generation unit cuts out waveform data in a to-be-synthesized band from first speech data,

synthesizes waveform data extracted from a synthesizing band of vibration data with the to-be-synthesized band to generate second speech data, and

encodes the second speech data to generate the file.(19)

An information processing method including:

acquiring, by a processor, speech waveform data and vibration waveform data from a file including the speech waveform data and the vibration waveform data,

in which the acquiring further includes separating speech data obtained by decoding the file on the basis of a to-be-synthesized band to acquire the speech waveform data and the vibration waveform data.(20)

A program causing a computer to function as an information processing apparatus including:

a file analysis unit configured to acquire speech waveform data and vibration waveform data from a file including the speech waveform data and the vibration waveform data,

in which the file analysis unit separates speech data obtained by decoding the file on the basis of a to-be-synthesized band to acquire the speech waveform data and the vibration waveform data.

REFERENCE SIGNS LIST