Sensory effect media generating and consuming method and apparatus thereof

Provided is a method and apparatus for generating and consuming sensory effect media. The method for generating sensory effect media includes receiving sensory effect information about sensory effects that are applied to media, and generating sensory effect metadata including the received sensory effect information. The sensory effect metadata includes effect description information for describing the sensory effects and device control description information for controlling devices that perform the sensory effects.

TECHNICAL FIELD

The present invention relates to a method and apparatus for generating and consuming media, and a computer-readable recording medium storing media metadata; and, more particularly, to a method and apparatus for generating and consuming sensory effect media, and a computer-readable recording medium storing sensory effect media metadata.

This work was supported by the IT R&D program of MIC/IITA [2007-S-010-01, “Development of Ubiquitous Home Media Service System based on SMMD”].

BACKGROUND ART

In general, media includes audio and video. The audio may be voice or sound, and the video may be motion pictures or images. When a user consumes or reproduces the media, the user can obtain information about the media by using metadata. The metadata is data about the media. Meanwhile, a device for reproducing media has also advanced from an analog-type device for reproducing analog media to a digital-type device for reproducing digital media.

Generally, an audio output device such as a speaker and a video output device such as a display device are used for reproducing the media.

FIG. 1is a diagram for describing a media technology according to the related art. Referring toFIG. 1, media102is outputted to a user using a media consuming method104. The media consuming method104according to the related art only includes devices for outputting audio and video.

Many researches have been made to develop a technology for effectively providing media to users. For example, an audio signal evolves into a multi-channel signal or a multi-object signal, and video technology has also advanced to high definition display, a stereoscopic image, or a 3-D image display technology.

Related to such a media technology, a media concept and a multimedia processing technology have also advanced. For example, Moving Picture Experts Group (MPEG) technologies were introduced from MPEG-1 to MPEG-21 as well as MPEG-2, MPEG-4, and MPEG-7. MPEG-1 defines a format for storing audio and video, MPEG-2 defines specifications for transmitting media, MPEG-4 defines an object-based media structure, MPEG-7 defines specifications for metadata of media, and MPEG-21 defines a framework for distributing media.

As described above, the media according to the related art is limited to audio and video. That is, it is impossible to maximize the effect of reproducing the media by interacting with various devices.

DISCLOSURE

Technical Problem

An embodiment of the present invention is directed to providing a method and apparatus for generating and consuming sensory effect media to maximize the effect of reproducing media.

Other objects and advantages of the present invention can be understood by the following description, and become apparent with reference to the embodiments of the present invention. Also, it is obvious to those skilled in the art of the present invention that the objects and advantages of the present invention can be realized by the means as claimed and combinations thereof.

Technical Solution

In accordance with an aspect of the present invention, there is provided a method for generating sensory effect media, including receiving sensory effect information about sensory effects that are applied to media, and generating sensory effect metadata including the received sensory effect information, wherein the sensory effect metadata includes effect description information for describing the sensory effects and device control description information for controlling devices that perform the sensory effects.

In accordance with another aspect of the present invention, there is provided an apparatus for generating sensory effect media, including an input unit for receiving sensory effect information about sensory effects that are applied to media, and a sensory effect metadata generator for generating sensory effect metadata including the received sensory effect information, wherein the sensory effect metadata includes effect description information for describing the sensory effects and device control description information for controlling devices that perform the sensory effects.

In accordance with another aspect of the present invention, there is provided a method for consuming sensory effect media, including receiving sensory effect metadata including sensory effect information about sensory effects that are applied to media, and searching for devices that perform the sensory effects and controlling the devices according to the sensory effect information, wherein the sensory effect metadata includes effect description information for describing the sensory effects and device control description information for controlling devices that perform the sensory effects.

In accordance with another aspect of the present invention, there is provided an apparatus for consuming sensory effect media, including an input unit for receiving sensory effect metadata having sensory effect information about sensory effects that are applied to media, and a controller for searching for devices that perform the sensory effects and controlling the devices according to the sensory effect information, wherein the sensory effect metadata includes effect description information for describing the sensory effects and device control description information for controlling devices that perform the sensory effects.

In accordance with another aspect of the present invention, there is provided a computer-readable recording medium storing metadata, the metadata including sensory effect metadata including sensory effect information for sensory effects that are applied to media. The sensory effect metadata may include effect description information for describing the sensory effects and device control description information for controlling devices that perform the sensory effects according to the effect description information, and the devices are searched for and controlled according to the sensory effect metadata.

Advantageous Effects

The method and apparatus for generating and consuming sensory effect media of the present invention can maximize the effect of reproducing media.

BEST MODE FOR THE INVENTION

Following description exemplifies only the principles of the present invention. Even if they are not described or illustrated clearly in the present specification, one of ordinary skill in the art can embody the principles of the present invention and invent various apparatuses within the concept and scope of the present invention. The use of the conditional terms and embodiments presented in the present specification are intended only to make the concept of the present invention understood, and they are not limited to the embodiments and conditions mentioned in the specification.

Also, all the detailed description on the principles, viewpoints and embodiments and particular embodiments of the present invention should be understood to include structural and functional equivalents to them. The equivalents include not only currently known equivalents but also those to be developed in future, that is, all devices invented to perform the same function, regardless of their structures.

For example, block diagrams of the present invention should be understood to show a conceptual viewpoint of an exemplary circuit that embodies the principles of the present invention. Similarly, all the flowcharts, state conversion diagrams, pseudo codes and the like can be expressed substantially in a computer-readable media, and whether or not a computer or a processor is described distinctively, they should be understood to express various processes operated by a computer or a processor.

Functions of various devices illustrated in the drawings including a functional block expressed as a processor or a similar concept can be provided not only by using hardware dedicated to the functions, but also by using hardware capable of running proper software for the functions. When a function is provided by a processor, the function may be provided by a single dedicated processor, single shared processor, or a plurality of individual processors, part of which can be shared.

The apparent use of a term, ‘processor’, ‘control’ or similar concept, should not be understood to exclusively refer to a piece of hardware capable of running software, but should be understood to include a digital signal processor (DSP), hardware, and ROM, RAM and non-volatile memory for storing software, implicatively. Other known and commonly used hardware may be included therein, too.

In the claims of the present specification, an element expressed as a means for performing a function described in the detailed description is intended to include all methods for performing the function including all formats of software, such as combinations of circuits for performing the intended function, firmware/microcode and the like.

To perform the intended function, the element is cooperated with a proper circuit for performing the software. The present invention defined by claims includes diverse means for performing particular functions, and the means are connected with each other in a method requested in the claims. Therefore, any means that can provide the function should be understood to be an equivalent to what is figured out from the present specification.

Other objects and aspects of the invention will become apparent from the following description of the embodiments with reference to the accompanying drawings, which is set forth hereinafter. The same reference numeral is given to the same element, although the element appears in different drawings. In addition, if further description on the related prior arts is determined to obscure the point of the present invention, the description is omitted. Hereafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

An apparatus for generating and consuming (or reproducing) media according to the related art outputs and displays audio and video only. However, human beings have not only a visual sense and an auditory sense but also an olfactory sense and a tactile sense. Lately, many researches have been made to develop a device that stimulates the five senses of a user, such as the tactile sense and the olfactory sense.

Meanwhile, home appliances were controlled generally by an analog signal. However, the home appliances have advanced to be controlled by a digital signal.

Accordingly, a concept of media has also advanced to include not only audio and/or video data but also sensory effect information to control various devices that stimulate the olfactory sense and the tactile sense in order to maximize the effect of reproducing the media.

Recently, a single media single device (SMSD) based service is available. The SMSD based service is a media service that enables a user to reproduce one media through one device. However, many researches have been made to develop a single media multi device (SMMD)-based service for maximizing the effect of reproducing media in Ubiquitous home. The SMMD based service is a media service that enables a user to reproduce one media through a plurality of devices by interacting with the plurality of devices. Therefore, it is necessary to advance from media to sensory effect media that enables a user not only to watch/hear the media but also to sense the sensory effects of the media through the five senses of the user. It is expected that the sensory effect media will expand a media industry and a market of a sensory effect device and provide rich experience to a user by maximizing the effect of reproducing the media. Therefore, the sensory effect media encourage users to consume more media.

FIG. 2is a conceptual diagram describing sensory effect media in accordance with an embodiment of the present invention. Referring toFIG. 2, media202and sensory metadata are inputted to a sensory effect media consuming method204. For example, the media202may be provided from a media provider (not shown), and the sensory effect metadata may be provided from a sensory effect provider (not shown).

The media202includes audio and video, and the sensory effect metadata includes sensory effect information for reproducing sensory effects. The sensory effect metadata may include all information that can maximize the effect of reproducing the media202. For example, the sensory effects for a visual sense, an olfactory sense, and a tactile sense are shown inFIG. 2. Accordingly, the sensory effect information includes visual effect information, olfactory effect information, and tactile effect information.

The sensory effect media consuming method204according to the present embodiment controls a media output device206to receive and reproduce the media202. The sensory effect media consuming method204controls sensory effect devices208,210, and222based on the visual effect information, the olfactory effect information, and the tactile effect information. For example, a dimmer208is controlled according to the visual effect information, a perfumer210is controlled according to the olfactory effect information, and a vibrating device212such as a chair is controlled according to the tactile effect information.

When a device reproduces video including a scene of lightning and thunder, the dimmer208is turned on and off, or when a device reproduces video having a scene of foods or a green field, the perfumer210is controlled. Furthermore, when a device reproduces video having a scene of car chasing, the vibrating device212is controlled. Therefore, the corresponding sensory effects can be provided to users with the video.

The sensory effect metadata includes effect description information for describing sensory effects and device control description information for controlling devices that perform sensory effects according to the effect description information. Various users may be enabled to maximally reproduce sensory effects using sensory effect devices that the user owns by defining information to be included in the sensory effect information as described above. For example, if a user owns the dimmer208only, the user may reproduce the sensory effects by controlling only the dimmer210. If a user owns the dimmer208and the perfumer210, the user may reproduce the sensory effects more realistically by controlling not only the dimmer208but also the perfumer210. The sensory effect metadata may further include general information having sensory effect metadata generation information.

The effect description information may include effect type information about a type of sensory effect, device identification information about identification of a device, device condition information for reproducing sensory effects, and initial effect information of a device. The device control description information may include reference information for the device identification information, synchronization information for synchronizing media with sensory effects, and parameter information for controlling a device.

Since it is necessary to synchronize such sensory effects with audio or video of the media, the control description information may include synchronization information for synchronizing the media with the sensory effect.

Hereinafter, the apparatus and method for generating and consuming sensory effect media according to the present invention will be described in detail.

<Generation of Sensory Effect Media>

Hereinafter, a method and apparatus for generating sensory effect media according to an embodiment of the present invention will be described.

The method for generating sensory effect media according to the present embodiment includes receiving sensory effect information on sensory effects applied to media, and generating sensory effect metadata including the received sensory effect information. The sensory effect metadata includes effect description information for describing sensory effects and device control description information for controlling devices that perform the sensory effects. Accordingly, a user owning various types of sensory effect devices is enabled to reproduce proper sensory effects based on the generated sensory effect metadata. The generated sensory effect metadata may be transferred to a user through various paths.

The method may further include transmitting the sensory effect metadata to a user terminal. If a sensory effect service provider generates the sensory effect metadata, the sensory effect metadata may be directly provided to a user independently from the media. For example, if a user already owns media of a predetermined movie, the user may request sensory effect metadata of the predetermined movie to a sensory effect service provider, receive the requested sensory effect metadata from the sensory effect service provider, and reproduce sensory effects of the predetermined movie using the sensory effect metadata.

The method may further include generating sensory effect media by packaging the generated sensory effect metadata and the media, and transmitting the sensory effect media to the user terminal. The sensory effect service provider may provide the media and the sensory effect metadata at the same time. The sensory effect service provider generates a sensory effect metadata, generates sensory effect media by combining or packaging the generated sensory effect metadata with the media, and transmits the generated sensory effect media to a user terminal. The sensory effect media may be formed in a file of a sensory effect media format to reproduce sensory effects. The sensory effect media format may be a standard file format for sensory effect reproduction.

The sensory effect information may include special effect information for reproducing sensory effects and control information for controlling devices that perform the sensory effects. The sensory effect information may further include device information on devices that perform sensory effects.

The sensory effect metadata may further include general information having sensory effect metadata generation information. The effect description information may include effect type information about a type of sensory effect, device identification information of devices, condition information of devices that perform sensory effect, and initial effect information of devices. The device control description information may include reference information for device identification information, synchronization information for synchronizing media and sensory effects, and parameter information for controlling devices.

The sensory effect information may differ according to scenes of media. The sensory effect may include susceptibility as well as the five senses of sensory organs. For example, the special effect information may information for moving curtains or vibrating windows for making audiences of a horror movie to fear. The sensory effect information may include information for turning on or off dimmers for reproducing the special effect of lighting or thunder. The device information is information on devices that perform the sensory effects. Such device control description information includes control information for controlling a device that reproduces sensory effects according to the sensory effect information. The device control description information may include synchronization information for synchronizing the media with the sensory effects. The synchronization information makes the sensory effects to be reproduced according to the progression of scenes of media.

FIG. 3is a diagram illustrating an apparatus for generating a sensory effect media in accordance with an embodiment of the present invention. Referring toFIG. 3, the sensory effect media generating apparatus302includes an input unit304for receiving sensory effect information on sensory effects that are applied to media, and a sensory effect metadata generator306for generating sensory effect metadata including the received sensory effect information. The sensory effect metadata includes effect description information for describing sensory effect and device control description information for controlling devices that perform sensory effects according to the effect description information. The sensory effect media generating apparatus302may further include a transmitter308for transmitting the sensory effect metadata to a user terminal.

The sensory effect generating apparatus may further include a sensory effect generator for generating sensory effect media by packaging the generated sensory effect metadata and the media. The transmitter may transmit the sensory effect media to the user terminal. In case of generating the sensory effect media, the input unit304may receive media and the sensory effect media generator310generates the sensory effect media by combining or packaging the received media and the sensory effect metadata generated by the sensory effect metadata generator306.

The sensory effect metadata may further include general information having sensory effect metadata generation information. The effect description information may includes effect type information about a type of sensory effect, device identification information of devices, condition information of devices that perform sensory effect, and initial effect information of devices. The device control description information may include reference information for the device identification information, synchronization information for synchronizing media with sensory effects, and parameter information for controlling devices.

The sensory effect metadata may further include general information having sensory effect metadata generation information. The effect description information may include effect type information about a type of sensory effect, device identification information of devices, condition information of device that perform sensory effects, and initial effect information of devices. The device control description information may include reference information for the device identification information, synchronization information for synchronizing media with sensory effects, and parameter information for controlling devices.

Since other details of the sensory effect generating apparatus are identical to those of the sensory effect media generating method, the description thereof is omitted.

<Consumption of Sensory Effect Media>

Hereinafter, a method and apparatus for consuming sensory effect media according to an embodiment of the present invention will be described.

A method for consuming sensory effect media according to the present embodiment includes receiving sensory effect metadata including sensory effect information on sensory effects that are applied to media, and searching for devices capable of reproducing the sensory effects and controlling the devices according to the sensory effect information. The sensory effect metadata includes effect description information for describing sensory effects and device control description information for controlling devices that perform sensory effects according to the effect description information. If a user terminal already has media, the sensory effect metadata is received together with media. When the sensory effect metadata is received, the sensory effect metadata is analyzed to determine what kinds of sensory effect information are included therein, and devices owned by a user are searched for to reproduce the sensory effects. Then, the sensory effects are properly reproduced according to the combination of the devices of the user by controlling the searched devices.

While receiving sensory effect metadata, the media may be received too. That is, the sensory effect metadata maybe received together with the media. When the sensory effect metadata and the media are received together, the media may be packaged with the sensory effect metadata. The packaging of the media and the sensory effect metadata may be a file of a sensory effect media format.

The sensory effect metadata may further include general information having sensory effect metadata generation information. The effect description information may include effect type information about a type of sensory effect, device identification information of devices, condition information of devices that perform sensory effect, and initial effect information of devices. The device control description information may include reference information for device identification information, synchronization information for synchronizing media and sensory effects, and parameter information for controlling devices.

Since other details of the sensory effect media consuming method and apparatus are identical to those of the generation of sensory effect media, it is omitted here.

FIG. 4describes an apparatus for consuming a sensory effect media in accordance with an embodiment of the present invention. Referring toFIG. 4, the sensory effect media consuming apparatus402includes an input unit404for receiving sensory effect metadata having sensory effect information on sensory effects that are applied to media, and a controller406for searching for devices408that reproduce the sensory effects and controlling the devices according to the sensory effect information. The sensory effect metadata includes effect description information for describing sensory effects and device control description information for controlling devices that perform sensory effects according to the effect description information. Here, the sensory effect media consuming apparatus402is not limited to a device for reproducing the sensory effect only. The sensory effect media consuming apparatus402may be any device that can consume the media, for example, a cellular phone, a mobile terminal such as a personal media player (PMP), TV, and an audio system.

The input unit404may further receive the media. In this case, the media is packaged with the metadata.

The sensory effect metadata may further include general information having sensory effect metadata generation information. The effect description information may include effect type information about types of sensory effects, identification information of devices, condition information of devices that perform sensory effect, and initial effect information of devices. The device control description information may include reference information for device identification information, synchronization information for synchronizing media and sensory effects, and parameter information for controlling devices.

Since other details of the sensory effect media consuming apparatus are identical to that of the generation of the sensory effect media, the description thereof is omitted here.

Hereinafter an overall system for reproducing sensory effects according to an embodiment of the present invention will be described.

FIG. 5describes reproducing sensory effects in accordance with an embodiment of the present invention. Referring toFIG. 5, the sensory effect metadata generator502receives sensory effect information and generates sensory effect metadata. The media may be transferred to a user independently from the sensory effect metadata. However, the media is transferred together with the sensory effect metadata to the user inFIG. 5. The sensory effect media generating apparatus504generates the sensory effect media using the media and the sensory effect metadata generated by the sensory effect metadata generator502. The sensory effect media may be formed in a predetermined file format for providing the sensory effect media.

The sensory effect media generated by the sensory effect media generating apparatus504is transferred to the sensory effect media consuming apparatus506. The sensory effect media consuming apparatus506searches for sensory effect devices that a user owns. InFIG. 5, a user owns a digital TV514, a vibration chair508, a dimmer510, an audio system512, an air-conditioner516, and a perfumer518. The sensory effect media generating apparatus504senses the sensory effect devices of the user, for example, the vibration chair508, the dimmer510, the audio system512, the air-conditioner516, and the perfumer518, and controls the searched sensory effect devices to reproduce sensory effects. The sensory effect media generating apparatus504also synchronize scenes reproduced at the digital TV514with the sensory effect devices.

The sensory effect media consuming apparatus506may be connected to the sensory effect devices508,510,512,514,516, and518through a network in order to control the sensory effect devices. For example, various network technologies such as LonWorks and universal plug and play (UPnP) may be applied to.

Meanwhile, MPEG media technologies such as MPEG-7 and MPEG-21 may be applied together in order to effectively provide media.

Hereinafter, embodiments of the present invention will be described based on subjects of providing and consuming services, such as a sensory effect service provider for providing a sensory effect service, a media service provider for providing media, and a user for reproducing the sensory effects.

FIGS. 6 to 9describes various embodiments of the present invention.

FIG. 6is a block diagram illustrating an apparatus for generating and consuming sensory effect media in accordance with a first embodiment of the present invention. Referring toFIG. 6, a service provider602transmits sensory effect metadata604including sensory effect information and media606to a service consumer608. The service provider602provides media and information for reproducing sensory effects of the media to the service consumer605at the same time. The service provider605may include a broadcasting service provider. The service consumer608receives the sensory effect metadata604including the sensory effect information and the media606. The received media606is reproduced by a media reproducing device618, and the received sensory effect metadata604is inputted to the sensory effect media consuming apparatus610. The sensory effect media consuming apparatus610is connected to first, second, and third sensory effect devices612,614, and616through a network and controls the first, second, and third sensory effect devices612,614, and616according to the received sensory effect metadata604. The sensory effect media consuming apparatus610receives the media606for synchronizing reproducing the media606with reproducing the sensory effects by the first to third sensory effect devices612,614, and616and controls the media reproducing device618and the sensory effect devices612,614, and616.

FIG. 7is a block diagram illustrating an apparatus for generating and consuming sensory effect media in accordance with a second embodiment of the present invention. Referring toFIG. 7, a sensory effect service provider702for providing a sensory effect service is separated from a media service provider706for providing media708. The media service provider is a service provider who provides media708. The sensory effect service provider702is a service provider who provides sensory effect metadata704including sensory effect information for reproducing sensory effects in order to provide a sensory effect service for the media708. The sensory effect service provider702transmits the sensory effect metadata704to the service consumer710, and the media service provider706transmits the media708to the service consumer710. The transmitted media708is reproduced by a media reproducing device720of the service consumer710, and the sensory effect media consuming apparatus712controls the sensory effect devices714,716, and718using the sensory effect metadata. Also, the sensory effect media consuming apparatus712synchronizes the media708with the sensory effect devices714,716, and718.

FIG. 8is a block diagram illustrating method and apparatus for generating and consuming sensory effect media in accordance with a third embodiment of the present invention. InFIG. 8, a service consumer side owns information for reproducing not only media but also sensory effects. Here, the service consumer side may include devices of a consumer, such as a DVD player. If the service consumer side is the DVD player, a disk stores information for reproducing the media and the sensory effects. The information for reproducing sensory effects may be stored in a form of metadata. The sensory effect metadata804is transmitted to the sensory effect media consuming apparatus808and controls the first to third sensory effect devices810,812, and814. The sensory effect media consuming apparatus808may include a DVD player. The media806is reproduced by a media reproducing device816and outputted through a TV. For example, the DVD player may perform a function of the media reproducing device816together. The sensory effect media consuming apparatus808synchronizes the media806and the first to third sensory effect devices810,812, and814.

FIG. 9is a block diagram illustrating method and apparatus for generating and consuming sensory effect media in accordance with a fourth embodiment of the present invention. InFIG. 9, a service provider902, equivalent to a sensory effect service provider in the third embodiment, provides information for reproducing sensory effects, and a service consumer906owns media908in the fourth embodiment. The service consumer906wants to reproduce sensory effects while reproducing the media908. In order to reproduce the sensory effects, the service consumer906requests information for reproducing the sensory effects to the service provider902. The service provider902transmits information for reproducing sensory effects to the service consumer906. The information for reproducing sensory effects may be transmitted to the service consumer906in a form of sensory effect metadata. The service consumer906reproduces the media908using a media reproducing device918, and the sensory effect media consuming apparatus910controls first to third sensory effect devices912,914, and916using the sensory effect metadata904. The sensory effect media consuming apparatus910synchronizes the media908with the first to third sensory effect devices912,914, and916.

Hereinafter, the sensory effect metadata according to an embodiment of the present invention will be described.

The sensory effect metadata according to the present embodiment include sensory effect information about sensory effects that are applied to media. To be specific, the sensory effect metadata according to the present embodiment includes effect description information for describing sensory effects and device control description information for controlling devices that perform sensory effects according to the effect description information. Herein, the sensory effect metadata may further include general information having sensory effect metadata generation information. A sensory effect media consuming apparatus including a user terminal can search for and control devices using the sensory effect metadata.

The effect description information may include effect type information about a type of sensory effect, identification information of a device, condition information of a device that performs sensory effect, and initial effect information of a device. The device control description information may include reference information for device identification information, synchronization information for synchronizing media and sensory effects, and parameter information for controlling devices.

Hereinafter, the sensory effect metadata according to embodiments of the present invention will be described with reference toFIGS. 10 to 18. The sensory effect metadata is referred to as device control information (DCI).

Meanwhile, the sensory effect metadata according to the present embodiment may be integrated with media such as MPEG-7 and a network technology such as LonWorks. For the Network technology such as LonWorks, standard network variable types (SNVTs) may be used. In this case, a namespace prefix may be used for identifying a type of metadata according to connected technology. Table 1 shows details of the namespace prefix.

FIG. 10describes device control information (DCI) in accordance with an embodiment of the present invention. Referring toFIG. 10, the device control information1001includes effect description information EffectDescription1003and device control description information DeviceControlDescription1004. The device control information1001further includes general information GeneralInfo1002. Table 2 shows details of the device control information.

TABLE 2NameDefinitionGeneralInfoThis element is the container for theinformation on the DCI creationEffectDescriptionThis element is the container for theevery effect applied to the mediaDeviceControlDescriptionThis element is the container for thecontrol parameters for each device

The general information GeneralInfo1002includes information related to the generation of the device control information and the effect description information EffectDescription1003includes information related to effects that applied to media. The device control description information DeviceControlDescription1004may include information related to control parameters for each device.

For example, a schema for the device control information DCI1001relatedFIG. 10is as follows.

Hereinafter, the general information GeneralInfo1002, effect description information EffectDesription1003, and the device control description information DeviceControl Description1004will be described.

FIG. 11is a diagram for describing generation information GeneralInfo included in sensory effect metadata in accordance with an embodiment of the present invention. The general information GeneralInfo includes information related to the generation of the device control information. Referring toFIG. 11, the general information GeneralInfo1102may include Confidence1102, Version1103, Comment1104, PublicIdentifier1106, PrivateIdentifier1107, Creator1108, CreationLocation1109, CreationTime1110, Instrument1111, and Rights1112. The general information GeneralInfo1102includes information related to the generation of the device control information. Therefore, the general information may include information related to the generation of the general metadata. For example, the general information may include information about a version, a last update date, a producer, a producing time, a producing place, and a copyright. Here, “GeneralInfo” defined at MPEG-7 may be referred. An exemplary schema for the general information GeneralInfo1102is as follows.

Hereinafter, the effect description information EffectDescription will be described.

FIG. 12describes effect description information EffectDescription included in sensory effect metadata in accordance with an embodiment of the present invention. The effect description information EffectDescription may include information about all of effects that are applied to media. The sensory effect media consuming apparatus including a user terminal determines what type of effect is applied to media and maps devices for each of the effects based on transferred information. Referring toFIG. 12, the effect description information EffectDescription1201may include effect information Effect1202. The effect information Effect1202may include a plurality of information. Table 3 shows the definition of the effect information.

TABLE 3NameDefinitionEffectThis element describes the information of an effect

The effect information Effect1202describes sensory effects. An exemplary schema for the effect description information EffectDescription1202related toFIG. 12is as flows.

Hereinafter, the effect information Effect will be described.

FIG. 13describes effect information included in sensory effect metadata in accordance with an embodiment of the present invention. The effect information Effect may include sensor effect type information, device condition information, and initial effect information. The effect information Effect functions as connecting devices with sensory effects. Referring toFIG. 13, the effect information Effect1301may include device identification information TargetID1302, effect type information TypeOfEffect1303, condition information Condition1304, and initial effect information InitialEffect1305.

Table 4 shows details of the effect information.

TABLE 4NameDefinitionTypeOfEffectThis attribute contains the enumeration set of effect type.Enumeration ValueDescription“VisualEffect”Visual effect“SoundEffect”Sound effect“WindEffect’Wind effect“TemperatureEffect”Temperature effect“LightEffect’Light effect“ShadingEffect”Shading effect“VibrationEffect”Vibration effect“ScentEffect”Scent effectTargetIDThis attribute describes the ID of individual device thatwill present the effect. It is necessary in case the authorwants to give an effect with multiple devices.ConditionThis element describes the conditions information for thedevicesInitialEffectThis element contains the device control parameters forinitial effect

The device identification information TargetID1302is information that describes the identifications of individual devices that perform the sensory effects. The device identification information TargetID1302is used to identify devices if sensory effects are reproduced using a plurality of devices. The effect type information TypeOfEffect1303may include enumeration for effect types of sensory effects. Table 4 shows the enumeration for effect types included in the effect type information TypeOfEffect1303.

As shown in Table 4, the enumeration may include visual effect VisualEffect, sound effect SoundEffect, wind effect WindEffect, temperature effect TemperatureEffect, light effect LightEffect, shading effect ShadingEffect, vibrating effect VibrationEffect, and scent effect ScentEffect. An exemplary schema for the effect information1301related toFIG. 13is as follows.

Hereinafter, the condition information Condition will be described.

FIG. 14describes condition information included in sensory effect metadata in accordance with an embodiment of the present invention. The condition information Condition may include information about details that devices need to perform sensory effects. For example, the condition information Condition may provide hint information to a sensory effect consuming apparatus including a user terminal for mapping the most suitable devices to corresponding sensory effects among devices that a user owns. The condition information Condition may include information about what parameters are used for controlling devices. For example, if the condition information Condition includes information for controlling a direction of a device, a control parameter for setting a direction is used. Meanwhile, the condition information may set a control value within an operation range in order to stably control devices. It is because an extreme control value, which is set regardless of a device property, may damage the device or harm a user.

TABLE 5NameDefinitionMaxResolution ModeThis element describes the max resolution mode that the visualeffect device should support.MaxLuxThis element describes the maximum brightness value by lux(lx) that the lighting effect device should support.The type is dci:LuxType.MaxCandelaThis element describes the maximum candela (cd) value that thevisual effect device should support. Candela is the S1 base unitof luminous intensity. It is used by visual devices such asmonitor, PDP TV, LCD TV. The type is dci:CandelaType.MaxContrastThis element describes the maximum contrast that the visualeffect device should support. This type is dci:ContrastType.MaxSoundVolumeThis element describes the maximum sound volume by decibel(db) that the sound device should support. The type isdci:soundVolumetype.MaxWindSpeedThis element describes the maximum wind speed value by meterper second (mps) that the wind effect device should support.The type is dci:WindSpeedType.MaxTurnSpeedThis element describes the maximum value for turning speedthat the device should support. This will be used in suchdevice as rotation fan device or rotation lighting device.The type is dci:TrunSpeedType.DirectioncontrollableThis element indicates the controllability for direction.If the value is true, the device should support thedirection controllability. If false, the device maynot support it.SwingControllableThis element indicates the controllability for swing.If the value is true, the device should support theswing controllability. If false, the device maynot support it.LocationModeThis element describes the mode of locationMaxColorModeThis element describes the maximum number of colorsthat the device should support. This is necessary incase the device can change color

TABLE 6NameDefinitionMonoColorThis element describes a mono color that the device should support. This isnecessary in case the device cannot change its color. The type is dci:ColorTypeas follows. The color can be expressed by r, g, b intensity.DimmingControllableThis element indicates the controllability for dimming. If the value is true, thedevice should support dimming controllability. If false, the device shall notsupport itMinTemperatureThis element describes the minimum value of temperature by Celsius that thedevice should support. The type is dci:MinTempType.MaxTemperatureThis element describes the maximum value of temperature by Celsius that thedevice should support. The type is dci:MaxTempType.WindSpeedControllableThis element indicates the controllability for wind speed. If the value is true, thedevice should support wind speed controllability. If false, the device shall notsupport itOpenModeThis element describes the open mode that the shading device should support.MaxVibrationThis element describes the maximum value of vibration by rpm that thevibration device should support. The type is dci:VibrationType.VibraionControllableThis elemenet indicates the controllability for vibration. If the value is true, thedevice should support vibration controllability. If false, the device should notsupport itMaxDiffusionThis element describes the maximum value of diffusion by milligram (mil) thatthe diffusion device should support. The type is dci:DiffusionType.DiffusionControllableThis elemenet indicates the controllability for diffusion. If the value is true, thedevice should support diffusion controllability. If false, the device should notsupport itScentModeThis element describes the mode of scent that the scent device should support.OtherConditionThis element reserves a hole for extension

MaxResolutionMode1402describes the maximum resolution mode that a visual effect device should supports. Table 7 shows details of resolution modes.

MaxLux1403describes the maximum brightness value by lux (lx) that the lighting effect device should support. MaxCandela1404describes the maximum luminance in a unit of candela cd that a visual effect device should support. The candela is a SI unit for luminance intensity. It is used for a visual effect device such as a monitor, a PDP TV, and a LCD TV. MaxContrast1405describes the maximum contrast that a visual effect device should support. MaxSoundVolume1406describes the maximum sound volume in a unit of decibel db, which a sound effect device should support.

MaxWindSpeed1407describes the maximum wind speed in a unit of meter per second m/s that a wind effect device should support. MaxTurnSpeed1408describes the maximum value of a turning speed that a device should support. It may be used for a rotation fan device or a rotation lighting device.

DirectionControllable1409indicates controllability for a direction. If DirectionControllable1409is true, a corresponding device should support the direction controllability. If DirectionControllable1409is false, a corresponding device should not support it.

SwingControllable1410indicates controllability for swing. If the value thereof is true, a corresponding device should support the swing controllability. If the value is false, the corresponding device should not support it.

LocationMode1411describes a mode for a location. Table 8 defines details of LocationMode.

MaxColorMode1412describes the maximum number of colors that a device should support. It is necessary if the device can change colors. Table 9 defines modes for colors.

MonoColor1413describes a mono colors that a device should support. MonoColor1413is necessary if a device cannot change colors. Color may be expressed as R, G, and B intensity.

DimmingControllable1414indicates controllability for dimming. If DimmingControllable1414is true, a corresponding device should support dimming controllability. If DimmingControllable1414is false, a corresponding device should not support dimming controllability.

MinTemperature1415describes the minimum temperature value in Celsius that a device should support. MaxTemperature1416describes the maximum temperature value in Celsius that a device should support.

WindSpeedControllable1417indicates controllability for a wind speed. If WindSpeedControllable1417is true, a corresponding device should support wind speed controllability. If WindSpeedControllable1417is false, a corresponding device should not support wind speed controllability. OpenMode1418describes an open mode that a shading device should support. Table 10 shows details of the open modes.

TABLE 10Value ofOpen ModeOpen Mode1Bottom to Top2Top to Bottom3Left to Right4Right to Left5Center to Side

Information related to OpenMode1418includes OpeningTimeControllable1419, MaxOpeningTime1420, and MinOpeningTime1421.

MaxVibration1422describes the maximum vibration value in rpm that a vibration effect device should support. VibrationControllable1423indicates controllability for vibrating. If VibrationControllable1423is true, a corresponding device should support vibration controllability for vibrating. If VibrationControllable1423is false, a corresponding device should not support vibration controllability. MaxDiffusion1424describes the maximum diffusion value of a diffusion effect device in milligram. DiffusionControllable1425indicates controllability for diffusion. If DiffusionControllable1425is true, a corresponding device should support diffusion controllability. If DiffusionControllable1425is false, the corresponding device should not support diffusion controllability.

ScentMode1426describes scent modes that a scent effect device should support. Table 11 shows details of scent modes.

TABLE 11Scent ModeDescription1Scent of rose2Scent of acacia3Scent of chrysanthemum4Scent of lilac5Scent of mint6Scent of jasmine7Scent of pine tree8Scent of orange9Scent of grape10~1024reserved

OtherCondition1427is an element reserved for extensibility of the condition information Condition1401.

An exemplary schema of the condition information Condition1401is as follows.

Hereinafter, the initial effect information InitialEffect will be described in more detail.

FIG. 15is a diagram for describing initial effect information included in sensory effect metadata in accordance with an embodiment of the present invention. The initial effect information InitialEffect includes information about initial states before sensory effect media is consumed. The initial effect information InitialEffect may include a list for device control parameters.

TABLE 12NameDefinitionSetOnOffThis element sets state of device into on or off. The type is boolean.SetBrightnessLevelThis element sets brightness level of visual device. The value is expressed inpercentage (0%~100%). The type is dci:LevelType.SetContrastLevelThis element sets contrast level of visual device. The value is expressed inpercentage (0%~100%). The type is dci:LevelTypeSetResolutionModeThis element sets resolution mode of visual device. The resolution mode isdefined as table in Annex A. 1.SetSoundVolumeLevelThis element sets volume level of sound device. The value is expressed inpercentage (0%~100%). The type is dci:LevelType.SetWindSpeedThis element sets wind speed of wind device. The value is expressed in Meter perSecond (mps). The type is dci:WindSpeedTypeSetWindSpeedThis element sets wind speed level of wind device. The value is expressed inLevelpercentage (0%~100%). The type is dci:LevelTypeSetDirectionThis element sets direction of the device. The type is dci:DirectionType.dci:DirectionType has two attributes, xAngle and yAngle, to express thedirection. xAngle and yAngle attributes are represented by snvt:angle_degType.SetTurnSpeedThis element sets turning speed level of the device. The value is expressed inLevelpercentage (0%~100%). The type is dci:LevelType.SetColorThis element sets color of lighting device which enables to represent certaincolor. The type of SetColor is dci:ColorType.SetDimmingLevelThis element sets dimming level of lighting device which enable to dim thebrightness level. The type is dci:LevelType.SetTemperatureThis element sets temperature of temperature control device. The type isdci:TemperatureType.SetTemperatureLevelThis element sets temperature level of temperature control device. The type isdci:LevelType.SetOpenLevelThis element sets open level of shading device, the type is dci:LevelType.SetVibrationThis element sets vibration level of vibration device. The type is dci:LevelType.LevelSetDiffusionThis element sets diffusion level of diffusion device. The type is dci:LevelType.LevelSetScentModeThis element sets a scent mode for scent device.OtherSettingsThis element sets other control data for effects. This element is for user definedcontrol data. The type of OtherSettings is dci:OtherType.

SetOnOff1502sets a state of a device into on or off.

SetBrightnessLevel1503sets a brightness level of a visual effect device. The value thereof is expressed in percentage. SetContrastLevel1504sets a contrast level of a visual effect device. The value thereof is expressed in percentage.

SetResolutionMode1505sets a resolution mode of a visual effect mode. SetSoundVolumeLevel1506sets a volume level of a sound effect device. The value thereof is expressed in percentage.

SetWindSpeed1507sets a wind speed of a wind effect device. The value thereof is expressed in a unit of meter per second m/s. SetWindSpeedLevel1508sets a wind speed level of a wind effect device. The value thereof is expressed in percentage.

SetDirection1509sets a direction of a device. SetDirection1509has two attributes, x-angle and y-angle, which will be described in later with SNVTs. SetTurnSpeedLevel1511describes a turning speed of a device. The value thereof is expressed in percentage.

SetColor1512sets a color of a light effect device that emits a predetermined color of light. SetDimmingLevel1513sets a dimming level of a lighting device that dims the brightness level. SetTemperature1514sets a temperature of a temperature control device. SetTemperatureLevel1515sets a temperature level of a temperature control device. SetOpenLevel1516sets an open level of a shading device. Related to SetOpenLevel1516, SetOpenSpeedLevel1517may be further set.

SetVibrationLevel1518sets a vibration level of a vibrating device. SetDiffusionLevel1519sets a diffusion level of a diffusion device. SetScentMode1520sets a scent mode of a scent effect device.

OtherSettings1521sets control data for controlling other sensory effects. OtherSettings1521may be defined for elements defined in the condition information Condition. In addition, OtherSettings1521may be defined for elements in SetSwingOn1510.

An exemplary schema for the initial effect information InitialEffect1501is as follows.

Hereinafter, the device control description information DeviceControlDescription will be described in detail.

FIG. 16is a diagram for describing device control description information included in sensory effect metadata in accordance with an embodiment of the present invention. Referring toFIG. 16, the device control description information DeviceControlDescription1601may include device control data DeviceControlData1602. The device control description information DeviceControlDescription1601may include a plurality of device control data DeviceControlData1602according to the number of sensory effect devices. Table 13 shows details of device control description information.

TABLE 13NameDefinitionDeviceControlDataThis element describes device control data for eachdevice

The device control data DeviceControlData1602may include control data of each device. An exemplary schema for the describe control description information1601is as follows.

Hereinafter, the device control data DeviceControlData will be described.

FIG. 17is a diagram for describing device control data included in sensory effect metadata in accordance with an embodiment of the present invention. Referring toFIG. 17, the device control data DeviceControlData1701includes reference information RefTargetID1702about device identification information, synchronization information Sync1703for synchronizing media and sensory effects, and parameter information ControlData1704for controlling a device. The device control data DeviceControlData1701may include control information of each device. The device control data DeviceControlData1701may be matched with effect information Effect included in the effect description information with reference to the device identification information TargetID. The device control data DeviceControlData1701includes synchronization information Sync1703for interlocking with media. Table 14 shows the details of the elements of the device control data.

TABLE 14NameDefinitionRefTargetIDThis element points to TargetID of Effect in EffectDescription. Through thiselement, the terminal can recognize this control data is belongs to which device.SyncThis element describes time information for synchronization with media. Thetype is dci:SyncType as follows. dci:syncType has two attributes, start andduration. These attributes are represented by mpeg7:mediaTimePointType andmpeg7:mediaDrationType.ControlDataThis element describes various control parameters for device. The type isdci:ControlDataType which is explained in InitialEffect in EffectDescription.

The reference information RefTargetID1702indicates effect information Effect included in the effect description information EffectDescription. The sensory effect media consuming apparatus including a user terminal uses the reference information RefTargetID1702to determine which device is related to control data.

The synchronization information Sync1703describes time information for synchronizing with media. The synchronization information Sync1703may include attributes such as start and duration.

The parameter information ControlData1704describes various control parameters for a device.

FIG. 18is a diagram for describing parameter information ControlData included in sensory effect metadata in accordance with an embodiment of the present invention. Referring toFIG. 18, the parameter information ControlData1801includes SetOnOff1802, SetBrightnessLevel1803, SetContrastLevel1804, SetResolutionMode1805, SetSoundVolumeLevel1806, SetWindSpeed1807, SetWindSpeedLevel1808, SetDirection1809, SetSwingOn1810, SetTurnSpeedLevel1811, SetColor1812, SetDimmingLevel1813, SetTemperature1814, SetTemperatureLevel1815, SetOpenLevel1816, SetOpenSpeedLevel1817, SetVibrationLevel1818, SetDiffusionLevel1819, SetScentMode1820, and OtherSettings1821. The parameter information ControlData1704is related the initial effect information InitialEffect included in the effect description information EffectDescription. The detail description thereof is omitted.

Tables 15 to 17 are for describing a simple type. The simply type may include restriction condition information for sensory effects.

TABLE 15NameDefinition & SourceLuxTypeThis simple type represents degree of brightnessusing lux. The restriction base is snvt:luxType.The value is restricted from 0 to 5000 lux.<simpleType name=“LuxType”><restriction base=“snvt:luxType”><maxInclusive value=“5000”/></restriction></simpleType>CandelaTypeThis simple type represents intensity of lightusing candela. The value is restricted from 0 to5000.<simpleType name=“CandelaType”><restriction base=“unsignedInt”><minInclusive value=“0”/><maxInclusive value=“5000”/></restriction></simpleType>MaxTemperatureTypeThis simple type represents maximumtemperature using centigrade. The restriction baseis snvt:temp_pType. The value is restricted from0 to 45.<simpleType name=“MaxTemperatureType”><restriction base=“snvt:temp_pType”><minInclusive value=“0”/><maxInclusive value=“45”/></restriction></simpleType>MinTemperatureTypeThis simple type represents minimum temperatureusing centigrade. The restriction base issnvt:temp_pType. The value is restricted from−15 to 0.<simpleType name=“MinTemperatureType”><restriction base=“snvt:temp_pType”><minInclusive value=“−15”/><maxInclusive value=“0”/></restriction></simpleType>

TABLE 17NameDefinition & SourceDiffusionTypeThis simple type represents mass using milligram. Therestriction base is snvt:mass_milType. The value isrestricted from 0 to 200.<simpleType name=“DiffusionType”><restriction base=“snvt:mass_milType”><maxInclusive value=“200”/></restriction></simpleType>ContrastTypeThis simple type represents contrast using the expressionlike 1000:1.<simpleType name=“ContrastType”><restriction base=“string”><pattern value=“\d{4}:\d{1}”/></restriction></simpleType>LevelTypeThis simple type represents percentage. The value isrestricted from 0 to 100.<simpleType name=“LevelType”><restriction base=“unsignedInt”><minInclusive value=“0”/><maxInclusive value=“100”/></restriction></simpleType>VibrationTypeThis simple type represents intensity of vibration usingrpm. The restriction base is snvt:rmp_Type. The value isrestricted from 0 to 20000.<simpleType name=“VibrationType”><restriction base=“snvt:rpm_Type”><maxInclusive value=“20000”/></restriction></simpleType>

Related to networks of sensory effects, SNVT related to Lonworks may be exemplarily defined as follows.

Here, “Valid Tpe Range” denotes that a range of set values is limited. “Type resolution” denotes a resolution for expressing values. “Unit” denotes a unit used in SNVT.

The method of the present invention described above can be realized as a program and stored in a computer-readable recording medium such as CD-ROM, RAM, ROM, floppy disks, hard disks, magneto-optical disks and the like. Since the process can be easily implemented by those skilled in the art to which the present invention pertains, further description will not be provided herein.

INDUSTRIAL USABILITY

The method and apparatus for generating and consuming sensory effect media according to the present invention is used to generate and consume the sensory effect media.