# An Ontology for Spatio-Temporal Media Management and an Interactive Application
## 3 The Software-Defined Media Ontology
### **Media** Class  
The **Media** class represents spatio-temporal media data. One can trace all the objects specified in SDM Ontology, such as media data, related devices and functions, and semantic information that constitutes the spatio-temporal media, starting from the **Media** object. Consequently, we believe that media objects represent digital copies of recorded objects and have the potential to be used as comprehensive descriptions of the history and semantic information of digital media. Figure 5 shows the structure of the **Media** class. The inclusion of the **MediaEvent** class in the definition of the **Media** class differs from the definitions of the **Recorder**, **Processor**, and **Player** classes. The **Media** object can represent any playback section in a media file by specifying the media file and the start and end times in the media file. The default origin of the time axis is the beginning of the media file; however, the time origin can be changed using the **mediaStartAt** property. The start time of the playback section is represented by the **startAt** property, and the end time by the **endAt** property and expresses an offset value from the time origin. A media file is a file that records video, audio, and various observation data along with time information. Audio media files are time-series recordings of audio data and video media files are time-series recordings of image data. A data media file is a time-series record of various types of observational data. Typical media files are mp3, AAC, and WAV format files for audio, mp4, H.264/265, and other compressed video formats. JSON files that record sensor data with timestamps are also media files. Data that does not have a time representation, such as image data, can also be treated as a media file by interpreting it as having a playback time of 0. The point cloud data acquired by LiDAR is a spatio-temporal media that is a time-series aggregate of many ranging results and can be expressed as **DataMedia**. By processing this point cloud data together with the movement trajectory of LiDAR device, which is expressed as the location information of **Recorder**, it can be converted into a 3D object [50], and the resulting media file (e.g., mesh data) can also be expressed as **DataMedia**. In addition, technologies have been developed to synthesize realistic 3D images by combining point clouds with video images or photographs [51]. In the near future, it is expected that CG images will increasingly be used instead of video images, and the media files used for this purpose can be described as **DataMedia**. In the SDM Ontology, there is no definitional distinction between the **AudioMedia** subclass, **VideoMedia** subclass, and **DataMedia** subclass, and the definition of each subclass is for convenience. It is up to the data creator to decide which subclass to use or to describe the object simply as a **Media** class. An SMF (Standard MIDI File) describes MIDI protocol data with time information ([[https://www.midi.org/specifications-old/item/standard-midi-files-smf](https://www.midi.org/specifications-old/item/standard-midi-files-smf)]([https://www.midi.org/specifications-old/item/standard-midi-files-smf](https://www.midi.org/specifications-old/item/standard-midi-files-smf)) (accessed on 1 June 2023)); for example, if the SMF file is a file of performance data that assumes the playback of musical sounds by a MIDI tone generator, it is appropriate to treat it as **AudioMedia**. However, if the SMF file describes a control sequence of stage equipment using the MIDI Show Control protocol ([[http://www.richmondsounddesign.com/docs/midi-show-control-specification.pdf](http://www.richmondsounddesign.com/docs/midi-show-control-specification.pdf)]([http://www.richmondsounddesign.com/docs/midi-show-control-specification.pdf](http://www.richmondsounddesign.com/docs/midi-show-control-specification.pdf)) (accessed on 1 June 2023)), it may be more appropriate to treat it as a **DataMedia** object or simply a Media object.