gaspard7/wiki_gpac huggingface dataset:
Grouping

When several renditions are possible for a set of inputs, the default behavior is as follows:

- if video is present, it is used as the main content
- otherwise, audio is used as the main content

The main content is repeated for each combination with other content. For example with 1 video and 3 audios, you will have:
```
#EXT-X-STREAM-INF:BANDWIDTH=1000000,CODECS="avc1.42C01F,mp4a.40.2",RESOLUTION=1280x720,FRAME-RATE="25",AUDIO="audio1"
video.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1000000,CODECS="avc1.42C01F,mp4a.40.2",RESOLUTION=1280x720,FRAME-RATE="25",AUDIO="audio2"
video.m3u8

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio1",NAME="1",LANGUAGE="fra",AUTOSELECT=YES,URI="audio_1.m3u8",CHANNELS="1"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio2",NAME="2",LANGUAGE="eng",AUTOSELECT=YES,URI="audio_2.m3u8",CHANNELS="1"
```

This behavior can be changed by specifying a group for input streams using the PID property `HLSGroup` :
```gpac -i source1 -i audio1:#HLSGroup=MyAudio -i audio2:#HLSGroup=MyAudio -o res/live.m3u8```

This will result in:

```
#EXT-X-STREAM-INF:BANDWIDTH=1000000,CODECS="avc1.42C01F,mp4a.40.2",RESOLUTION=1280x720,FRAME-RATE="25",AUDIO="MyAudio"
video.m3u8

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="MyAudio",NAME="1",LANGUAGE="fra",AUTOSELECT=YES,URI="audio_1.m3u8",CHANNELS="1"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="MyAudio",NAME="2",LANGUAGE="eng",AUTOSELECT=YES,URI="audio_2.m3u8",CHANNELS="1"
```

**Warning**  
**__GPAC will not check if streams within a group have the same codec properties, you must only put in the same group streams with the same coding configurations, i.e. streams for which the `CODECS` parameter is the same.__**
Naming

The component names are the DASH representation ID. By default this is an integer value, but it can be set to whatever suits you:

```gpac -i source1 -i audio1:#HLSGroup=MyAudio:#Representation=Soustitres -i audio2:#HLSGroup=MyAudio:#Representation=Subtitles -o res/live.m3u8```

This will result in:

```
#EXT-X-STREAM-INF:BANDWIDTH=1000000,CODECS="avc1.42C01F,mp4a.40.2",RESOLUTION=1280x720,FRAME-RATE="25",AUDIO="MyAudio"
video.m3u8

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="MyAudio",NAME="Soustitres",LANGUAGE="fra",AUTOSELECT=YES,URI="audio_1.m3u8",CHANNELS="1"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="MyAudio",NAME="Subtitles",LANGUAGE="eng",AUTOSELECT=YES,URI="audio_2.m3u8",CHANNELS="1"
```
Dual DASH + HLS Generation

GPAC can generate a manifest for both HLS and DASH by using the [dual option](dasher#dual).


```gpac -i source1 -i source2 -o res/live.mpd:dual:profile=live```

This will generate `live.mpd`and `live.m3u8` together with one file per segment produced.

```gpac -i source1 -i source2 -o res/live.m3u8:dual:profile=onDemand```

This will generate `live.m3u8` and `live.mpd` together with a single file per quality. Since the DASH profile is onDemand, the ISOBMFF file of each quality will also contain a segment index (not used in HLS).
Foreword

Please make sure you are familiar with [DASH terminology](DASH-basics) before reading. 

In this howto, we will study various setups for DASH live streaming in low latency mode, using both [MP4Box](MP4Box) and [gpac](gpac_general). This approach follows standards. It is compatible with any low-latency aware DASH implementation such as [dash.js](https://dashif.org/tools/dashjs/)

As a reminder, you can always get more info on option `foo in GPAC by using:
```
gpac -h foo
MP4Box -h foo
```

You can also check the filter graph used by your session:
```
gpac -graph REST_OF_COMMAND
MP4Box -fgraph REST_OF_COMMAND 
```

And when using gpac, you can enable real-time reporting of filters activities using `gpac -r`.


The `gpac` application can be used for dashing whenever `MP4Box` is used, but the opposite is not true. Especially MP4Box cannot:

- use complex custom filter chains while dashing, such as transcoding in several qualities
- produce two DASH sessions at the same time


The basic syntax for MP4Box dashing is:  
```MP4Box -dash SEGMENT_DUR -profile live -out file.mpd source1 ... sourceN```

The equivalent syntax for gpac dashing is:  
```gpac -i source1 ... -i sourceN -o file.mpd:profile=live:segdur=2```

MP4Box can accept filter options in each of the source URLs as well as in the destination URL.  
```MP4Box -dash SEGMENT_DUR -profile live -out file.mpd:OPT1=VAL1[...] source1:OPT2=VAL2 ... sourceN:OPTN=VALN```

The equivalent syntax for gpac dashing is:  
```gpac -i source1:OPT2=VAL2 ... -i sourceN:OPTN=VALN -o file.mpd:profile=live:segdur=2:OPT1=VAL1```
 
Options specified on a source or destination filter are inherited by any filter needed to link the source to the destination, as [explained here](filters_general#arguments-inheriting). When dashing, options specified on the destination manifest (file.mpd) are also specified on the target segmented outputs.

MP4Box can accept filter chains for each source URLs.  
```MP4Box -dash SEGMENT_DUR -profile live -out file.mpd source1:@@cecrypt:cfile=drm1.xml ... sourceN:@@cecrypt:cfile=drmN.xml```

The equivalent syntax for gpac dashing requires setting up filter IDs (FID) and source filtering (SID) on your filters as explained in the [encryption howto](encryption-filters):  
```gpac -i source1:FID=S1 cecrypt:cfile=drm1.xml:FID=C1:SID=S1 ... -i sourceN:FID=SN cecrypt:cfile=drm1.xml:FID=CN:SID=SN -o file.mpd:SID=C1,CN:profile=live:segdur=2```



__Note__  
_MP4Box dashing is a simple wrapper around a filter session running the [dasher](dasher) filter, with all MP4Box DASH options remapped to the filter. For backward compatibility reasons, the old options of GPAC 0.8 or below were  kept, but the syntax can also be migrated to GPAC filters:_

```MP4Box -dash 1000 -ast-offset 9000 -profile live [...]```

_is equivalent to_

```MP4Box --segdur=1000 --asto=9000 --profile=live [...]```
DASH Origin Server setup

You now will want to distribute your DASH Low Latency over HTTP(S). The difficulty is that regular HTTP severs will usually not understand how to deliver a file while it is being produced without a great deal of tricks - for example, see our experimental node.js [low latency server](https://github.com/gpac/node-gpac-dash).

To simplify this process, GPAC features a simple HTTP server that understands DASH low latency and is able to push segments while they are being produced, without any file system monitoring. Please read the documentation of the [HTTP Server](httpout) for more details.

The server will need a local directory where files are stored and can be delivered to clients behind the live edge. This is done by indicating to the server a read directory.


```
MP4Box -ast-offset 9000 -frag-rt -dash-live 10000 -frag 1000 -profile live 
	-out http://localhost:8080/live.mpd:gpac:rdirs=outdir source1 source2


gpac -i source1 -i source2 reframer:rt=on -o http://localhost:8080/live.mpd:gpac:segdur=10:cdur=1:profile=live:dmode=dynamic:rdirs=outdir:asto=9
```

Note that since this example only uses a single DASH session, all options specified for the dasher can be set globally:
```
MP4Box -ast-offset 9000 -frag-rt -dash-live 10000 -frag 1000 -profile live
	-out http://localhost:8080/live.mpd --rdirs=outdir --wdir=outdir source1 source2

gpac -i source1 -i source2 reframer:rt=on -o http://localhost:8080/live.mpd
	--segdur=10 --cdur=1 --profile=live --dmode=dynamic --asto=9  --rdirs=outdir
```
DASH Origin PUSH setup

You may rather want to use a regular HTTP server as your origin server, and have GPAC push segments to that server while they are being produced. Again, please read the documentation of the [HTTP Server](httpout).

In this case, the httpout filter does not work as a server but as an HTTP client issuing PUT or POST requests, and does not need any local directory.


```
MP4Box -ast-offset 9000 -frag-rt -dash-live 10000 -frag 1000 -profile live
	-out http://ORIG_SERVER_IP_PORT/live.mpd:gpac:hmode=push source1 source2

gpac -i source1 -i source2 reframer:rt=on
	-o http://ORIG_SERVER_IP_PORT/live.mpd:gpac:segdur=10:cdur=1:asto=9:profile=live:dmode=dynamic:hmode=push
```

Note that since this example only uses a single DASH session, all options specified for the dasher can be set globally:
```
MP4Box -ast-offset 9000 -frag-rt -dash-live 10000 -frag 1000 -profile live
	-out http://ORIG_SERVER_IP_PORT/live.mpd --hmode=push source1 source2

gpac -i source1 -i source2 reframer:rt=on -o http://ORIG_SERVER_IP_PORT/live.mpd 
	--segdur=10 --cdur=1 --profile=live --dmode=dynamic --asto=9 --hmode=push
```
DASH Low Latency live encoding

We will now use a live source (webcam), encode it in two qualities, DASH the result and push it to a remote server. Please check the [encoding howto](encoding) first.
Compared to what we have seen previously, we only need to modify the input part of the graph:

- take as a live source the default audio video grabbed by the [libavdevice](ffavin) filter
- rescale the video  as 1080p and 720p 
- encode the rescaled videos at 6 and 3 mbps 
- encode the audio in aac at 128 kbps
- feed the encoders outputs to the dasher

```
gpac 
	#specify the source
	-i av://:FID=1

	#specify first rescaler and encoder for 1080p
	ffsws:osize=1920x1080:SID=1 @ enc:c=avc:fintra=1:FID=EV1:b=6m

	#specify second rescaler and encoder for 720p
	ffsws:osize=1280x720:SID=1 @ enc:c=avc:fintra=1:FID=EV2:b=3m

	#specify AAC encoding	
	enc:c=aac:SID=1:FID=EA:b=128k

	#specify http push output in DASH, using only sources from video and audio encoders 
	-o http://ORIG_SERVER_IP_PORT/live.mpd:gpac:SID=EV1,EV2,EA

	#http and dash options specified globally
	 --hmode=push --profile=live --segdur=1 --cdur=0.1 --asto=0.9

```

The `fintra`  parameter is used to force the encoders to generate an intra / IDR frame every one second, thus ensuring a proper segmentation of the result.

In the above command, we have a  ultra-low latency setup where fragments are only 100ms for 1s segments. Note that this assumes that the encoders are able to always encode 100ms of data in less than 100ms, which can be problematic (intra / IDR are heavier to code). To avoid potential 404s, you can either increase the fragment duration or just only reduce the `asto, which will slightly increase the latency but keep pushing at the same rate. 

__Note__  
_The [rescaler](ffsws) filter will automatically move to pass-through mode if the input size and pixel format match the output ones (desired by the encoder)._
MPEG-2 TS Sequences

*   **simple** profile in a [single file](http://download.tsi.telecom-paristech.fr/gpac/DASH_CONFORMANCE/TelecomParisTech/mpeg2-simple/mpeg2-simple-mpd.mpd)
*   **simple** profile in multiple files [with templates](http://download.tsi.telecom-paristech.fr/gpac/DASH_CONFORMANCE/TelecomParisTech/mpeg2-simple-files/mpeg2-simple-files-mpd-template.mpd) and [without templates](http://download.tsi.telecom-paristech.fr/gpac/DASH_CONFORMANCE/TelecomParisTech/mpeg2-simple-files/mpeg2-simple-files-mpd.mpd)
ISO File Format Sequences

All DASH MPDs are generated with:

*   10s segments, beginning with RAP
*   1s “moof” fragments (which corresponds to the GOP size)

Each adaptation set (described below) are available in the following ways:

*   Multiple segments:
    *   segment list conforming Main profile, except for GDR which complies with Full profile; MPD for these sequences are named `XXX-**files**.mpd,`
    *   segment URL Template conforming Live profile, except for GDR which complies with Full profile; MPD for these sequences are named `XXX-**url**.mpd`.
*   One file:
    *   indexes (n “sidx”) conforming Main profile, except for GDR which complies with Full profile; MPD for these sequences are named `XXX-**indexes**.mpd,`
    *   single segment (1 “sidx”) conforming OnDemand profile, except for GDR which complies with Full profile; MPD for these sequences are named `XXX-**single**.mpd.`

Currently the following combinations are provided:

*   1 Adaptation Set, with alternate MP3 or AAC audio representations;
*   2 Adaptation Sets, one with alternate MP3 or AAC audio representations, the other with baseline AVC video;
*   1 Adaptation Set, with audio (AAC) and video (open-GOP), using 2 components. Each segment is made of 2 sub-segments indexed as:
    *   1 primary “sidx” indexing both subsegments (“reference\_type” 1);
    *   each subsegment has a secondary A/V “sidx” (“reference\_type” 0) with 5 entries (one per “moof”).
*   1 Adaptation Set, with audio (AAC) and video (GDR with 8 frames roll recovery), using 2 components. Each segment is made of 5 sub-segments indexed with daisy chained SIDX; each SIDX contains two "reference\_type 0" subsegments (one per moof) and one "reference type 1" subsegment pointing to the next SIDX (except for the last SIDX).
MPEG-2 TS Sequences

All DASH MPDs are generated with 10s segments, beginning with RAP. Only the baseline AVC|H264 profile is currently used. Each adaptation set (described below) are available in the following ways:

*   Multiple segments:
    *   segment list conforming Simple@TS profile; MPD for these sequences are named `XXX-**files**.mpd,`
    *   segment URL Template conforming Simple@TS profile; MPD for these sequences are named `XXX-**url**.mpd`.
*   One file:
    *   single segment (1 “sidx”) conforming Simple@TS profile; MPD for these sequences are named `XXX-**single**.mpd.`
Sequences Generation

To generate these DASH sequences, several open-source software tools have been used (in the sequential order of stream generation):

*   GPAC/MP4Box to process the BIFS visual data and export it to a raw video track in AVI,
*   Audacity \[1.3 Beta\] to generate the raw audio track,
*   x264 \[0.118.2085 8a62835\] for the AVC|H.264 encoding,
*   faac \[as included in FFmpeg build\] for AAC encoding,
*   LAME \[as included in FFmpeg build\] for MP3 encoding,
*   GPAC/MP42TS \[REV3926\] for Transport Stream packaging,
*   GPAC/MP4Box \[REV3926\] for ISOBMF packaging, MPD generation and DASH segmentation (TS and ISOBMFF).

For more info on DASH generation, see [[DASH Support in MP4Box]].
Multimedia scene description {:data-level="all" }

A scene description is a language describing animations, interactivity, 2D and 3D shapes, audio and video relationship in a presentation. 
GPAC supports a variety of scene description languages:

- MPEG-4 BIFS, in its binary form, [text](MPEG-4-BIFS-Textual-Format) form and [XML](MPEG-4-XMT-Format) form
- Web3D VRML97 and X3D
- SVG 1.2 Tiny profile
- MPEG-4 LASeR
- A subset of SWF (Adobe Flash)

MP4Box supports encoding of BIFS and LASeR, as well as various conversions operations. A detailed list of supported operations is provided [here](mp4box-scene-opts).

Rendering is supported in GPAC through the [Compositor](compositor) filter.

For MPEG-4 Systems scene coding and media packaging, GPAC relies on standard MPEG-4 descriptions but defines its own extensions for media multiplexing, as well as for some [BIFS commands](MPEG-4-Scene-Commands). You may also have a look at the [tips and tricks](MP4Box-tips-and-tricks-with-BT-and-XMT) page.
Command declaration

### BT format

```
RAP AT time IN esID { .... }
```

`RAP` and `IN esID` can be omitted most of the time.


### XMT format

```xml
<par begin="time" atES_ID="esID" isRAP="yes" > ... </par>
```

`isRAP` and `atES_ID` can be omitted most of the time. `isRAP` is a GPAC extension.
Replacing a value in a multiple field

### BT format
```
REPLACE nodeName.fieldName[idx] BY newValue
```
### XMT format
```xml
<Replace atNode="nodeName" atField="fieldName" position="idx" value="newValue" />
```

For XMT-A, `idx` can also take the special values 'BEGIN' and 'END'. Replacement of a node in an MFNode field is the combination of both syntax
Replacing a multiple field

### BT format
```
REPLACE nodeName.fieldName BY [value1 ... valueN]
```
```
REPLACE nodeName.fieldName BY [Node { ... } ... Node { ... }]
```
### XMT format
```xml
<Replace atNode="nodeName" atField="fieldName" value="value1 ... valueN" />
```
```xml
<Replace atNode="nodeName" atField="fieldName" >
<NodeXXX>...</NodeXXX>
<NodeXXX>...</NodeXXX>
</Replace>
```

Replacement of a node in an MFNode field is the combination of both syntax
Inserting a simple value in a multiple field

### BT format
```
INSERT AT nodeName.fieldName[idx] newValue
```
```
APPEND TO nodeName.fieldName newValue
```
### XMT format
```xml
<Insert atNode="nodeName" atField="fieldName" position="idx" value="newValue" />
```

For XMT-A, `idx` can also take the special values 'BEGIN' and 'END'.
Inserting a node in a node list field

### BT format
```
INSERT AT nodeName.fieldName[idx] Node { }
```
```
APPEND TO nodeName.fieldName Node { }
```
### XMT format
```xml
<Insert atNode="nodeName" atField="fieldName" position="idx" >
<NodeXXX>...</NodeXXX>
</Insert>
```

For XMT-A, `idx` can also take the special values 'BEGIN' and 'END'.
BT Format {:data-level="all"}

BT stands for BIFS Text and is an exact textual representation of the MPEG-4 BIFS scene. Its syntax is the same as the VRML/X3D (.wrl and .x3dv files) ones for the scene description part, and it has been extended for other MPEG-4 tools (OD, OCI, IPMP).

You will find plenty of example BT files in the tutorial and regression test suite, and we strongly recommend using these to get more familiar with the BT syntax.

The BT language has been originally developed at ENST as the textual format of the late MP4Tool. The format is still enhanced at ENST for GPAC needs, but we try not to mess up too much with it to keep VRML compatibility.

There are 3 major parts in a BT file:

*   The root scene, made of a collection of PROTO nodes if desired, a single top level node and a list of routes for interaction if needed. This is the part common to BT and VRML formats.
*   The InitialObjectDescriptor, describing the streams that must be opened when opening the MPEG-4 scene. It usually contains a BIFS stream description and an OD stream description when visual/audio media are present in the scene.
*   A succession of modification to the scene and their associated timing.
MP4Box specific BT syntax

The syntax of BIFS/OD commands in BT has been enhanced to enable animation stream and scalable description encodings. The new elements are placed at the access unit declaration 'AT TIMING' element:

*   '`RAP`' element: specifies the following access unit is a random access points. This is needed because MP4Box cannot currently compute the scene random access state per stream. Note however that the first access unit of any systems stream is considered as a random access point
*   '`IN`' element: specifies the following access unit happens in the stream of given ID.

For example, `RAP AT 1000 IN 20 { ... }` means that the access unit is a random access point, its timing is 1000 (in stream timescale) and it happens in stream whose ES\_ID is 20.

The syntax of the '`AT TIMING`' element has been extended to support differential timing to enable authors to specify relative time between commands rather than absolute timing. The differential timing is signaled by using a capital D before the timing itself. For example, `AT D2000 { ... }` means that this access unit occurs 2000 time ticks after the preceding access unit.
Minimal Stream Descriptors for MP4Box (BT and XMT) {:data-level="all"}

When encoding a BIFS or OD ES\_Descriptor, MP4Box must find at least:

*   The decoderConfigDescriptor with the right streamType set

When encoding an interaction stream descriptor, MP4Box must find at least:

*   The decoderConfigDescriptor with the right streamType set
*   The UIConfig descriptor with the deviceName field. Note however that most of the time you will need to specify the ES\_ID and the OCR\_ES\_ID of this descriptor to make sure events are not tied to any timeline.

When encoding a regular ES\_Descriptor, MP4Box must find at least:

*   The MuxInfo descriptor with the fileName (StreamSource in XMT) set.

For systems streams, if the SLConfigDescriptor is not found, MP4Box uses a stream timescale of 1000, otherwise it uses `SLConfigDescriptor.timestampResolution` as stream timescale.

When the InitialObjectDescriptor is not found in the BT file, MP4Box will create one for you.
Non-linear Parsing

MP4Box can perform non-linear parsing of text files (BT,XMT,VRML,X3D/XML), in other words it understands usage of a node before its definition. This greatly simplifies content authoring, in terms of complexity (for example, 2 conditionals referencing each-other) and readability (you don't have to declare things at specific places). When encoding, node declarations are put back before node referencing (thus the decoded file will not look the same as the original one).
Simple Macro processor (BT only)

MP4Box can use simple macros in BT in order to help authoring. **A BT (or WRL) file with such macros cannot be understood by any other tools than MP4Box!**.

The macro must be defined on a single line as follows:

```c
#define MACRO_NAME REP LAC E MENT STR ING
```

The BT parser will simply replace any occurrence of the macro by its value.

Here is an example of a BT macro:

```c
#define MYAPP Appearance { material Material2D { emissiveColor 1 0 0 filled TRUE} }
#define MYCOORDS 200 20

Transform2D {

    translation MYCOORDS

    children [
        Shape {
            appearance MYAPP
            geometry Rectangle { size 20 20 }
        }
    ]
}
```
Importing media with MP4Box and XMT

There is no standard element in XMT-A allowing to describe multiplexing parameters for MP4. In order to do this MP4Box uses an XML representation of `MuxInfo` in the `StreamSource` sub elements (encoding hints) called `MP4MuxHints`.

The complete syntax is:

```xml
<StreamSource url="filename" >
    <MP4MuxHints GroupID="..." startTime="..." duration="..." useDataReference="..." noFrameDrop="..." SBR_Type="..." frameRate="..." compactSize="..." textNode="..." fontNode="..." />
</StreamSource>
```

The streamFormat parameter is not represented in XMT-A.

_**Note**_ When decompressing an mp4 file to bt (XMT-A), a MuxInfo (MP4MuxHints) is automatically generated for all streams other than BIFS and OD in order to keep track of media location in the original MP4. You can then simply modify and re-encode the BT or XMT file without specifying the media data location.
Context {:data-level="all"}

HEIF is a new image format defined within MPEG, by companies such as Apple, Nokia, Canon, ... and by the GPAC team and Telecom Paris ! 

HEIF is based on the various constructs of ISOBMFF, supported in GPAC. For example, HEIF enables the storage of single images or image collections through the `meta` box and so-called items. HEIF also supports the storage of image sequences (like animated GIFs) using usual tracks, but with a new handler type called `pict`. HEIF was developed as a codec-agnostic container format (like ISOBMFF), but a first derivation of this generic format was standardized for images coded using the High Efficiency Video Codec (HEVC). 

Nice examples of HEIF images and of its benefits compared to other formats can also be found [on the website of some playback tools developed by Nokia](http://nokiatech.github.io/heif/), or on your iOS devices.
Generating HEIF images

GPAC supports generating HEIF images using MP4Box, with the following command line:

```
MP4Box -add-image file.hvc:primary -ab heic -new image.heic
```

This will take the first image of the HEVC file, create a `meta` box, add one image item, make it a primary item and add the `heic` brand to the output file.

The following command line will do the same but for the next IDR frame after the given time and the `heix` brand.

```
MP4Box -add-image file.hvc:time=1.2:primary -ab heix -new image.heic
```

Finally, the following command line will take a tiled HEVC stream (as described in [[this page|HEVC Tile-based adaptation guide]]) and generate one item per tile and one item for the entire image.

```
MP4Box -add-image tiled.hvc:split_tiles:primary -ab heic -new tiled.heic
```

For more options, look at `MP4Box -h meta`.
Playback

Playback of HEIF (avc and hevc codecs) is also possible using the usual syntax:

```
gpac -i heif_url vout
gpac -play heif_url
gpac -gui heif_url
```

for HEIF collections, the source can be played as a file sequence:
```
gpac -play heif_url:itt
gpac -gui heif_url:itt
```