gaspard7/wiki_gpac huggingface dataset:
Filter Design Principles

A filter object obeys the following principles:

- may accept (consume) any number of data stream (named `PID` in this architecture)
- may produce any number of PIDs
- can have its input PIDs reconfigured at run time or even removed
- decide when to drop input packets and create new ones, and which source packet properties to transfer to output packets
- only executed by a single thread at any time, although executing threads may change
- can be configured through options, some of which may be changed at runtime. Options are typed, using the same types as PID properties (see below) 
- carries its documentation (general help and options)
- describes possible input and output connections, using property values (see below)
- can load input filters (e.g. DASH/HLS source or playlist filters) and can load output filters (e.g. dash segmenter)
- can send and receive events up/down the filter chain

All filters in GPAC follow the same design, and there is no conceptual difference between a source, a sink, a raw audio/video filter, a demultiplexer or a multiplexer. What differs is the set of possible input(s) and/or output(s) a filter can handle.

A filter may process anything, be it a media stream, a file, a binary blob, etc. For example, GPAC unit test filters have no notion of what is a file or what is a media stream.
Filter Session Design Principles

The filter session main features are: 
- provide automatic link resolution between filters (based on Dijkstra)
- execute filters whenever input packets are available for a filter
- run in mono or multi-threaded mode
- use as few locks as possible, all packets exchanged being done using lock-free queues by default
- handle data packets and properties (see below) through reference counting
- recycle memory as much as possible
- real-time scheduling of filters requiring it (audio/video input/output, network input/output)
- manage blocking mode of the chain to avoid having a filter dispatching too many packets
- reconfigure filters whenever required, potentially replacing a sub-chain with another one
- handle filters capability negotiation, usually inserting a filter chain to match the desired format

The filter session operates in a semi-blocking mode:

- it prevents filters in blocking mode (output PIDs buffers full) to operate
- it will not prevent a running filter to dispatch a packet; this greatly simplifies demultiplexers writing
PID description

A PID object (in charge of transmitting data) such as a media stream usually has a bunch of information associated such as width, height, ... Since these properties will likely differ from one media type to another, and since the PID itself might carry anything but our usual media type, this information is expressed as a set of dynamic properties assigned to a PID by the filter in charge of emitting the packets, rather than struct/class members.  

Common properties for media streams are built-in within [libgpac](filters_properties), but filters can also use custom properties, without having to modify the filter core engine.  

These properties can be used to restrict the set of possible connections, or used to generate file names based on templating mechanisms. 

These properties may also be overloaded by the user, e.g. to assign a ServiceID for MPEG-2 TS or an AdaptationSet ID for MPEG-DASH.
Media Streams internal representation

In order to be able to exchange media stream data between filters, a unified data format had to be set, as follows:

- a frame is defined as a single-time block of data (Access Unit in MPEG terminology), but can be transferred in multiple packets
- frames or fragments of frames are always transferred in processing order (e.g. decoding order for MPEG video) 
- multiplexed media data is identified as `file` data, where a frame is a complete file.
- un-multiplexed media data is identified as `audio`, `video`, ... data, with a given codec identifier (including uncompressed media). 
 - if the frame payload follows the default internal format for that codec, the media stream is implicitly marked as `framed`
 - if the frame payload does not follow the default internal format for that codec, the media stream is explicitly marked as `unframed`

Consequently:
  - if `unframed` data has to be processed by a filter accepting only `framed` data (e.g. a decoder), this will require an intermediate filter to move from `unframed` to `framed`; such a filter is usually called a _reframer_ filter. Example of `unframed` data are AVC|H264 or HEVC in Annex B format (using start codes), AAC encapsulated in ADTS or LATM.  
  - if `framed` data has to be processed by a filter accepting only `unframed` data (e.g. a multiplexer or a raw stream writer), this will require an intermediate filter to move from `framed` to `unframed`; such a filter is usually called a _rewriter_ filter.

The default internal format used for frame payload usually follows the format defined for the storage of this media type in ISOBMFF, if any. Otherwise it is internally defined.  

The payload of a compressed frame __never__ contains any decoder configuration data such as AVC|H264 or HEVC parameter sets. This configuration data shall be set as a property of the PID, and will trigger reconfiguration of the filter whenever a packet with the new configuration is processed.

Packets carrying frames come with a set of built-in variables to express timing, random access, frame fragmentation, but they can also have associated properties just like PIDs. For example, NTP sampling clock of a frame or CENC subsample information are carried as properties since in many applications they will never be present.
MP4Box

Media importers and exporters in MP4Box have been replaced by a filter session used to import to ISOBMFF multiplexer or export to a given file.  

```MP4Box -add source.avc -new test.mp4```
  
This is equivalent to  
  
```gpac -i source.avc -o test.mp4```

However, adding to an existing file will require using MP4Box.

DASH segmentation has been replaced by a filter session used to segment a given set of files.  

```MP4Box -dash 1000 -out test.mpd source1.mp4 source2.mp4```  

This is equivalent to  

```gpac -i source1.mp4 -i source2.mp4 -o test.mpd```  

Setting the dash duration to something else than the default 1 second can be done by passing it as an option of the [dasher](dasher) filter, usually passed through arguments inheriting (see [general filter documentation](filters_general)): 

```gpac -i source1.mp4 -i source2.mp4 -o test.mpd:segdur=2.5```  

__WARNING__
The dasher filter is quite different from the old segmenter and the default algorithm will likely lead to different segmentation bounds. To have the same segmentation results, use [-bound](mp4box-dash-opts#bound) or [-closest](mp4box-dash-opts#closest) options.

Encryption and decryption have been replaced by a filter session used to en/de-crypt a single file.

```
MP4Box -crypt DRM.xml source.mp4 -out protected.mp4
MP4Box -decrypt DRM.xml protected.mp4 -out unprotected.mp4
```  

This is equivalent to  

```
gpac -i source.mp4 cecrypt:cfile=DRM.xml -o protected.mp4:xps_inband=auto
gpac -i protected.mp4 cdcrypt:cfile=DRM.xml -o unprotected.mp4:xps_inband=auto
```  

File splitting has been replaced by a filter session used to split source file.

```
MP4Box -splitx 2:4 source.mp4 -out dest.mp4
```  

This is equivalent to  

```
gpac -i source.mp4 reframer:xround=closest:splitrange:xs=2:xe=4 -o dest.mp4
```  


All other functionalities of MP4Box are not available through a filter session. Some might make it one day (BIFS encoding for example), but most of them are not good candidates for filter-based processing and will only be available through MP4Box (track add/remove to existing file, image item add/remove to existing file, file hinting, ...).

__Note__ For operations using a filter session in MP4Box, it is possible to view some information about the filter session:

- -fstat: this will print the statistics per filter and per PID of the session
- -fgraph: this will print the connections between the filters in the session
MP4Client [DEPRECATED as of GPAC 2.2]

MP4Client (and consequently the _GF_Terminal_ API) is now a wrapper to a filter session running in multi-threaded mode by default, and using the [compositor](compositor) filter as a sink filter for video. 

All avi/raw extraction functions from MP4Client have been deprecated, as they are provided by the generic filter management [gpac](gpac_general) application. e.g.:

```MP4Client -avi test.bt```

no longer works, but is now achieved using

```gpac -i test.bt compositor -o test.avi```

See [compositor filter](compositor) for options such as fps, duration, etc.

The player mode of MP4Client is still here obviously and cannot be completely emulated by the gpac application due to its handling of user events (navigation/hyperlinks, command-line interactivity etc).
MP42TS

MP42TS has been deprecated, replaced by the generic filter management gpac application. 

For file production:

```MP42TS -src source.mp4 -dst-file test.ts```

This is now achieved using  

```gpac -i source1.mp4 -o test.ts```  


For live production:

```MP42TS -src source.mp4 -dst-udp 127.0.0.1:1234```

This is now achieved using  

```gpac -i source1.mp4 -o udp://127.0.0.1:1234/:ext=ts```  

MP42TS was limited to RTP and MP4 input. This is no longer the case with gpac, you can use any source you want to build your TS (e.g. pipes, MPD session, ...).
DashCast

[DashCast](https://gpac.io/dashcast/) has been deprecated, replaced by the generic filter management gpac application.

For example, this will produce a dash session using a single source and two qualities/rates encoding  

```gpac -i source.avc:FID=1 ffsws:osize=512x512:SID=1 @ ffenc:c=avc:fintra=1:FID=EV1 ffsws:osize=256x256:SID=1 @ ffenc:c=avc:fintra=1:FID=EV2 -o file.mpd:profile=live:SID=EV1,EV2```

or using GPAC 2.0 implicit linking:

```gpac -i source ffsws:osize=512x512 c=avc:fintra=1 @@ ffsws:osize=256x256 c=avc:fintra=1 -o dash/file.mpd -graph```
  
You can also use a live audio/video grabber using the filter [ffavin](ffavin), or any other filter you want!
Options  {.no-collapse}

<div markdown class="option">  
<a id="force_pf" data-level="basic">__force_pf__</a> (bool, default: _false_): ignore possible mapping to GPAC pixel formats  
</div>  
<div markdown class="option">  
<a id="no_tile" data-level="basic">__no_tile__</a> (bool, default: _false_): ignore tiling info (debug)  
</div>
Overview {:data-level="all"}

This part of the wiki describes general concepts of the GPAC filter architecture, available starting from GPAC 0.9.0.

The new architecture allows describing media pipelines as a sequence of processing blocks called *filters*. Filters in GPAC can be pretty much anything: file/pipe/network access objects, (de-)multiplexers, de/encoders, media segmenters (for HTTP Adaptive Streaming), RTSP server, playlist manager and of course raw domain effects.

For more information on this rearchitecture, [look here](Rearchitecture).

Historical applications of GPAC (MP4Box, players) typically provide a fixed media pipeline (import or playback) based on a subset of GPAC filters; these pipelines may optionally be extended with custom filters, as [illustrated for MP4Box](mp4box-filters).

These applications cannot however create other media pipelines than their built-in ones; a new application, called [gpac](gpac_general), has been added to GPAC to allow building completely custom media pipelines, as described [here](filters_general).
 
In this section of the website, you will find the documentation of all options of libgpac, the gpac application and all filters currently built-in. As of 0.9.0, this documentation is automatically generated from the source tree at each new commit.
DVB for Linux

Register name used to load filter: __dvbin__  
This filter may be automatically loaded during graph resolution.  
  
Experimental DVB support for linux, requires a channel config file through [chcfg](#chcfg)  
    
The URL syntax is `dvb://CHANNAME[@FRONTEND]`, with:  

    - CHANNAME: the channel name as listed in the channel config file  
    - frontend: the index of the DVB adapter to use (optional, default is 0)
Options  {.no-collapse}

<div markdown class="option">  
<a id="src" data-level="basic">__src__</a> (cstr): URL of source content  
</div>  
<div markdown class="option">  
<a id="block_size">__block_size__</a> (uint, default: _65536_): block size used to read file  
</div>  
<div markdown class="option">  
<a id="chcfg" data-level="basic">__chcfg__</a> (cstr): path to channels.conf file  
</div>
Options  {.no-collapse}

<div markdown class="option">  
<a id="index" data-level="basic">__index__</a> (dbl, default: _1.0_): indexing window length (not implemented), use 0 to disable stream probing for duration),   
</div>  
<div markdown class="option">  
<a id="expart" data-level="basic">__expart__</a> (bool, default: _false_): expose pictures as a dedicated video PID  
</div>
MPEH-H Audio Stream reframer

Register name used to load filter: __rfmhas__  
This filter may be automatically loaded during graph resolution.  
  
This filter parses MHAS files/data and outputs corresponding audio PID and frames.  
By default, the filter expects a MHAS stream with SYNC packets set, otherwise tune-in will fail. Using [nosync](#nosync)=false can help parsing bitstreams with no SYNC packets.  
The default behavior is to dispatch a framed MHAS bitstream. To demultiplex into a raw MPEG-H Audio, use [mpha](#mpha).
Options  {.no-collapse}

<div markdown class="option">  
<a id="index" data-level="basic">__index__</a> (dbl, default: _1.0_): indexing window length  
</div>  
<div markdown class="option">  
<a id="mpha" data-level="basic">__mpha__</a> (bool, default: _false_): demultiplex MHAS and only forward audio frames  
</div>  
<div markdown class="option">  
<a id="pcksync" data-level="basic">__pcksync__</a> (uint, default: _4_): number of unknown packets to tolerate before considering sync is lost  
</div>  
<div markdown class="option">  
<a id="nosync" data-level="basic">__nosync__</a> (bool, default: _true_): initial sync state  
</div>
MPEG Transport Stream splitter

Register name used to load filter: __tssplit__  
This filter is not checked during graph resolution and needs explicit loading.  
  
This filter splits an MPEG-2 transport stream into several single program transport streams.  
Only the PAT table is rewritten, other tables (PAT, PMT) and streams (PES) are forwarded as is.  
If [dvb](#dvb) is set, global DVB tables of the input multiplex are forwarded to each output mux; otherwise these tables are discarded.
Options  {.no-collapse}

<div markdown class="option">  
<a id="dvb">__dvb__</a> (bool, default: _true_): forward all packets from global DVB PIDs  
</div>  
<div markdown class="option">  
<a id="mux_id">__mux_id__</a> (sint, default: _-1_): set initial ID of output mux; the first program will use mux_id, the second mux_id+1, etc. If not set, this value will be set to sourceMuxId*255  
</div>  
<div markdown class="option">  
<a id="avonly">__avonly__</a> (bool, default: _true_): do not forward programs with no AV component  
</div>  
<div markdown class="option">  
<a id="nb_pack">__nb_pack__</a> (uint, default: _10_): pack N packets before sending  
</div>
Compressed layered bitstream aggregator

Register name used to load filter: __bsagg__  
This filter is not checked during graph resolution and needs explicit loading.  
Filters of this class can connect to each-other.  
  
This filter aggregates layers and sublayers into a single output PID.  
  
The filter supports AVC|H264, HEVC and VVC stream reconstruction, and is passthrough for other codec types.  
  
Aggregation is based on temporalID value (start from 1) and layerID value (start from 0).  
For AVC|H264, layerID is the dependency value, or quality value if `svcqid` is set.  
  
The filter can also be used on AVC and HEVC DolbyVision dual-streams to aggregate base stream and DV RPU/EL.  
  
The filter does not forward aggregator or extractor NAL units.
ROUTE input

Register name used to load filter: __routein__  
This filter may be automatically loaded during graph resolution.  
  
This filter is a receiver for file delivery over multicast. It currently supports ATSC 3.0, generic ROUTE and DVB-MABR flute.  

- ATSC 3.0 mode is identified by the URL `atsc://`.  
- Generic ROUTE mode is identified by the URL `route://IP:PORT`.  
- DVB-MABR mode is identified by the URL `mabr://IP:PORT` pointing to the bootstrap FLUTE channel carrying the multicast gateway configuration.  

  
The filter can work in cached mode, source mode or standalone mode.
Cached mode

The cached mode is the default filter behavior. It populates GPAC HTTP Cache with the received files, using `http://gmcast/serviceN/` as service root, `N being the multicast service ID.`  
In cached mode, repeated files are always pushed to cache.  
The maximum number of media segment objects in cache per service is defined by [nbcached](#nbcached); this is a safety used to force object removal in case DASH client timing is wrong and some files are never requested at cache level.  
    
The cached MPD is assigned the following headers:  

- `x-mcast`: boolean value, if `yes` indicates the file comes from a multicast.  
- `x-mcast-first-seg`: string value, indicates the name of the first segment (completely or currently being) retrieved from the broadcast.  
- `x-mcast-ll`: boolean value, if yes indicates that the indicated first segment is currently being received (low latency signaling).  
- `x-mcast-loop`: boolean value, if yes indicates a loop (e.g. pcap replay) in the service has been detected - only checked if [cloop](#cloop) is set.  

    
The cached files are assigned the following headers:  

- `x-mcast`: boolean value, if `yes` indicates the file comes from a multicast.  

  
If [max_segs](#max_segs) is set, file deletion event will be triggered in the filter chain.
Source mode

In source mode, the filter outputs files on a single output PID of type `file`. The files are dispatched once fully received, the output PID carries a sequence of complete files. Repeated files are not sent unless requested.  
If needed, one PID per TSI can be used rather than a single PID. This avoids mixing files of different mime types on the same PID (e.g. HAS manifest and ISOBMFF).  
Example
```
gpac -i atsc://gcache=false -o $ServiceID$/$File$:dynext
```
  
This will grab the files and forward them as output PIDs, consumed by the [fout](fout) filter.  
  
If [max_segs](#max_segs) is set, file deletion event will be triggered in the filter chain.
File Repair

In case of losses or incomplete segment reception (during tune-in), the files are patched as follows:  

- MPEG-2 TS: all lost ranges are adjusted to 188-bytes boundaries, and transformed into NULL TS packets.  
- ISOBMFF: all top-level boxes are scanned, and incomplete boxes are transformed in `free` boxes, except mdat kept as is if [repair](#repair) is set to simple.  

  
If [kc](#kc) option is set, corrupted files will be kept. If [fullseg](#fullseg) is not set and files are only partially received, they will be kept.
Interface setup

On some systems (OSX), when using VM packet replay, you may need to force multicast routing on your local interface.  
For ATSC, you will have to do this for the base signaling multicast (224.0.23.60):  
Example
```
route add -net 224.0.23.60/32 -interface vboxnet0
```
  
Then for each multicast service in the multicast:  
Example
```
route add -net 239.255.1.4/32 -interface vboxnet0
```
Thumbnail collection generator

Register name used to load filter: __thumbs__  
This is a JavaScript filter. It is not checked during graph resolution and needs explicit loading.  
Author: GPAC team  
  
This filter generates screenshots from a video stream.  
  
The input video is down-sampled by the [scale](#scale) factor. The output size is configured based on the number of images per line and per column in the [grid](#grid).   
Once configured, the output size is no longer modified.  
  
The [snap](#snap) option indicates to use one video frame every given seconds. If value is 0, all input frames are used.  
  
If the number of rows is 0, it will be computed based on the source duration and desired [snap](#snap) time, and will default to 10 if it cannot be resolved.  
  
To output one image per input frame, use `:grid=1x1`.  
  
If a single image per output frame is used, the default value for [snap](#snap) is 0 and for [scale](#scale) is 1.  
Otherwise, the default value for [snap](#snap) is 1 second and for [scale](#scale) is 10.  
  
A single line of text can be inserted over each frame. Predefined keywords can be used in input text, identified as `$KEYWORD$`:  

- ts: replaced by packet timestamp  
- timescale: replaced by PID timescale  
- time: replaced by packet time as HH:MM:SS.ms  
- cpu: replaced by current CPU usage of process  
- mem: replaced by current memory usage of process  
- version: replaced by GPAC version  
- fversion: replaced by GPAC full version  
- mae: replaced by Mean Absolute Error with previous frame  
- mse: replaced by Mean Square Error with previous frame  
- P4CC, PropName: replaced by corresponding PID property  

  
Example
```
gpac -i src reframer:saps=1 thumbs:snap=30:grid=6x30 -o dump/$num$.png
```
  
This will generate images from key-frames only, inserting one image every 30 seconds. Using key-frame filtering is much faster but may give unexpected results if there are not enough key-frames in the source.  
  
Example
```
gpac -i src thumbs:snap=0:grid=5x5 -o dump/$num$.png
```
  
This will generate one image containing 25 frames every second at 25 fps.  
  
If a single image per output frame is used and the scaling factor is 1, the input packet is reused as input with text and graphics overlaid.  
  
Example
```
gpac -i src thumbs:grid=1x1:txt='Frame $time$' -o dump/$num$.png
```
  
This will inject text over each frame and keep timing and other packet properties.  
  
A json output can be specified in input [list](#list) to let applications retrieve frame position in output image from its timing.
Scene change detection

The filter can compute the absolute and/or square error metrics between consecutive images and drop image if the computed metric is less than the given threshold.  
If both [mae](#mae) and [mse](#mse) thresholds are 0, scene detection is not performed (default).  
If both [mae](#mae) and [mse](#mse) thresholds are not 0, the frame is added if it passes both thresholds.  
  
For both metrics, a value of 0 means all pixels are the same, a value of 100 means all pixels have 100% intensity difference (e.g. black versus white).  
  
The scene detection is performed after the [snap](#snap) filtering and uses:  

- the previous frame in the stream, whether it was added or not, if [scref](#scref) is not set,  
- the last added frame otherwise.  

  
Typical thresholds for scene cut detection are 14 to 20 for [mae](#mae) and 5 to 7 for [mse](#mse).  
  
Since this is a costly process, it is recommended to use it combined with key-frames selection:  
  
Example
```
gpac -i src reframer:saps=1 thumbs:mae=15 -o dump/$num$.png
```
  
  
The [maxsnap](#maxsnap) option can be used to force insertion after the given time if no scene cut is found.