gaspard7/wiki_gpac huggingface dataset:
Inspecting filters

You can query the number of input and output PIDs of a filter, the source filter of an input PID, the destination filters of an output PID, their possible options, update options, send events, ...

Please check the  [API documentation](https://doxygen.gpac.io/group__nodejs__grp.html) and refer to the [NodeJS example](https://github.com/gpac/gpac/blob/master/share/nodejs/test/gpac.js).

Note that  (as in GPAC JS or Python) properties referring to constant values are not exposed as their native types but as strings. This is the case for these important types:

- StreamType: string containing the stream type name (e.g. 'Visual', 'Audio', ...)
- CodecID: string containing the codec name
- PixelFormat: string containing the pixel format name
- AudioFormat: string containing the audio format name
Custom Filters {: data-level="expert" }

You can define your own filter(s) to interact with the media pipeline. As usual in GPAC filters, a custom filter can be a source, a sink or any other filter. It can consume packets from input PIDs and produce packets on output PIDs. 

Custom filters are created through the `new_filter` function of the filter session object. The custom filter can then assign its callbacks functions:

- `GF_Err process()` method called whenever the filter has some data to process.
- `GF_Err configure_pid(pid, is_remove)` method called whenever a new PID must be configured, re-configured or removed in the custom filter
- `Bool process_event(evt)` method called whenever an event is passing through the filter or one of its PIDs
- `GF_Err reconfigure_output(pid)` method called whenever an output PID should be reconfigured

These callbacks are all optional, but `process` should be set if you want your filter to perform any action.

Filters accepting input PIDs and/or producing output PIDs must configure their capabilities using `push_cap` function.

_NOTE When running the session in multi-thread mode, custom filter callbacks are always executed by the main thread (NodeJS main)._
Custom Sink example

The following defines a custom filter doing simple inspection of the pipeline (sink filter) 
```

let cust = fs.new_filter('MyFilterJS');
cust.ipids=[];

//indicate what we accept and produce - this can be done ether in the constructor or after, but before running the session
//here we only accept video streams as input, and do not produce any output
cust.push_cap("StreamType", "Visual", gpac.GF_CAPS_INPUT);

//we accept one or more input video PID, we must configure them
cust.configure_pid = (pid, is_remove) => {
	if (is_remove) {
		console.log('PID removed !');
		return gpac.GF_OK;
	}
	//PID is already registered with our filter, this is a reconfiguration
	if (this.ipids.indexOf(pid) < 0) {
		this.ipids.push(pid);
		console.log('PID initial configure !');
	} else {
		console.log('PID reconfigure !');
	}

	//enumerate all props 
	pid.enum_props( (type, val) => {
			console.log('\t' + type + ': ' + val);
	});
	
	//we are a sink, we MUST fire a play event
	evt = new gpac.FilterEvent(gpac.GF_FEVT_PLAY);
	pid.send_event(evt);
	return gpac.GF_OK;
};

cust.process() = () => {
	this.ipids.forEach(pid => {
		pck = pid.get_packet();
		if (!pck) break;
		console.log('Got Packet DTS ' + str(pck.dts) + ' CTS ' + str(pck.cts) + ' SAP ' + str(pck.sap) + ' dur ' + str(pck.dur) + ' size ' + str(pck.size) );
		pid.drop_packet();
	}
	return gpac.GF_OK;
};

//load a source
let my_src = fs.load_src("source.mp4");

//if needed, setup links between filters (in this example, only 2 filters explicitly loaded, no need for links)

//run the session
fs.run();

```
Custom Forwarding example

The following defines a custom filter doing packet forwarding for input AV streams in the middle of the pipeline, exercising all possible packet creation modes (new, clone, copy, forward by ref, forward data by ref).
```


let cust_f = fs.new_filter("NodeJS_Test");
//we accept any number of input PIDs
cust_f.set_max_pids(-1);
cust_f.pids = [];
cust_f.push_cap('StreamType', 'Visual', gpac.GF_CAPS_INPUT_OUTPUT);
cust_f.push_cap('StreamType', 'Audio', gpac.GF_CAPS_INPUT_OUTPUT);

cust_f.configure_pid = function(pid, is_remove)
{
	if (this.pids.indexOf(pid) < 0) {
		this.pids.push(pid);

		//create output PID
		pid.opid = this.new_pid();
		pid.opid.copy_props(pid);
	} else if (is_remove) {
		console.log('PID remove !');
	} else {
		console.log('PID reconfigure !');
		pid.opid.copy_props(pid);
	}
	return gpac.GF_OK;
}

cust_f.pck_clone_cache = null;

cust_f.process = function() {
	let nb_eos=0;

	this.pids.forEach(pid =>{
	if (pid.eos) {
		nb_eos++;
		return;
	}
	let pck = pid.get_packet();
	if (!pck) return;

	//send by reference
	if (this.fwd_mode==1) {
		let dst = pid.opid.new_pck_ref(pck);
		dst.copy_props(pck);
		dst.send();
	}
	//full copy mode
	else if (this.fwd_mode==2) {
		let dst = pid.opid.new_pck(pck.size);
		dst.copy_props(pck);
		new Uint8Array(dst.data).set(new Uint8Array(pck.data));
		dst.send();
	}
	//keep ref to packet and send new packet using shared data
	else if (this.fwd_mode==3) {
		let ab = new ArrayBuffer(pck.size);
		new Uint8Array(ab).set(new Uint8Array(pck.data));
		//keep ref
		pck.ref();
		let dst = pid.opid.new_pck_shared(ab, () => { pck.unref();});
		dst.copy_props(pck);
		dst.send();
	}
	//pck_clone
	else if (this.fwd_mode==4) {
		let dst = pid.opid.new_pck_clone(pck);
		dst.send();
	}
	//pck_copy
	else if (this.fwd_mode==5) {
		let dst = pid.opid.new_pck_copy(pck);
		dst.send();
	}
	//else (0) direct packet forward
	else {
		pid.opid.forward(pck);
	}

	this.fwd_mode++;
	if (this.fwd_mode==6) this.fwd_mode=0;

	pid.drop_packet();
	});

	if (nb_eos == this.pids.length)
	return gpac.GF_EOS;

	return gpac.GF_OK;
}

//load a source filter
let src=fs.load_src("source.mp4");

//load a destination filter
let dst=fs.load("vout");

//we need to indicate that our destination only gets its input from our custom filter !
dst.set_source(cust_f);
Remotery interaction

GPAC is by default compiled with [Remotery](https://github.com/Celtoys/Remotery) support for remote profiling. 
You can interact with Remotery websocket server by sending messages to the remote browser, or receiving messages from it:

```
gpac.set_rmt_fun( text => {
	console.log('Remotery got message ' + text);
	gpac.rmt_send('Some response text');
});

```

You will need to enable Remotery in GPAC by setting the option `-rmt`, as this cannot be enabled or disabled at run time.

You can however enable or disable Remotery profiler using `gpac.rmt_enable(true)`.
DASH Client

You can override the default algorithm used by the DASH client with your own algorithm. See [the documentation](https://doxygen.gpac.io/classlibgpac_1_1_filter.html#a05de5bc6b3cb9a3573e00d9f4ccfc056) for further details.

The principle is as follows:

- the script can get notification when a period start/end to reset your stats and setup live vs on demand cases
- the script can get notified of each created group (AdaptationSet in DASH, Variant Stream in HLS) with its various qualities. For HEVC tiling, each tile will be declared as a group, as well as the base tile track
- the script is notified after each segment download on which quality to pickup next
- the script can get notified while downloading a segment to decide if the download should be aborted
 
```

let dash_algo = {};
dash_algo.on_period_reset = (type) => {
	console.log('period reset type ' + type);
};
dash_algo.on_new_group = (group) => {
	console.log('Got new group ' + JSON.stringify(group) );
};

dash_algo.on_rate_adaptation = (group, base_group, force_lower_complexity, stats) =>
{
	console.log('Rate adaptation on group ' + group.idx + ' - stats ' + JSON.stringify(stats) );
	//always use lowest quality in this example
	return 0;
};

dash_algo.on_download_monitor = (group, stats) =>
{
	console.log('Download monitor on group ' + group.idx + ' - stats ' + JSON.stringify(stats) );
	return -1;

};

let fs = new gpac.FilterSession();

fs.on_filter_new = (f) => {
	if (f.name == "dashin")
		f.bind(dash_algo);
};

//load a source, here to TelecomParis DASH test sequences
let f1 = fs.load_src("https://download.tsi.telecom-paristech.fr/gpac/DASH_CONFORMANCE/TelecomParisTech/mp4-live-1s/mp4-live-1s-mpd-AV-BS.mpd");

//load a sink, here video out
let f2 = fs.load("vout");

//run the session in blocking mode
fs.run();


```
HTTP Server

You can override the default behaviour of the httpout filter. See [the documentation](https://doxygen.gpac.io/group__nodehttp__grp.html) for further details.

The principle is as follows:

- the script can get notification of each new request being received
- the script can decide to let GPAC handle the request as usual (typically used for injecting http headers, throttling and monitoring)
- the script can feed the data to GPAC (GET) or receive the data from GPAC (PUT/POST)

 
```

let http_req = {
	on_request = (request) => {
		console.log('got request ' + JSON.stringify(request) );
		request.reply=0;
		//throttle the connection, always delaying by 100 us
		request.throttle = function(done, total) {
			return 100;
		}
		//check end of request
		request.close = function(error) {
			print('closed with code ' + error_string(error));
		}
		request.send();
	};
};

let fs = new gpac.FilterSession();

fs.on_filter_new = (f) => {
	if (f.name == "httpout")
		f.bind(http_req);
};

//load the server
let f2 = fs.load("httpout:port=8080:rdirs=.");

//run the session in blocking mode
fs.run();

```

The following script always serves the same file content using nodejs instead of GPAC for GET, and monitor bytes received for PUT/POST:

```
let http_req = {
	on_request = (request) => {
		console.log('got request ' + JSON.stringify(request) );
		request.reply=200;
		this.file = filesys.openSync("source.mp4", "r");
		request.read = function(ab) {
			let nb_bytes = 0;
			try {
				nb_bytes = filesys.readSync(this.file, ab, 0, ab.length);
			} catch (e) {
				console.log('read error: ' + e);
				return 0;
			}
			return nb_bytes;
		}
		request.write = function(ab) {
			console.log("got " + ab.length + " bytes");
		}
		//check end of request
		request.close = function(error) {
			print('closed with code ' + error_string(error));
		}
		//send reply - this can also be done later on, e.g. in a user tasjk
		request.send();
	};
};
```
Multithread support {: data-level="expert" }

Multithreaded filter sessions can be used with NodeJS, however the binding currently only supports executing callbacks into NodeJS from the main thread (main NodeJS or worker). 

A multithreaded session is created by specifying the `-threads=N` option to libgpac:

```
gpac.set_args(['libgpac', '-threads=2']);

```
 
This implies a few limitations detailed below.
Remotery handling

Remotery executes in its own thread, therefore remotery messages are always queued and flushed while running a session.

If no filter session is active, remotery messages will not be dispatched.

If you want to flush these messages independently from your media sessions, the simplest way is to create a non-blocking session with a user task running until the end of your program.

```
gpac.set_args(['libgpac', '-rmt']);

gpac.set_rmt_fun( (msg) => {
	console.log('RMT got message ' + msg);
	gpac.rmt_send('ACK for ' + msg);
}); 

let fs_rmt = new gpac.FilterSession(gpac.GF_FS_FLAG_NON_BLOCKING);
//create a simple task running forever every 100 ms - you can return false once you are done
let rmt_task = {
	execute: function() { return 100; }
};
fs_rmt.post_task(rmt_task);
//run as promise
```
Simple reception of data

Assume you have a source that dispatches AVC|H264 in Annex B format. If you direct this source to a pipe `myavcpipe`, you can read from it using:
 
```gpac -i pipe://myavcpipe:ext=avc -o test.mp4```

The above command will open the pipe  `myavcpipe`, read AVC|H264 Annex B data from it and use it to mux MP4 data. Obviously you can use whatever output and processing:
```gpac -i pipe://myavcpipe:ext=avc ffsws:osize=640x360 -o test.yuv```

This will grab the source, resize it and dump it to file.


__Discussion__  
The  `ext` option is used to indicate the input format of the data carried on the pipe. If not set, the first received block of data will be used to probe for media format. Most mux format used by GPAC can be used on input pipes; a notable exception is ISOBMFF which is not supported, even in fragmented mode (this should be patched soon).
Advanced reception of data

Assume you want gpac to grab from a pipe some  AVC|H264 content which could be sent sequentially by sender(s). typically, an encoder processing a playlist and sending each result to a pipe. Having the encoder create the pipe will result in closing the pipe at each file, which is not the goal. We need to
- create the pipe, either manually or by asking the [pipe](pin) input to create it for us using [mkp](pin) option
- tell the pipe input to keep the session alive even when broken pipe messages are received, using [ka](pin) option


```gpac -i pipe://myavcpipe:ext=avc:mkp:ka -o test.mp4```

This will open the pipe, creating it if not found, and we can now start sending data on the pipe. 

__Discussion__
The above command will run forever, since broken pipe messages are ignored. You can abort the session using `ctr+c` and ask to flush pending data to disk. If you have enabled prompt interactivity in gpac [-k](gpac_general), simply press `q`.
There is currently no way to signal from the sender that the session should be closed, we might add this feature in the near future.
Output pipe

Assume you have an app that consumes AVC|H264 in Annex B format from a pipe `myavcpipe`. You can direct GPAC output to this pipe:
 
```gpac -i source.mp4 -o pipe://myavcpipe:ext=avc```

The above command will open the file `source.mp4`, open the pipe  `myavcpipe` and send ([potentially transcoding](encoding)) the AVC|H264 data to this pipe. Obviously you can use whatever input and processing:
```gpac -i source.yuv ffsws:osize=640x360 -o pipe://myavcpipe:ext=avc```

This will grab the source, resize it, encode it and write it to the pipe.


__Discussion__  
The  `ext` option is used to indicate the output format of the data carried on the pipe. If SHALL be set. Any mux format used by GPAC can be used on output [pipes](pout). 

You should however be very careful using containers (e.g. ISOBMFF) as a format, since some containers format will require the entire file to be produced to make it valid. For example, using ISOBMFF (`mp4`) as a data format will fail, since we need to patch either the `moov` or `mdat` box at the end of the multiplexing. However, if using fragmented ISOBMFF, it will be possible to push the mux result to a pipe.
This problematic of patching previous blocks of data in a file is signalled by the property `DisableProgressive` on the `file` PID. At the current time, only the [file output](fout) filter can support PIDs with this property set.
Overview {: data-level="all" }

As of GPAC 2.0, MP4Box supports in-place editing of MP4 files.

In-place editing is used whenever the following conditions are true:

- the media data has not been modified during the edit operations
- no storage mode is specified
- no output file name is specified
- no movie fragments are present

This implies that import, removal or concatenation of tracks or items will disable inplace editing.

_Note_ In-place storage is enabled by default but can be disabled using `-no-inplace` option.

When in-place editing is used, MP4Box will display the following message:
```
Saving FILE: In-place rewrite
```

In-place editing is useful when you need to modify very large media files without modifying the media data.
In-place editing works by shifting media data in the file and updating file offsets in the structure info, therefore the speed of the edition depends on the original storage mode and file structure, as discussed below.   

In-place editing is supported for both movies and image (HEIF) non-fragmented files. Fragmented files are always de-fragmented when doing edits on the file.

For example, adding a brand and itunes tags to your `movie.mp4` file:

```
MP4Box -ab TEST -itags artist=GPAC movie.mp4
```

Tests for in-place storage are available [here](https://github.com/gpac/testsuite/blob/master/scripts/mp4box-inplace.sh).
Flat storage files

In files stored with flat storage (`-flat` in MP4Box), the media data is placed before the structured data (`moov` and `meta`  box) and usually does not need any shifting. There is one exception to this: when adding brands, since they must be located first in the file, it is necessary to shift the media data. There is currently no way to reserve space for future brand edition in MP4Box, and any brand add operation will result in media data shift. 

We therefore recommend using interleaved storage (`moov`/ `meta`  first).
Interleaved files

In files stored with interleaved storage (`-inter` in MP4Box), the media data is placed after the structured data (`moov` and `meta`  box) and may need shifting whenever the size of {ftyp, moov, meta} boxes changes. In order to avoid shifting when this size decreases, a `free` box is inserted before the media data.
When this size increases, the media data is shifted if writing the structured data will overwrite the start of the media data. Otherwise, a `free` box is written between the `moov+meta` boxes and  the media data.

You can control how much free space you want to reserve for future editions by using `-moovpad` option.
   
```
MP4Box -add SRC -new movie.mp4 -moovpad 1000
```

This will create a new file from `SRC`, leaving 1000 bytes of free space before the start of the media data, thereby allowing for roughly 1000 bytes size increase of the structured data without having to shift the media data. If the structured size increases over this reservoir, the media data will be shifted.

You can use `-moovpad` option in your in-place edit operations, to force moving the media data (reserving more space):
 
```
MP4Box -ab GPAC movie.mp4 -moovpad 5000
```

This will perform in-place edit if the previously created file, but shift the media data to have 5000 bytes reserved for future edition; if more than 5000 free bytes are available, the media data is not shifted (i.e. `moovpad` is ignored in this case).


_WARNING_ When using in-place editing with `moov/meta` stored first,  MP4Box reclaims all possible empty space in `moov/meta`  boxes by recursively purging all child boxes with type `free` or `skip`.
Foreword

This page contains one-liners illustrating the many possibilities of GPAC filters architecture. For a more detailed information, it is highly recommended that you read:

- the [general concepts](filters_general) page 
- the [gpac application](gpac_general)  help

To get a better understanding of each command illustrated in this case, it is recommended to:

- run the same command with `-graph` specified to see the filter graph associated
- read the help of the different filters in this graph using `gpac -h filter_name`

Whenever an option is specified, e.g. `dest.mp4:foo`, you can get more info and locate the parent filter of this option  using `gpac -h foo`.

The filter session is by default quiet, except for warnings and error reporting. To get information on the session while running, use [-r](gpac_general#r) option. To get more runtime information, use the [log system](core_logs).
sectionLevel === null || 
Given the configurable nature of the filter architecture, most examples given in one context can be reused in another context. For example:

- from the dump examples:
```
gpac -i source reframer:saps=1 -o dump/$num$.png
```

- from the encode examples:
```
gpac -i source ffsws:osize=320x240 enc:c=avc:b=1m -o dst.mp4
```

This can be recombine as dump key frames to png while rescaling:
```
gpac -i source reframer:saps=1 ffsws:osize=128x72 -o dump/$num$.png
```

_NOTE The command lines given here are usually using a local file for source or destination URLs, but they can work with remote URLs as well._


_Reminder_
Most filters are never specified at the prompt, they are dynamically loaded during the graph resolution.
GPAC filters can use either:

- global options, e.g. `--foo`, applying to each instance of any filter defining the `foo` option,
- local options to a given filter and any filters dynamically loaded, e.g. `:foo`. This is called [argument inheriting](filters_general#arguments-inheriting).  



_NOTE This page is under permanent construction, feel free to contribute !_
Source Inspection {: data-level="beginner" }

Check if a source is supported and get quick information on the media streams - _see [filter](inspect), [howto](inspecting)_  
```
gpac -i source inspect
gpac -info source
```

Check if a source is supported and get verbose information on the media streams - _see [filter](inspect), [howto](inspecting)_  
```
gpac -i source inspect:full
```


Check if a source is supported and get all possible media stream states in the media streams (codec change & co)  - _see [filter](inspect), [howto](inspecting)_  
```
gpac -i source inspect:allp
```


Dump all packets info of a source from 60 to 80 seconds, listing one stream after the other  - _see  [filter](inspect)_  
```
gpac -i source inspect:deep:start=60:dur=20:interleave=false
```


Dump CTS, DTS, random access and size of packets of a source to a log file - _see [filter](inspect), [howto](inspecting)_  
```
gpac -i source inspect:deep:fmt="%dts% %dts% %sap% %size%%lf":log=mylogs.txt
```


Analyze packets of a source - _see [filter](inspect), [howto](inspecting)_  
```
gpac -i source.264 inspect:deep:analyze=on
```


_Note_ This is similar to (but more complete than) MP4Box [-dnal](mp4box-dump-opts#dnal) option

Count all streams in a source - _see [filter](probe),  [doc](filters_general#complex-links)_  
 ```
res = `gpac -i source probe`
```

Count all AC3 audio streams in a source - _see [filter](probe),  [doc](filters_general#complex-links)_  
 ```
res = `gpac -i source @#CodecID=ac3 probe`
```
Dumping and decoding

Extract AVC track, potentially transcoding - _see [filter](writegen)_  
```
gpac -i source -o dst.264
```

Extract AVC track if any - _see [filter](writegen)_  
```
gpac -i source @#CodecID=avc -o dst.264
```

Extract AAC track - _see [filter](writegen)_  
```
gpac -i source -o dst.aac
```

Decode audio track to 16 bit pcm - _see [filter](writegen)_  
```
gpac -i source -o dst.wav
```

Decode video track to YUV 420 - _see [filter](writegen)_  
```
gpac -i source -o dst_$Width_$Height$.yuv
```

Decode video track starting from frame 2s for 1s to YUV 420 - _see [filter](writegen)_  
```
gpac -i source -o dst_$Width_$Height$.yuv:start=2:dur=1
```

Dump video track key frames to PNG _see [filter](reframer)_  
```
gpac -i source reframer:saps=1 -o dump/$num$.png
```

Extract all tracks - _see [filter](writegen)_  
```
gpac -i source reframer -o dump_$ID$:dynext
```

Extract all audio tracks - _see [doc](filters_general#complex-links)_  
```
gpac -i source reframer @#audio -o dump_$ID$:dynext
```

Extract all AAC audio tracks - _see [doc](filters_general#complex-links)_  
```
gpac -i source reframer @#CodecID=aac -o dump_$ID$:dynext
```
Multiplexing {: data-level="beginner" }

Mux sources to MP4 - _see [filter](mp4mx)_  
```
gpac -i source -i source2 -o dst.mp4
```

Mux sources to MP4 specifying language - _see [doc](filters_general#assigning-pid-properties)_  
```
gpac -i source -i source2:#Language=fr -o dst.mp4
```

Mux sources to fMP4 - _see [filter](mp4mx)_  
```
gpac -i source -i source2 -o dst.mp4:frag
```

Force remultiplexing an MP4 (all syntaxes are equivalent) - _see [filter](mp4mx)_  
```
gpac -i source.mp4 reframer -o dst.mp4
gpac -i source.mp4 -o dst.mp4:remux
gpac -i source.mp4 -o dst.mp4:store=inter
gpac -i source.mp4 -o dst.mp4:inter
```

Mux sources to single program MPEG-2 TS - _see [filter](m2tsmx)_  
```
gpac -i source -i source2 -o dst.ts
```

Mux sources to MPEG-2 TS, specifying service ID - _see [filter](m2tsmx)_  
```
gpac -i source:#ServiceID=1 -i source2:#ServiceID=2 -o dst.ts
```

Mux sources to MPEG-2 TS with a target mux rate - _see [filter](m2tsmx)_  
```
gpac -i source:#ServiceID=1 -i source2:#ServiceID=2 -o dst.ts:rate=10m
```

Mux sources to MKV (for builds with FFmpeg support) - _see [filter](ffmx)_  
```
gpac -i source -o dst.mkv
```