gaspard7/wiki_gpac huggingface dataset:
TextSampleDescription

The TextSampleDescription element may be present as many times as desired in a TextStreamHeader. It defines the default styles text samples may refer to in the stream. The TextSampleDescription may also contain zero or one `FontTable`, `TextBox` and `Style`.

### Syntax

```xml
<TextSampleDescription horizontalJustification="..." verticalJustification="..." backColor="..." verticalText="..." fillTextRegion="..." continuousKaraoke="..." scroll="..." scrollMode="..." >
    ...
</TextSampleDescription>
```

### Semantics

*   `horizontalJustification` : specifies horizontal text justification in the TextBox. Possible values are `"center"`, `"left"` and `"right"`. Default value is `"left"`.
*   `verticalJustification` : specifies vertical text justification in the TextBox. Possible values are`"center"`, `"top"` and `"bottom"`. Default value is `"bottom"`.
*   `backColor` : specifies the color to use for background fill. Expressed as space-separated, hexadecimal R, G, B and A values - for example, semi-transparent red is "ff 00 00 7f". Default value is "00 00 00 00" (no back color).
*   `verticalText` : specifies whether the text shall be drawn vertically or not. Possible values are `"yes"` and `"no"`. Default value is `"no"`.
*   `fillTextRegion` : specifies whether the entire text stream rectangle shall be filled with the backColor, or only the TextBox. Possible values are `"yes"` and `"no"`. Default value is `"no"`.
*   `continuousKaraoke` : specifies whether karaoke is continuous (all characters from beginning of text samples are highlighted) or not. Possible values are `"yes"` and `"no"`. Default value is `"no"`.
*   `scroll` : specifies text scrolling mode. Possible values are `"In"` (text is scrolling in), `"Out"`(text is scrolling out), `"InOut"` (text is scrolling in then out) and `"None"` (text is not being scrolled). Default value is `"None"`.
*   `scrollMode` : specifies text scrolling mode. Possible values are `"Credits"` (scroll from bottom to top), `"Marquee"` (scroll from right to left), `"Down"` (scroll from top to bottom) and`"Right"` (scroll from left to right). Default value is `"Credits"`.
FontTable

The FontTable element specifies all fonts used by samples referring to this stream description. There should be one and only one FontTable element in a TextSampleDescription. If not found, the default "Serif" font will be used with an ID of 1.

### Syntax

```
<FontTable>
    <FontTableEntry fontID="..." fontName="..." />
</FontTable>
```

### Semantics

The FontTable has no attribute, it is a collection of `FontTableEntry` elements. The FontTableEntry element has two attributes:

*   `fontName` : specifies the font name to use. A terminal shall understand the names `Serif`,`Sans-Serif` and `Monospace`.
*   `fontID` : specifies the associated ID.

**NOTE** : There are no default values for this element, omitting values will result in undefined stream behaviour.
TextBox

The TextBox element specifies where the text should be drawn within the stream main display rectangle. There should be one TextBox in a TextSampleDescription element. If the text box is not found or empty, the entire display rectangle will be used for the default text box.

### Syntax

```
<TextBox top="..." left="..." bottom="..." right="..." />
```

### Semantics

*   `top` : specifies vertical offset from stream main display rectangle top edge (type: signed integer). Default value: 0 pixels.
*   `left` : specifies horizontal offset from stream main display rectangle left edge (type: signed integer). Default value: 0 pixels.
*   `bottom` : specifies vertical extend of the text box (type: signed integer). Default value: 0 pixels.
*   `right` : specifies horizontal extend of the text box (type: signed integer). Default value: 0 pixels.

### Style

The Style element specifies the default text style for this sample description. There should be one and only one Style element in a TextSampleDescription. If not found, all default values are used with a fontID of 1.

### Syntax

```
<Style fromChar="..." toChar="..." fontID="..." fontSize="..." color="..." styles="..." />
```

### Semantics

*   `fromChar` : specifies the first character (0-based index) in the string this style applies to (type: unsigned integer). Default value is 0. Note: This field MUST be set to 0 when used in TextSampleDescription.
*   `toChar` : specifies the first character (0-based index) in the string this style stops applying to (type: unsigned integer). Default value is 0. Note: This field MUST be set to 0 when used in TextSampleDescription.
*   `fontID` : specifies the ID of the font to use, as defined in the FontTable element (type: unsigned integer). Default value: 1.
*   `fontSize` : specifies the font size to use (type: unsigned integer). Default value: 18.
*   `color` : specifies the color to use for text. Expressed as space-separated, hexadecimal R, G, B and A values - for example, semi-transparent red is "ff 00 00 7f". Default value is "ff ff ff ff" (opaque white).
*   `styles` : specifies the font styles to use (type: string). If set, it shall consist of a space-separated list of the following styles: `"Bold"` (text is in bold), `"Italic"` (text is in italic) and`"Underlined"` (text is underlined). Default value: no style ("").
TextSample

The TextSample element describes a given text sample and all its associated style. Most of the time, no children elements will be specified.

### Syntax

```
<TextSample sampleTime="..." sampleDescriptionIndex="..." text="..." scrollDelay="..." highlightColor="..." wrap="..." >
...
</TextSample>
```

### Semantics

*   `sampleTime` : specifies the time at which the text sample shall be displayed. Time can be expressed in the usual "hh:mm:ss.ms" in hours, minutes, seconds and milliseconds format, or as a double-precision number in second unit. Default value is "0" second.
*   `sampleDescriptionIndex` : specifies the TextSampleDescription this sample referred to. This is a 1-based index, value 0 is forbidden. Default value is "1", which means you do not need to specify this field most of the time if the main sample description is the first one.
*   `text` : the text data itself. String shall be formatted as a series of lines, each line being enclosed by single-quote characters ('). This text MUST follow XML text encoding conventions. Currently only UTF-8 text is supported.
*   `scrollDelay` : specifies the scrollDelay in seconds (type: double-precision number. This is the delay after a scroll In and/or before scroll Out. Default value "0".
*   `highlightColor` : specifies the highlight color to be used by any enclosed highlighted strings (including karaoke). Expressed as other colors (see `Styles` above). Default value "0 0 0 0".
*   `wrap` : specifies whether text should be wraped or not. Possible values are "Automatic" (text is wrapped by terminal) or "None" (text is not wrapped). Default value is "None".

When text style modification or any special text effects are desired, they are described through children elements of the TextSample element. It is invalid to specify several modifiers of the same type (for instance two hyper-links) on the same character, you must make sure modifiers of a same type do not have overlapping character ranges.
Highlight

The Highlight modifier indicates a given substring shall be highlighted.

### Syntax

```
<Highlight fromChar="..." toChar="..."/>
```

### Semantics

*   `fromChar` : specifies the first character (0-based index) in the string this style applies to (type: unsigned integer). Default value is 0.
*   `toChar` : specifies the first character (0-based index) in the string this style stops applying to (type: unsigned integer). Default value is 0.
Blinking

The Blinking modifier indicates a given substring shall blink. The blinking rate is up to the terminal.

### Syntax

```
<Blinking fromChar="..." toChar="..."/>
```

### Semantics

*   `fromChar` : specifies the first character (0-based index) in the string this style applies to (type: unsigned integer). Default value is 0.
*   `toChar` : specifies the first character (0-based index) in the string this style stops applying to (type: unsigned integer). Default value is 0.
HyperLink

The Hyperlink modifier indicates that a given substring shall be treated as a hyper link.

### Syntax

```
<Hyperlink fromChar="..." toChar="..." URL="..." URLToolTip="..."/>
```

### Semantics

*   `fromChar` : specifies the first character (0-based index) in the string this style applies to (type: unsigned integer). Default value is 0.
*   `toChar` : specifies the first character (0-based index) in the string this style stops applying to (type: unsigned integer). Default value is 0.
*   `URL` : URL this HyperLink is linking to. UTF-8 only format supported.
*   `URLToolTip` : "tooltip" presented to the user if supported by decoder. UTF-8 only format supported.
Karaoke

The Karaoke element indicates dynamic highlighting, or karaoke, applies to this sample. At most one Karaoke element shall be set per TextSample. The Karaoke is defined as a sequence of highlighting times, all specified as children element of the Karaoke through the `KaraokeRange`element.

### Syntax

```
<Karaoke startTime="...">
    <KaraokeRange fromChar="..." toChar="..." endTime="..."/>
< /Karaoke >
```

### Semantics

*   `startTime` : specifies the highlighting start time offset in seconds from the beginning of the sample (type: double-precision number, default value "0").
*   `fromChar` : specifies the first character (0-based index) in the string this style applies to (type: unsigned integer). Default value is 0.
*   `toChar` : specifies the first character (0-based index) in the string this style stops applying to (type: unsigned integer). Default value is 0.
*   `endTime` : specifies the highlighting end time offset in seconds from the beginning of the sample (type: double-precision number, default value "0"). The start time of a karaoke segment is the end time of the previous one, or the start time of the `Karaoke` element.
Encoding of text streams

Using text streams in a 3GPP file or outisde the MPEG-4 Systems scene description is quite straightforward. All you need to do is add the text stream to your (existing) file with MP4Box. 

For example, adding a text track and an AVI file into a new 3GP file can be done in a single call:  
 `MP4Box -3gp -add movie.avi -add text.ttxt dest.3gp`.

_Note:_ By default, when importing ttxt with MP4Box, the duration of the last text sample is the same as the duration of the previous sample. To specify the duration of the last sample, add an extra text sample at the end with no text content.

Note that you can use either SRT/SUB subtitles or TTXT files with the `-add` option. Other subtitles formats are currently not supported.

Using text streams within MPEG-4 scene description is much more tricky (is it?). You will add an ObjectDescriptor to your scene as you add an audio/video object, specifying the file name with the usual MuxInfo. **Note**: when using SRT/SUB, make sure you don't have `TextNode` specified in the MuxInfo, as this ALWAYS triggers subtitles to BIFS conversion and not 3GPP timed text.

The second step is controlling the new text object in the same way you control an audio or visual object. This is done through the AnimationStream node. Instead of controlling a BIFS stream, you can start/stop/play the text stream with this node.
GPAC Implementation

*   During playback, GPAC does not support dynamic highlighting (Karaoke) nor soft text wrapping (wrapping is only done at newline characters).
*   GPAC should support vertical text drawing and alignment, but this has not be really tested yet.
*   GPAC should support UTF-16 text in decoding and hinting, but DOES NOT SUPPORT UTF-16 text at encoding time yet.
*   GPAC should support text streams placed below the main video or on its right, but cannot currently handle text streams placed above or on the left of the main video.
Sample TTXT Files

The following files are taken from a 3GPP test suite and are pure translations to TTXT with MP4Box. They should give you a good overview of the format and help you author your own test tracks.

Download GPAC TTXT sample streams [.zip](http://gpac.sourceforge.net/downloads/ttxt_sample_streams.zip) [.tgz](http://gpac.sourceforge.net/downloads/ttxt_sample_streams.tar.gz)
Javascript Filters

The JS filter API makes it easy to **extend gpac** using the internal QuickJS runtime, giving access to the Filter API for frame and packet processing, but also APIs for adaptative streaming, compositing, storage, ...

Some of the gpac built-in filters - eg. [avgen](Filters/avgen), [avmix](Filters/avmix) - are actualy implemented as custom javascript filters. Their source code can be found under the[`share/scripts/jsf`](https://github.com/gpac/gpac/tree/master/share/scripts/jsf) directory.

[JSF documentation](/Howtos/jsf/jsfilter){ .md-button }
What is GPAC ?

The **GPAC Filter API** is at the core of the [MP4Box and GPAC](Howtos/gpac-mp4box) applications.

The `gpac` application allows building media pipelines by conveniently [combining and configuring Filters](Filters/filters_general) from the command line. 

Filters are configurable processing units consuming and producing data packets. 

GPAC provides a wide range of Filters supporting advanced media protocols, formats, codecs, for input, output and processing tasks. 

It defines the infrastructure and helper classes to develop applications with advanced media capabilities.


### Concepts

**Filter Session**

- [API documentation](https://doxygen.gpac.io/group__fs__grp.html#details)
- [tutorial](https://git.gpac-licensing.com/slarbi/API_FIlters_tutos/src/branch/master/T0_Filters_session/simple%20gpac%20session.md)

**Filter**
- [API documentation](https://doxygen.gpac.io/group__fs__filter.html#details)

**Filter Properties**
- [API documentation](https://doxygen.gpac.io/group__fs__props.html#details)
- [Built in Properties](https://wiki.gpac.io/Filters/filters_properties/?h=properties)

**Filter Events**
- [API documentation](https://doxygen.gpac.io/group__fs__evt.html#details)

**Filter PIDs & Capabilities**
- [API documentation](https://doxygen.gpac.io/group__fs__pid.html)

**Filter Packet**
- [API documentation](https://doxygen.gpac.io/group__fs__pck.html#details)

**Custom Filter**
- [doxygen](https://doxygen.gpac.io/group__filters____cust__grp.html#details)
- [writing a custom Filter]()
Creating a custom GPAC filter

Custom filters are filters created by the app with no associated registry. Therefore there is no internal representation for the custom filter (No filter registry). So capabilities and different behaviors of the custom filter must be specified by the app with the helper callback functions (listed below).

The app is responsible for assigning capabilities to the filter, and setting callback functions. Each callback is optional, but a custom filter should at least have a process callback, and a configure_pid callback if not a source filter.

  

**Custom filter limitations:**

*  Custom filters do not have any arguments exposed.    
*  The filter cannot be used as source of filters loading a source filter graph dynamically, such as the dashin filter.
*  The filter cannot be used as destination of filters loading a destination filter graph dynamically, such as the dasher filter.
*  The filter cannot be cloned.
Callback functions

### [gf_fs_new_filter()](https://doxygen.gpac.io/group__filters____cust__grp.html#ga45d6cd5535614e43d6ebb77d8cb1a4bc)

This function is responsible for loading a custom filter into the specified filter session. It allows the application to create filters without associated registries, providing the flexibility to assign capabilities and set callback functions. The function parameters include the filter session, the name of the filter (optional), flags for filter registry, and an optional error code parameter.

### [gf_filter_push_caps()](https://doxygen.gpac.io/group__filters____cust__grp.html#ga878516ab46d5bfc96692a91f3ba6b124)

This function facilitates the addition of new capabilities to a custom filter. Parameters such as capability code, value, name, flags, and priority are specified to define the capabilities. This function returns an error if any issues occur during the process.

An alternative way is to push the caps to a specific PID within the filter callbacks function, typically inside the process callback , or the configure callback for non source filters.

### [gf_filter_set_process_ckb()](https://doxygen.gpac.io/group__filters____cust__grp.html#gad1f20440c6de9b009a4fde85d429ad39)

This function is employed to set the process callback function for a custom filter. Typically callback defines the processing logic of the filter. The function takes the target filter and the process callback as parameters.

This one callback is mandatory in order to use the custom filter.

### [gf_filter_set_configure_ckb()](https://doxygen.gpac.io/group__filters____cust__grp.html#gab21f4169d1de1a0876f8b6856301ff28)

For custom filters that are not source filters, the **gf_filter_set_configure_ckb()** function is utilized to set the **PID** configuration callback. The assigned  function enables and manages the configuration of the PID(s) for the filter.

### [gf_filter_set_process_event_ckb()](https://doxygen.gpac.io/group__filters____cust__grp.html#gadefa24ff7508d8838334d381f7c9004c)

Set the process event callback for a custom filter. This callback handles events related to the filter. for example specifying what happens if a Play or a Stop event is received.  

### [gf_filter_set_reconfigure_output_ckb()](https://doxygen.gpac.io/group__filters____cust__grp.html#gae22ecc90654524434483f204713989fb)

The reconfigure_output callback may be needed to reconfigure the output PID(s) of a filter during the execution of a filters session. 

### [gf_filter_set_probe_data_cbk()](%28https://doxygen.gpac.io/group__filters____cust__grp.html#ga3b067ec9d2067d683ea5d3fdb4d32833%29)

Set the data prober function for a custom filter.
Example 2 - Getting data from a custom filter to an application

Alternatively to using custom filters we can create a filter class and add it to the session using [gf_fs_add_filter_register()](https://doxygen.gpac.io/group__fs__grp.html#ga4ae302f59379de2544c17b374eae3516)  
The following is the definition of the mem_out filter register with simply two functions process() and configure_pid(). (This very minimalistic more options are possible )  

```C
  GF_FilterRegister  memOutRegister  = {
       .name  =  "mem_out",
       .private_size  =  sizeof(MemOutCtx),
       .process  =  mem_out_process,
       .configure_pid  =  mem_out_configure_pid,
      };
```
 
The mem_out_process logic is also straight forward.

```C
MemOutCtx *ctx = (MemOutCtx *)gf_filter_get_udta(filter);
static GF_Err mem_out_process(GF_Filter *filter) {
  if (!ctx)
    return GF_BAD_PARAM;
  ctx->pushData = &outputPushData;
  ctx->pushDsi = &outputPushMetadata;

  GF_FilterPid *ipid = gf_filter_get_ipid(filter, 0);

  const GF_PropertyValue *prop =
      gf_filter_pid_get_property(ipid, GF_PROP_PID_DECODER_CONFIG);

  if (prop && prop->value.data.ptr && prop->value.data.size) {
    ctx->pushDsi(ctx->parent, prop->value.data.ptr, prop->value.data.size);
  }

  GF_FilterPacket *pck = gf_filter_pid_get_packet(ipid);
  if (pck) {
    u32 data_size = 0;
    u64 dts = gf_filter_pck_get_dts(pck);
    u64 pts = gf_filter_pck_get_cts(pck);
    const u8 *data = gf_filter_pck_get_data(pck, &data_size);

    ctx->pushData(ctx->parent, data, data_size, dts, pts);
    gf_filter_pid_drop_packet(ipid);
  }

  return GF_OK;
}
```

we note here:

* The use of [gf_filter_get_ipid()](https://doxygen.gpac.io/group__fs__filter.html#ga03a9a73e8d044d7737b4dd9d74e23a79) to retrieve the input pid. 
* The use of [gf_filter_pid_get_property()](https://doxygen.gpac.io/group__fs__pid.html#ga1e28b43fba75976755ef3004edfe2a2a) to get the properties of the input pid.
* The use of [gf_filter_pid_get_packet()](https://doxygen.gpac.io/group__fs__pid.html#gaf373afc8a944b4a5b20952e8a3121d2f) to retrieve the packet from the input pid.
* The use of [gf_filter_pck_get_data()](https://doxygen.gpac.io/group__fs__pck.html#ga6727b4c6fa4a4366ccc23244c6191570) to retrieve the data from the packet of the input pid.


### Registry and loading of the filter to the session 

```C
// register and load destination filter
GF_Filter  *dst_filter  =  NULL;
const  GF_FilterRegister  *mem_out_register(GF_FilterSession  *);
gf_fs_add_filter_register(session, mem_out_register(session));
dst_filter  =  gf_fs_load_filter(session, "mem_out", &gf_err);
```

The memory output filter is now ready to be used inside the filters session.
Building the filter session graph using the reframer filter

Here is an example of a filters session using both filters examples provided before that we build using two different approaches ( custom filter & class filter). sandwiching  the reframer as an example of a filters chain.     

```C
int  main() {
    
  GF_Err gf_err = GF_OK;
  
  GF_FilterSession *session = gf_fs_new_defaults(0u);
  if (session == NULL) {
    fprintf(stderr, "Failed to create GPAC session\n");
    goto exit;
  }
        
  // adding custom input filter
  GF_Filter *src_filter = gf_fs_new_filter(session, "mem_in", 0, &gf_err);
  MemInCtx *ctxIn = gf_malloc(sizeof(MemInCtx));
  ctxIn->dts = 0;
  ctxIn->pts = 0;
  ctxIn->max_frames = 3;
  gf_err = gf_filter_set_rt_udta(src_filter, (void *)ctxIn);
  gf_err = gf_filter_set_process_ckb(src_filter, mem_in_process_ckb);
  GF_Filter *reframer_filter = gf_fs_load_filter(session, "reframer", &gf_err);
  if (gf_err != GF_OK) {
    fprintf(stderr, "Failed to load filter reframer: %s \n",
            gf_error_to_string(gf_err));
    goto exit;
  }
  gf_filter_set_source(reframer_filter, src_filter, NULL);
    
  // register and load destination filter
  GF_Filter *dst_filter = NULL;
  const GF_FilterRegister *mem_out_register(GF_FilterSession *);
  gf_fs_add_filter_register(session, mem_out_register(session));
  dst_filter = gf_fs_load_filter(session, "mem_out", &gf_err);
  if (gf_err != GF_OK) {
    fprintf(stderr, "Failed to load filter mem_out: %s \n",
            gf_error_to_string(gf_err));
    goto exit;
  }

  // finalize graph connections
  gf_filter_set_source(dst_filter, reframer_filter, NULL);

  // run
  gf_filter_post_process_task(src_filter);    
  gf_fs_run(session);

  // error handling
  if (gf_err >= GF_OK) {
    gf_err = gf_fs_get_last_connect_error(session);
    if (gf_err >= GF_OK)
      gf_err = gf_fs_get_last_process_error(session);
  }

  // print connections
  gf_fs_print_debug_info(session, 0);
  gf_fs_print_connections(session);
  gf_fs_print_stats(session);
  exit:
       gf_fs_del(session);
       session = NULL

  return gf_err == GF_OK ? EXIT_SUCCESS : EXIT_FAILURE;
}
```

Here we note:

* The use of [gf_filter_set_rt_udta()](https://doxygen.gpac.io/group__fs__filter.html#gac2f040600796f000ac4189fda1c76bc0) to set our runtime data of the mem_in filter. 
* We post the mem_in process task to the session using the [gf_filter_post_process_task(src_filter)](https://doxygen.gpac.io/group__fs__filter.html#ga5806cdb70097f7d5d181884928870842) function.
Execution report

If we define a static h264 frame to use it as a memory input 
 
```C
static const uint8_t h264_grey_frame_dsi[] = {
      0x01, 0x4d, 0x40, 0x0a, 0xff, 0xe1, 0x00, 0x15, 0x67, 0x4d,
      0x40, 0x0a, 0xe8, 0x8f, 0x42, 0x00, 0x00, 0x03, 0x00, 0x02,
      0x00, 0x00, 0x03, 0x00, 0x64, 0x1e, 0x24, 0x4a, 0x24, 0x01,
      0x00, 0x05, 0x68, 0xeb, 0xc3, 0xcb, 0x20 };
```
and we run our filters session that will result in the following filters graph being executed. 

![custom filter execution](images/T1_img0.png)
About the gpac filters API

The API enables you to use GPAC capabilities in your own personal code. By calling any filter in gpac’s set of built-in filters, or even create your own personalized custom filters to interact within the media pipelines. This can be useful to interface with other pieces of software or create faster ways of executing some media processing workflows.

To learn more about general concepts refer to the general wiki page of [gpac](/Filters/filters_general).