pertalde 0.1.4
Some command-line utilities on paraver traces.
Requires Python: >=3.8


# pertalde

_pertalde_ is a set of tools to manipulate Paraver traces.  Paraver is a
parallel performance analysis tool by the Performance Tools team at BSC,
and is a parallel trace visualization system allowing for large scale
trace execution analysis. Paraver can be obtained at
[https://tools.bsc.es/downloads](https://tools.bsc.es/downloads).

## How to install

_pertalde_ is distributed as a PyPI package and thus can be installed
with `pip`. 
```
python -m pip install pertalde
```

### Suggestions for HPC clusters

#### Create a python environment

```
python -m venv ~/.pertalde_env
source ~/.pertalde_env/bin/activate

python -m pip install pertalde
```

#### When you don't have internet access

If you want to install _pertalde_ in a remote machine where you don't
have internet access, you can first download the package.
```
python -m pip download pertalde
```

This will download the package (a .whl file) which you can the install
using
```
python -m pip install <package_name>.whl
```

## Utilities

### pertalde unir (i.e. parajuntar)

When you a trace with sparse semantic values, and/or with inconsistent
value per semantic across traces you can use `pertalde unir`. This will
use the values from the `pcf` files to unify the values based on their
semantic, and translate the `prv` files accordingly. You can also group
different event types together to unify their semantics.

#### How to use it.

**If you had a trace with different event** that share semantics but differ
in values. For a more specific example, lets say we have a `trace.prv`
with events `1000`, `1001`, `2000`, `2001`. Where events `1000` and
`1001` share meaning between them, and `2000` and `2001` between them.

```bash
ptd unir --types=1000,1001 --types=2000,2001 trace.prv
```

This would result in a `trace.unir.prv` file where now all values that
have the same name for `1000` and `1001` have also the same value. And
all the same for values `2000` and `2001`.

**If you have multiple traces with the same event** but this event has
inconsistent values for the same semantic across traces, you can also
use `pertalde unir`. By running the same command on different traces,
the resulting event values will be common across traces also.

```bash
ptd unir --types=1000,1001 trace1.prv trace2.prv
```

**A practical example that combines these two modes** is when you have
multiple traces with OpenMP, and you want to compare the values among
these traces:
```bash
ptd unir                        \
    --sufix unified_openmp      \
    --sort-values-by-name       \ # Keep similar names closer in value
    -t 60000018                 \ # Executed parallel function
    -t 60000118                 \ # Executed parallel function line and file
    -t 60000023                 \ # Executed OpenMP task function
    -t 60000123                 \ # Executed OpenMP task function line and file
    -t 60000024                 \ # Instantiated OpenMP task function
    -t 60000124                 \ # Instantiated OpenMP task function line and file
    -c                          \ # Compress the resulting trace
    trace1.prv trace2.prv       ; # Or more traces if necessary
```

**In the case you get a new trace** and you want it to have common
values with a trace you have already processed earlier you can make use
of the option `--base-pcf`.

```bash
# This produces trace1.unir.prv trace1.unir.pcf and trace1.unir.row
ptd unir --types=1000,1001 trace1.prv

# Now you get a new trace: trace2.prv
ptd unir --types=1000,1001 --base-pcf=trace1.unir.pcf trace2.prv
```

This will make event values from types `1000` and `1001`, form
`trace2.prv`, consistent with those in `trace1.prv`.

**This tool has preset gorups** for specific tools. As this tool was
initially implemented to solve a common problem with `neSmiK` and
`nsys2prv` we have presets for this tools. For example, for nesmik you
could use:

```bash
ptd unir --profile=nesmik trace.prv
```

### pertalde separar

When you have a big trace that you want to analyse its common practice
to filter it. This tool has a preset of filters that you can use without
having to rely on a configuration generated from Paraver. Furthermore,
it allows to run multiple filters on a single execution. Which results
in generating the filters faster. Right now we have event filters for neSmiK,
MPI, OpenMP, Counters, and Flushing. And we have a useful filter with a
minimum duration threshold.

#### How to use it.

```bash
ptd separar --compress --useful 100 --profile=mpi,counters trace.prv
```

This will generate a directory names `test.partials`, which will contain
3 different traces with names
`trace.{useful,mpi,counters}.{prv.gz,pcf,row}`. Notice that this tool
can also leverage processing the trace to compress it with the flag
`--compress` which is available for all subcommands that produce output
traces.

### pertalde comptar

This tool will count the occurences and estimate the uncompressed disk
size of different records.

```bash
ptd comptar --all trace.prv
```


### pertalde sincronitzar

If you get a trace that is not synchronized, you can use this tool to
synchronize it based on an event. This is helpful when the merging
process is very long, or you no longer have the unmerged files. This
tool can only syncronize the MPI tasks.

```bash
# By default it will use the MPI init event.
ptd syncronitzar trace.prv
```

### pertalde comprovar

This tool basically will test a trace to check if its well formatted. If
you are having problems with a trace, or are generating trace files,
this tool can tell you where the trace has format issues.

```bash
ptd comprovar trace.prv
```