dhdat 0.6.2
DHDAT is a python package with basic tools to produce interaction matrices and calculate several dominance hierarchy related metrics
License: MIT

# Dominance Hierarchy Development Analysis Tools (DHDAT)
<table style="text-align:left">
<tr><th>Author</th><th>Erik van Haeringen</th></tr>
<tr><th>Email</th><th>e.s.van.haeringen@vu.nl</th></tr>
<tr><th>Date</th><th>July 4th 2019</th></tr>
<tr><th>Description</th><th>DHDAT is a python package with basic tools to produce interaction matrices and calculate several dominance hierarchy related metrics</th></tr>
</table>

## Licence
Copyright (C) 2019, van Haeringen. This package is published under an MIT licence and you are welcome to use or improve upon it. For any publication, whether research or software that uses or includes (partial) copies of (modules of) this package, please cite this work.

## Prerequisites
* [Python 3](https://www.python.org/)
* [Pandas](https://pandas.pydata.org/)

## Install
To install the package for your default Python installation use your terminal to execute the command below.

```
pip install dhdat
```

For other Python installations replace `pip` with the path to its respective pip executable

## Contents
1. [Matrix](#1-matrix)
2. [CombinationMaker](#2-combinationmaker)
3. [Triads](#3-triads)
4. [Ttri](#4-ttri)
5. [NetworkState](#5-networkstate)
6. [ADI](#6-adi)
7. [Xi](#7-xi)
8. [Bursts](#8-bursts)
9. [PairFlips](#9-pairflips)
10. [TauKr](#10-taukr)

## Modules
### 1. Matrix
#### Description
Builds an interaction matrix based on a list of actors and fills this with rows from a pandas dataset.
These Matrix objects are used by the other modules, for example to calculate the dominance index Xi.
Interaction winners are the rows and the losers are the columns.

#### How to use
Create a new matrix object by initiating matrix with a list of the actors identifiers.
This matrix object contains three Pandas DataFrames, one interaction matrix of initiations of aggression (**d_mI**), one cumulative matrix of the outcomes of fights (**d_mC**), a another that is a non-cumulative matrix of the outcome of fights (**d_mNC**).

```python
from dhdat import Matrix

actorIDs = [1,2,3,4]
matrix = Matrix(actorIDs)       #new interaction matrix of size len(actorIDs)

print(matrix.d_mC)              #shows cumulative matrix
print(matrix.d_mNC)             #shows non-cumulative matrix
```

The matrix can be updated with a row from a Pandas DataFrame. This DataFrame should be structured with one row per interaction, containing at least the columns **'actor.id'**, **'actor.behavior'**, **'receiver.id'** and **'receiver.behavior'**.
The **'actor.id'** and **'receiver.id'** columns should contain a actorID provided to the matrix object on initialization to indicate who who initiated the dominance interaction (actor.id) and who was the receiver (receiver.id).
The **'actor.behavior'** and **'receiver.behavior'** columns should contian either the string "Fight" or "Flee", indicating the outcome of the fight (Fight = win and Flee = loss).
Thus actor can be the one who initialized the interaction but then lose the interaction and flee. The Pandas DataFrame should look similar to the example below.

```python
import pandas as pd
                                    #load data from csv file
df = pd.read_csv("exampleDataSet.csv", delimiter="\t")

print(df)
```

Example data set:
```
   run  time  actor.id  receiver.id actor.behavior receiver.behavior
0    0    23         4            2          Fight              Flee
1    0   112         2            3           Flee             Fight
2    0   278         1            3          Fight              Flee
3    0   315         4            2          Fight              Flee
4    0   801         4            2           Flee             Fight
5    0   932         1            3          Fight              Flee
6    0   966         3            1           Flee             Fight
```

Interactions can be added to the matrix by calling either `update()` which updates all matrices, or by calling the update functions for the individual matrices (`updateInitiated()`, `updateCumulative()`, `updateNonCumulative()`). 
The update functions require a row (interaction) of the Pandas DataFrame as described above. The example below shows a simple for-based loop adding all interactions in the DataFrame *df* using the update functions of the individual matrices. 
Note that the update statement from the example below can also be written out as 3 seperate matrix specific update statements with the same result (for example `matrix.updateCumulative(df.loc[interaction,:])`).

```python
from dhdat import Matrix
import pandas as pd

df = pd.read_csv("exampleDataSet.csv", delimiter="\t", index_col=0)

actorIDs = [1, 2, 3, 4]             #list of actor identifiers
matrix = Matrix(actorIDs)           #setup matrix object with actor identifiers

for interaction in df.index:
                                    #add new interaction to the matrices
    matrix.update(df.loc[interaction,:])

print("Initiations matrix\n", matrix.d_mI)
print("\nCumulative interaction matrix\n", matrix.d_mC)
print("\nNon-cumulative interaction matrix\n", matrix.d_mNC)
```

Output:
```
Initiations matrix
   1  2  3  4
1  0  0  2  0
2  0  0  1  0
3  1  0  0  0
4  0  3  1  0

Cumulative interaction matrix
   1  2  3  4
1  0  0  3  0
2  0  0  0  1
3  0  1  0  1
4  0  2  0  0

Non-cumulative interaction matrix
   1  2  3  4
1  0  0  1  0
2  0  0  0  1
3  0  1  0  1
4  0  0  0  0
```

The dataframes containing the matrices can directly be accessed as shown in the example above.
There are also the functions `exportInitiated()`, `exportCumulative()` and `exportNonCumulative()` that store the respective matrix as a tab-separated csv file.
The functions arguments are *filename* and *run_number*.
This results in the following filename structure: `[filename][run_number]_matrix[type].csv` where matrix type is I for initiated, C for cumulative and NC for non-cumulative.

```python
                                    #produces 'test_5_matrixNC.csv'
matrix.exportNonCumulative("test_", 5)

                                    #produces 'test2_14_matrixC.csv' in subdirectory 'figures'
matrix.exportCumulative("figures/test2_", 14)
```

-----------------------
### 2. CombinationMaker
#### Description
Class that uses a recursive function to generate all possible triangles based on a set of actors, and stores these combinations in a pandas dataframe.
The recursive algorithm was inspired on a example (in C) by [Bateesh](https://www.geeksforgeeks.org/print-all-possible-combinations-of-r-elements-in-a-given-array-of-size-n/).

#### How to use

A combination object is initialized with a list of the elements (actors) that will be combined, and the number of elements per combination. Below is an example for all combinations of three individuals that can be made with four individuals.

```python
from dhdat import CombinationMaker

actorIDs = [1, 2, 3, 4]
                                    #calculate all triad combinations of 4 actors
combinations = CombinationMaker(actorIDs, 3)
```

The combinations are stored in a Pandas DataFrame **d_result**.
This member can be accessed directly, or alternatively the function `getResults()` returns this member.

```python
combinations.getResults()
```
Output:
```
   0  1  2
0  1  2  3
1  1  2  4
2  1  3  4
3  2  3  4
```

-----------------------
### 3. Triads
#### Description
Counts triad motifs in a dominance network read from an interaction matrix.
See Wasserman & Faust (1994), or Shizuka & McDonald (2012) for details on triad coding.
This class can count either triad motifs with only directed relationships, in which case mutual (equal) relationships are ignored.
Or it can also count triad motifs that contain one or more mutual relationships.

#### How to use
Triads is initialized with the option for mutual triad motif count (False or True) and a [CombinationMaker](#2-combinationmaker) object containing all possible combinations of actors for triads.

```python
from dhdat import Matrix
from dhdat import CombinationMaker
from dhdat import Triads
import pandas as pd

df = pd.read_csv("exampleDataSet.csv", delimiter="\t")

actorIDs = [1, 2, 3, 4]
matrix = Matrix(actorIDs)
                                    #calculate all triad combinations of 4 actors
combinations = CombinationMaker(actorIDs, 3)
                                    #setup triads object without mutual relationships
triads = Triads(False, combinations.getResults())
```

A Triads object counts triads with the function `count()`, which requires either a cumulative or non-cumulative [matrix](#1-matrix) and the index of the current interaction.
The resulting motif count is stored in a Pandas DataFrame **d_triadCount** at the index supplied to the `count()` function.

```python
for interaction in df.index:
                                    #adds interaction to non-cumulative matrix
    matrix.updateNonCumulative(df.loc[interaction,:]) 

    triads.count(matrix.d_mNC, interaction)

                                    #shows triad count of interaction  
print(triads.d_triadCount)
```

Output:

```
   TRI_003  TRI_012  TRI_021D  TRI_021U  TRI_021C  TRI_030T  TRI_030C
0      2.0      2.0       0.0       0.0       0.0       0.0       0.0
1      1.0      2.0       0.0       1.0       0.0       0.0       0.0
2      0.0      2.0       0.0       1.0       1.0       0.0       0.0
3      0.0      2.0       0.0       1.0       1.0       0.0       0.0
4      0.0      2.0       0.0       0.0       2.0       0.0       0.0
5      0.0      2.0       0.0       0.0       2.0       0.0       0.0
6      0.0      2.0       0.0       0.0       2.0       0.0       0.0
7      0.0      1.0       0.0       0.0       2.0       1.0       0.0
```

-----------------------
### 4. Ttri
#### Description
Calculates T<sub>tri</sub> as described in Shizuka and McDonald (2012), based on the triad motif count of a dominance network.
If option 'mutual' is chosen, mutual triads are included in the calculation of T<sub>tri</sub>.
Otherwise T<sub>tri</sub> is calculated only over triads that have directed edges.

#### How to use
To calculate T<sub>tri</sub> first a Ttri object must be initialized with the option for triad count of mutual relations (True or False).
This option should correspond to the option chosen to [count](#3-triads) the triad motifs. Because triad count can be either over a cumulative matrix or a non-cumulative matrix, the T<sub>tri</sub> value either measures linearity over the last interaction in each pair (non-cumulative), or includes all previous interactions in each pair (cumulative) to determine the direction of a pair relation. In the paper cited above by Shizuka and McDonald, T<sub>tri</sub> is calculated over the final cumulative interaction matrix including all recorded interactions.

```python
from dhdat import Matrix
from dhdat import CombinationMaker
from dhdat import Triads
from dhdat import Ttri
import pandas as pd

df = pd.read_csv("exampleDataSet.csv", delimiter="\t")

actorIDs = [1, 2, 3, 4]
matrix = Matrix(actorIDs)
                                    #calculate all triad combinations of 4 actors
combinations = CombinationMaker(actorIDs, 3)
                                    #setup triads object without mutual relationships
triads = Triads(False, combinations.getResults())
                                    #setup ttri object without mutual relationships
ttri = Ttri(False)
```

Then the Ttri object can be fed with the triad count from a [Triads](#3-triads) object, and the index of the current interaction.
Ttri is stored in a Pandas DataFrame d_ttri with one column *'T_tri'*, which can be accessed directly.
Ttri can by definition only be determined when there is at least one complete triad (containing 3 links).
Thus the [example data set](#1-matrix) used in this manual results in no value for *'T_tri'* as there are no complete triads, as was shown in the demonstration of the previous module [Triads](#3-triads).

```python
for interaction in df.index:
                                    #adds interaction to non-cumulative matrix
    matrix.updateNonCumulative(df.loc[interaction,:])     
    triads.count(matrix.d_mNC, interaction)

    ttri.calculate(triads.d_triadCount, interaction)        

                                    #shows Ttri of interactions
print(ttri.d_Ttri)
```

Output:
```
  T_tri
0  None
1  None
2  None
3  None
4  None
5  None
6  None
7     1
```
T<sub>tri</sub> in a directed (non-mutual) network is a scaled ratio of transitive triad motifs divided by the transitive + cyclic triad motifs. In a mutual network it uses the ratio of transitive weights divided by the total number of complete triad motifs, as some mutual triad motifs are defined as partially transitive. This measure ignores motifs with missing links (also called relations or edges). In example given here of T<sub>tri</sub> calculated for a directed network, there are no complete motifs (either transitive, cyclic), which results in a empty field. See Shizuka and McDonald (2012) for further details.

-----------------------
### 5. NetworkState
#### Description
Determines the state of a network of 4 individuals based on triad motif count.
See Lindquist and Chase (2009) for an explanation of triad motifs network
states and nomenclature. Currently only networks of 4 individuals are supported.
Increasing the group size results in an exponential growth of possible network states, and thus quickly becomes unfeasible.

#### How to use
A NetworkState object is created with a list of the actors.
To determine the network state of an interaction matrix, member function `determine()` requires the triad state count **d_triadState** of a [Triads](#3-triads) object, and the number of the current interaction as an index to store the result.
Because for some states triads motif count alone is not enough to determine the network state, additionally the non-cumulative matrix **d_mNC** of a [Matrix](#1-matrix) object is a required argument.
States are stored in data member **d_state** as a Pandas DataFrame and can be accessed directly.

```python
from dhdat import Matrix
from dhdat import CombinationMaker
from dhdat import Triads
from dhdat import NetworkState
import pandas as pd

df = pd.read_csv("exampleDataSet.csv", delimiter="\t")

actorIDs = [1, 2, 3, 4]
matrix = Matrix(actorIDs)
                                    #calculate all triad combinations of 4 actors
combinations = CombinationMaker(actorIDs, 3)
                                    #setup Triads object without mutual relationships
triads = Triads(False, combinations.getResults())
state = NetworkState(actorIDs)      #setup NetworkState object

for interaction in df.index:
                                    #adds interaction to non-cumulative matrix
    matrix.updateNonCumulative(df.loc[interaction,:]) 
    triads.count(matrix.d_mNC, interaction)

    state.determine(triads.d_triadState, interaction, matrix.d_mNC)

                                    #shows network state of interaction  
print(state.d_state)
```

Output:
```
  State
0     1
1     4
2    13
3    13
4    15
5    15
6    15
7    20
```


-----------------------
### 6. ADI
#### Description
Calculates the average dominance index (ADI) from a cumulative interaction matrix as described in Hemelrijk et al. (2005).

#### How to use
A ADI object is created with a list of the actors.
Then to calculate the ADI for an interaction call member function `calculate()` with a cumulative interaction matrix **d_mC** from a [Matrix](#1-matrix) object, and the number of the current interaction that is used as an index to store the calculated ADI value.
ADI values are stored in Pandas DataFrame **d_ADI** with a column *ADI_[actorID]* for each actor, and can be accessed directly.

```python
from dhdat import Matrix
from dhdat import ADI
import pandas as pd

df = pd.read_csv("exampleDataSet.csv", delimiter="\t")

actorIDs = [1, 2, 3, 4]             #list of actor identifiers
matrix = Matrix(actorIDs)           
adi = ADI(actorIDs)

for interaction in df.index:
                                    #adds interaction to cumulative matrix
    matrix.updateCumulative(df.loc[interaction,:]) 

    adi.calculate(matrix.d_mC, interaction)

print(adi.d_ADI)                    #shows ADI values for all interactions
```

Output:
```
   ADI_1     ADI_2     ADI_3     ADI_4
0    NaN  0.000000       NaN  1.000000
1    NaN  0.000000  1.000000  1.000000
2    1.0  0.000000  0.500000  1.000000
3    1.0  0.000000  0.500000  1.000000
4    1.0  0.166667  0.500000  0.666667
5    1.0  0.166667  0.500000  0.666667
6    1.0  0.166667  0.500000  0.666667
7    1.0  0.166667  0.666667  0.333333
```

-----------------------
### 7. Xi
#### Description
Calculates the dominance index Xi, which is the proportion of aggressive
interaction won, for a cumulative interaction matrix as described in Lindquist
and Chase (2009)

#### How to use
A Xi object is created with a list of the actors.
Then to calculate the Xi for an interaction call member function `calculate()` with a cumulative interaction matrix **d_mC** from a [Matrix](#1-matrix) object,
and the number of the current interaction that is used as an index to store the calculated Xi value.
Xi values are stored in Pandas DataFrame **d_Xi** with a column *Xi_[actorID]* for each actor, and can be accessed directly.

```python
from dhdat import Matrix
from dhdat import Xi
import pandas as pd

df = pd.read_csv("exampleDataSet.csv", delimiter="\t")

actorIDs = [1, 2, 3, 4]             #list of actor identifiers
matrix = Matrix(actorIDs)           
xi = Xi(actorIDs)

for interaction in df.index:
                                    #adds interaction to cumulative matrix
    matrix.updateCumulative(df.loc[interaction,:]) 

    xi.calculate(matrix.d_mC, interaction)

print(xi.d_Xi)                      #shows Xi values for all interactions
```

Output:
```
  Xi_1  Xi_2      Xi_3      Xi_4
0  NaN     0       NaN         1
1  NaN     0         1         1
2    1     0       0.5         1
3    1     0       0.5         1
4    1  0.25       0.5  0.666667
5    1  0.25  0.333333  0.666667
6    1  0.25      0.25  0.666667
```

-----------------------
### 8. Bursts
#### Description
Detects whether bursts occur, a pattern of repeated consecutive attacks in the same direction within a dyad, as described by Lindquist and Chase (2009).
It does this by comparing the direction of the current interaction with the previous interaction. Note that this definition does not include a time component.

#### How to use
A new Bursts object can by defined without any arguments.
To determine a interaction is part of a burst event, the member function `detect()` requires a row of a Pandas DataFrame containing the current interaction, and the row containing the previous interaction, as well as the number of the current interaction to use as a index to store the resulting burst value.
The result (True or False) is stored in data member **d_bursts** and can be accesed directly as shown below.
This example shows how with a simple for loop burst events can be detected for a set of interactions, by storing the previous interaction index.
The first interaction can by definition never be a burst, and thus is skipped.

```python
from dhdat import Bursts
import pandas as pd

df = pd.read_csv("exampleDataSet.csv", delimiter="\t")

bursts = Bursts()                   #define a new Bursts object
prevInteraction = None              #holds index of previous interaction

for interaction in df.index:
    if prevInteraction != None:     #first interaction cannot be a burst
      bursts.detect(df.loc[interaction,:], df.loc[prevInteraction,:], interaction)

    prevInteraction = interaction

print(bursts.d_bursts)              #shows burst values for all interactions
```

Output:
```
   burst
1  False
2  False
3  False
4  False
5  False
6   True
7  False
```
The field of the first interaction is empty as a burst is a series of interactions, and thus can only occur if there is a previous interaction.

-----------------------
### 9. PairFlips
#### Description
Detects pair-flip events, which is the reversal of the relationship of pair based on a non-cumulative interaction matrix.
This means that one counter attack is enough to reverse the relation and be marked as a pair-flip event. To detect these events
the direction of the relation of the pair involved in the current interaction, is compared to the non-cumulative interaction matrix of the previous interaction.

#### How to use
A new PairFlips object can by defined without any arguments.
To determine whether a interaction is a pair-flip event, the member function `detect()` requires the non-cumulative matrix **d_mC** from a [Matrix](#1-matrix) object of the _previous_ interaction, a row of a Pandas DataFrame containing the current interaction, and the number of the current interaction to use as a index to store the resulting pair-flip value.
The result (True or False) is stored in data member **d_pairFlips** and can be accesed directly as shown below.
This example shows how with a simple for loop pair-flip events can be detected for a set of interactions, by storing the previous non-cumulative matrix.
The first interaction can by definition never be a pair-flip, and thus is skipped.

```python
from dhdat import Matrix
from dhdat import PairFlips
import pandas as pd

df = pd.read_csv("exampleDataSet.csv", delimiter="\t")

actorIDs = [1, 2, 3, 4]             #list of actor identifiers
matrix = Matrix(actorIDs)           
pairFlips = PairFlips()
                                    #will hold matrix previous interaction
prevNCMatrix = pd.DataFrame(index=actorIDs, columns=actorIDs) 
prevNCMatrix = prevNCMatrix.fillna(0)      

for interaction in df.index:
                                    #adds interaction to non-cumulative matrix
    matrix.updateNonCumulative(df.loc[interaction,:]) 

    pairFlips.detect(prevNCMatrix, df.loc[interaction, :], interaction)

                                    #make sure to copy, not assign a reference
    prevNCMatrix = matrix.d_mNC.copy()

print(pairFlips.d_pairFlips)        #shows burst values for all interactions
```

Output:
```
  pairFlip
0    False
1    False
2    False
3    False
4     True
5    False
6    False
7    False     
```
The field of the first interaction is empty as a pair-flip is a reversal of the direction of attack, and thus per definition requires a previous interaction.

-----------------------
### 10. TauKr
#### Description
Calculates TauKr as defined in Hemelrijk (1989), which measures unidirectionality between a set of matrices.
The example below demonstrates how reciprocity of aggression (initiation) can be calculated using this module.

#### How to use
A new TauKr object can be defined without any arguments. With the function `calculate()` TauKr can be calculated from two matrices directly, or by supplying one matrix to `calculate_T()` TauKr is calculated against the transposed version of the supplied matrix.
The matrix that is supplied must be a Pandas DataFrame as used by the [Matrix](#1-matrix) object. Both functions also require the interaction number to use as an index to store the outcome at.
In the example below reciprocity of aggression (initiations of fights) is determined by calculating the TauKr from the **d_mI** matrix from a [Matrix](#1-matrix) object.

```python
from dhdat import Matrix
from dhdat import TauKr
import pandas as pd

df = pd.read_csv("exampleDataSet.csv", delimiter="\t")

actorIDs = [1, 2, 3, 4]             #list of actor identifiers
matrix = Matrix(actorIDs)           
taukr = TauKr()

for interaction in df.index:
                                    #adds current interaction to initiations of aggression matrix
    matrix.updateInitiated(df.loc[interaction,:])

                                    #calculate reciprocity of aggression
    taukr.calculate_T(interaction, matrix.d_mI)

print(taukr.d_TauKr)                #shows TauKr values for all interactions
```

Output:
```
      TauKr
1      -0.5
2      -0.5
3      -0.5
4      -0.5
5      -0.5
6  0.465153
7       0.5   
```
Note index 0 is empty as there are insufficient values to calculate TauKr