femtools 0.0.5
Forecast-elicitation-Mechanism

# Forecast-elicitation-Mechanism

Implement 4 papers:

- Water from Two Rocks: Maximizing the Mutual Information (MCG)
- Dominantly Truthful Multi-task Peer Prediction with a Constant Number of Tasks (DMI)
- A Bayesian truth serum for subjective data (BTS)
- Informed Truthfulness in Multi-Task Peer Prediction (CA)

# Usage

## To begin
You can install the package `femtools` using pip

	pip install femtools

To begin, import femtools

	import numpy as np
	import femtools as fem

## BTS
For Bayesian Truth Serum, we implemented the version with finite players. Call the function `BTS` with answers `x` and predicted frequencies `y`, score for every respondent is returned. `x` and `y` can be given in the `numpy.array` form or `list` form. If there are n respondents and m possible answers, `x` should be an n-dimensional vector and  each answer in `x` should be an integer in `[0, m)`. Similarly, `y` is a `n*m` matrix denoting the predicted frequencies. BTS score is composed of information-score and prediction score, thus optional parameter alpha controlling the weight given to the prediction score could be assigned between `(0,1]`. By default, `alpha` is `1`.

Here are examples

	>>> fem.BTS([3, 2, 1, 1, 0],
	... [[0.1, 0.1, 0.3, 0.5],
	... [0.1, 0.2, 0.5, 0.2],
	... [0.3, 0.4, 0.2, 0.1],
	... [0.3, 0.4, 0.1, 0.2],
	... [0.1, 0.3, 0.2, 0.4]])
	array([-3.28030172, -2.40787449, -0.29706308, -0.29706308, -1.074341  ])

	>>> fem.BTS([0, 0, 0],
	... [[0.5, 0.5],
	... [0.5, 0.5],
	... [0.5, 0.5]], alpha = 0.5)
	array([0.51873113, 0.51873113, 0.51873113])

## CA
For Correlated Agreement Mechanism, we implemented the detail-free version. CA Detail-Free is designed for multi-task problem with n agents and m tasks. Call the function `CA` with a `n*m` report matrix `reports`, score for every agent is returned. `reports` can be given in the `numpy.array` form or `list` form. For convenience, matrix `reports` can be given transposed with optional parameter `agent_first = False`. By default, `agent_first` is set to `True`. In addition, function `CA` does not expect that elements are integers.

Here is the example

	>>> fem.CA([['subway', 'subway', 'subway', 'burgerK', 'burgerK', 'burgerK'],
	... ['burgerK', 'McDonald', 'subway', 'McDonald', 'burgerK', 'burgerK'],
	... ['burgerK', 'McDonald', 'subway', 'McDonald', 'burgerK', 'burgerK'],
	... ['KFC', 'KFC', 'KFC', 'PizzaHot', 'McDonald', 'McDonald'],
	... ['PizzaHot', 'PizzaHot', 'PizzaHot', 'PizzaHot', 'PizzaHot', 'McDonald'],
	... ['PizzaHot', 'PizzaHot', 'PizzaHot', 'KFC', 'PizzaHot', 'subway'],
	... ['McDonald', 'McDonald', 'McDonald', 'McDonald', 'McDonald', 'McDonald'],
	... ['burgerK', 'burgerK', 'McDonald', 'burgerK', 'burgerK', 'burgerK'],
	... ['burgerK', 'subway', 'subway', 'PizzaHot', 'subway', 'subway'],
	... ['burgerK', 'burgerK', 'McDonald', 'burgerK', 'burgerK', 'burgerK'],
	... ['PizzaHot', 'PizzaHot', 'PizzaHot', 'PizzaHot', 'PizzaHot', 'McDonald'],
	... ], agent_first = False)
	array([23, 20, 12, 23, 25, 25])

## DMI

Call the function `DMI` with answers `x` and the number of choices `C`. `x` should be given in the `numpy.array` form or `list` form. If there are n agents and m tasks,  `x` is a `n*m` matrix. Please make sure `m >= 2c` and each answer in `x` is an integer in `[0, c)`, otherwise the function will raise a `ValueError`.  DMI scores will return in `numpy.array` form.

Here is an example

```
>>> fem.DMI([[1, 1, 0, 1, 1, 0, 1, 1, 1], [1, 1, 0, 0, 1, 0, 1, 0, 1]], 2)
array([1.5, 1.5])
```

## MCG

We implemented the multi-task common ground mechanism MCG(f) for Bernoulli distribution case. Call the function `MCG` with answers, function `f` and prior. The answers should be a `2*n` matrix in `numpy.array` form or `list` form for 2 agents' prediction and all the number in answers should in `[0, 1]`.  The prior is a number in `[0, 1]`, too. `f` should be in `["TVD", "KLD"]` for Total Variation Distance and KL divergence, respectively. By default, `f = "TVD"` . More functions will be supported in the future. The payments will return in `numpy.array` form.

Here is an example

```
>>> fem.DMI([[0.2, 0.3, 0.2], [0.3, 0.5, 0.3]], 'TVD', 0.3)
array([0.3333333333333333, 0.3333333333333333])
```