Package: femtools
Forecast-elicitation-MechanismImplement 4 papers:Water from Two Rocks: Maximizing the Mutual Information (MCG)Dominantly Truthful Multi-task Peer Prediction with a Constant Number of Tasks (DMI)A Bayesian truth serum for subjective data (BTS)Informed Truthfulness in Multi-Task Peer Prediction (CA)UsageTo beginYou can install the packagefemtoolsusing pippip install femtoolsTo begin, import femtoolsimport numpy as np
import femtools as femBTSFor Bayesian Truth Serum, we implemented the version with finite players. Call the functionBTSwith answersxand predicted frequenciesy, score for every respondent is returned.xandycan be given in thenumpy.arrayform orlistform. If there are n respondents and m possible answers,xshould be an n-dimensional vector and  each answer inxshould be an integer in[0, m). Similarly,yis an*mmatrix denoting the predicted frequencies. BTS score is composed of information-score and prediction score, thus optional parameter alpha controlling the weight given to the prediction score could be assigned between(0,1]. By default,alphais1.Here are examples>>> fem.BTS([3, 2, 1, 1, 0],
... [[0.1, 0.1, 0.3, 0.5],
... [0.1, 0.2, 0.5, 0.2],
... [0.3, 0.4, 0.2, 0.1],
... [0.3, 0.4, 0.1, 0.2],
... [0.1, 0.3, 0.2, 0.4]])
array([-3.28030172, -2.40787449, -0.29706308, -0.29706308, -1.074341  ])

>>> fem.BTS([0, 0, 0],
... [[0.5, 0.5],
... [0.5, 0.5],
... [0.5, 0.5]], alpha = 0.5)
array([0.51873113, 0.51873113, 0.51873113])CAFor Correlated Agreement Mechanism, we implemented the detail-free version. CA Detail-Free is designed for multi-task problem with n agents and m tasks. Call the functionCAwith an*mreport matrixreports, score for every agent is returned.reportscan be given in thenumpy.arrayform orlistform. For convenience, matrixreportscan be given transposed with optional parameteragent_first = False. By default,agent_firstis set toTrue. In addition, functionCAdoes not expect that elements are integers.Here is the example>>> fem.CA([['subway', 'subway', 'subway', 'burgerK', 'burgerK', 'burgerK'],
... ['burgerK', 'McDonald', 'subway', 'McDonald', 'burgerK', 'burgerK'],
... ['burgerK', 'McDonald', 'subway', 'McDonald', 'burgerK', 'burgerK'],
... ['KFC', 'KFC', 'KFC', 'PizzaHot', 'McDonald', 'McDonald'],
... ['PizzaHot', 'PizzaHot', 'PizzaHot', 'PizzaHot', 'PizzaHot', 'McDonald'],
... ['PizzaHot', 'PizzaHot', 'PizzaHot', 'KFC', 'PizzaHot', 'subway'],
... ['McDonald', 'McDonald', 'McDonald', 'McDonald', 'McDonald', 'McDonald'],
... ['burgerK', 'burgerK', 'McDonald', 'burgerK', 'burgerK', 'burgerK'],
... ['burgerK', 'subway', 'subway', 'PizzaHot', 'subway', 'subway'],
... ['burgerK', 'burgerK', 'McDonald', 'burgerK', 'burgerK', 'burgerK'],
... ['PizzaHot', 'PizzaHot', 'PizzaHot', 'PizzaHot', 'PizzaHot', 'McDonald'],
... ], agent_first = False)
array([23, 20, 12, 23, 25, 25])DMICall the functionDMIwith answersxand the number of choicesC.xshould be given in thenumpy.arrayform orlistform. If there are n agents and m tasks,xis an*mmatrix. Please make surem >= 2cand each answer inxis an integer in[0, c), otherwise the function will raise aValueError.  DMI scores will return innumpy.arrayform.Here is an example>>> fem.DMI([[1, 1, 0, 1, 1, 0, 1, 1, 1], [1, 1, 0, 0, 1, 0, 1, 0, 1]], 2)
array([1.5, 1.5])MCGWe implemented the multi-task common ground mechanism MCG(f) for Bernoulli distribution case. Call the functionMCGwith answers, functionfand prior. The answers should be a2*nmatrix innumpy.arrayform orlistform for 2 agents' prediction and all the number in answers should in[0, 1].  The prior is a number in[0, 1], too.fshould be in["TVD", "KLD"]for Total Variation Distance and KL divergence, respectively. By default,f = "TVD". More functions will be supported in the future. The payments will return innumpy.arrayform.Here is an example>>> fem.DMI([[0.2, 0.3, 0.2], [0.3, 0.5, 0.3]], 'TVD', 0.3)
array([0.3333333333333333, 0.3333333333333333])