1

QnQ: Quality and Quantity based Unified Approachfor Secure and Trustworthy Mobile Crowdsensing
Shameek Bhattacharjee1 , Nirnay Ghosh2 , Vijay K. Shah3 and Sajal K. Das4
1
2

Dept. of Computer Science, Western Michigan University, USA, Email: shameek.bhattacharjee@wmich.edu
iTrust Research Centre, Singapore University of Technology and Design, Email: nirnay_ghosh@sutd.edu.sg
3
Dept. of Computer Science, University of Kentucky, USA, Email: vijay.shah@uky.edu
4Dept. of Computer Science, Missouri Univ. of Science & Technology, USA, Email: sdas@mst.edu

Abstract—A major challenge in mobile crowdsensing applications is the generation of false (or spam) contributions resulting
from selfish and malicious behaviors of users, or wrong perceptionof an event. Such false contributions induce loss of revenue
owing to undue incentivization, and also affect the operationalreliability of the applications. To counter these problems, we
propose an event-trust and user-reputation model, called QnQ,
to segregate different user classes such as honest, selfish, or
malicious. The resultant user reputation scores, are based on
both ‘quality’ (accuracy of contribution) and ‘quantity’ (degreeof participation) of their contributions. Specifically, QnQ exploits
a rating feedback mechanism for evaluating an event-specificexpected truthfulness, which is then transformed into a robust
quality of information (QoI) metric to weaken various effects
of selfish and malicious user behaviors. Eventually, the QoIs ofvarious events in which a user has participated are aggregated tocompute his reputation score, which in turn is used to judiciously
disburse user incentives with a goal to reduce the incentive
losses of the CS application provider. Subsequently, inspired bycumulative prospect theory (CPT), we propose a risk tolerance andreputation aware trustworthy decision making scheme to determine whether an event should be published or not, thus improving
the operational reliability of the application. To evaluate QnQ
experimentally, we consider a vehicular crowdsensing application
as a proof-of-concept. We compare QoI performance achievedby our model with Jøsang’s belief model, reputation scoringwith Dempster-Shafer based reputation model, and operational(decision) accuracy with expected utility theory. Experimental
results demonstrate that QnQ is able to better capture subtle
differences in user behaviors based on both quality and quantity,
reduces incentive losses and significantly improves operational
accuracy in presence of rogue contributions.
Index Terms—Crowdsensing, Quality of information, Trust,
Reputation, Dependable Systems, Participatory Sensing

I. I NTRODUCTION
Sophistication in mobile devices (e.g., smartphones, tablets)
and their widespread adoption have given rise to a novel interactive sensing paradigm, known as Participatory Sensing [9].
A variant of participatory sensing system involving explicit
human participation for sensing the environment, is termed as
Crowdsensing (CS) [14]. In such systems, a crowd of citizens
voluntarily submit certain observations termed as contributions(viz., report, image, audio) about some phenomena in their

immediate environment to a CS server, which in turn fuses
these contributions to conclude a summarized statistic (orinformation) and publishes for the benefit of the public at large.
An important category of CS applications is vehicular trafficmonitoring and management [5]. In such applications, a user’s
contributions are equivalent to ‘reports’ about various traffic
conditions that they might have observed. Based on certain
correlations among such reports, the CS application decides
whether a certain traffic ‘event’ has occurred, and publishes
this ‘information’ as a broadcast notification on a user’s smartphone application. Such information help to improve driving
experiences through dynamic route planning and re-routing oftraffic in busy cities. Two notable examples of real vehicular
CS applications include Google’s Waze (www.waze.com) and
Nericell [26]. Other practical examples include FourSquareand Yelp which help users to find best destinations in theirgeographical proximity for food, entertainment, and otherattractions or events of interest.
The real benefit of CS paradigm is that rich, fine grained andprecise sensory observations can be obtained quickly withoutestablishing dedicated infrastructure [32]. Thus, it reduces
significant infrastructure overheads incurred due to sensor
deployment, management, and periodic maintenance. However,a major drawback is its “open” nature (accessible to all) which
may expose CS applications to false contributions [18], [37],which result in publishing erroneous information.
Most of the CS applications depend on various incentive
mechanisms to motivate the users to keep contributing regularly, and thus preserve their viability [24]. It has been notedthat in most of these mechanisms, the deciding factor ofincentive is the user’s degree of participation (i.e. “quantity" or
how much they contribute). However, selfish users may takeadvantage of this loophole and intermittently generate false
contributions to boost their participation for gaining undue
incentives [32], thereby incurring revenue losses to the CS
application. Furthermore, there could be malicious users who
attempt to cripple the CS applications by generating a large
number of bogus contributions in collusion [37]. Recently, such

2

colluding attack was launched against Waze in Israel, by whichfake traffic jam reports were created to orchestrate traffic rerouting and unnecessary roadblocks [34]. Occasionally, false
contributions may also be generated owing to wrong perception. Regardless of the motive, false contributions incur revenue
loss due to unnecessary disbursement of incentives and alsotarnishes the operational reliability of the CS application.
In a preliminary work, we studied a real data set from
Waze [5], and established that the ‘quantity’ rather than ‘quality’ of contributions decides incentives (details presented in
Section II-B). Here we argue that besides the quantity, there isalso a simultaneous need for assessing quality of information(QoI) generated from user contributions. This QoI is essentiallya measure of trustworthiness of the summary statistic and isequivalent to its trust score. Additionally, user reputation basedon his level of truthful participation is required to determine:(i) if a user is honest, selfish, or malicious; (ii) the incentive
received by the user; and (iii) acceptance of future reports fromthe user for decision making.
A. Motivation of this Work
Apart from other expensive methods (e.g., ground truth,
sensor based) of evidence collection, a simple way to assess
QoI is to allow other users in the proximity to provide a
feedback rating (viz., positive, negative, or uncertain) for eachpublished information [20], [32]. Based on such feedbacks(serving as evidence), the event trust and user reputation arequantified. In the literature, most existing trust and reputation
models are based on Jøsang’s belief model [20], DempsterShafer reputation [42] model, or their variants (see Sec. II for
details).
A synthesis of existing works reveal that they only utilize theproportion of positive feedbacks in the QoI measure. However,
we show that accurate QoI scoring should also include theeffect of total number of feedbacks (i.e., feedback mass) thata published information has received. This step is important toweaken the ill effects of malicious ratings. Second, most existing works do not consider a dynamic discounting of uncertain
feedbacks to ensure that the QoI measure is null invariant (i.e.,
not influenced by high uncertainty or orchestrated inconclusive
feedbacks). Third, and most importantly, these models are
not able to propose a reputation scoring model that unifies
both degree of participation (quantity) as well the quality ofeach contribution. Fourth, existing models provide insufficient
provisions for embedding heterogeneity among various CS
providers in terms of economic behaviors such as risk tolerance
attitude under possibilities of threat and uncertainty.
B. Contributions of the Paper
This paper proposes a model, called QnQ, for trust andreputation scoring in a CS system in presence of maliciousand selfish users. First, we propose a QoI measure for every
published information. Based on the feedbacks received overa particular published information (event), we calculate theBayesian inference based belief, disbelief, and uncertainty
masses. Thereafter, we model the expected truthfulness of the
published information as a regression model using generalized
Richard’s curve and Kohlsrausch relaxation function as theweights to belief and uncertainty masses, respectively. This
step weakens the effect of malicious feedbacks (such as ballot
stuffing or bad mouthing) while also being null invariant
against obfuscation stuffing attack in the sense that our model is
not influenced by high uncertainty or orchestrated inconclusive
feedbacks. Subsequently, we transform the expected truthfulness to a QoI (trustworthiness) measure that captures the odds
of an event’s occurrence. The transformation is achieved using
cumulative prospect theory (CPT) inspired link function that
captures varying risk tolerance attitudes (risk seeking, risk
averse, risk neutral).
Second, we keep track of the QoI measure of all the
published information contributed by each user via reports,
and then calculate a raw user reputation score by aggregating them. The aggregated raw user reputation is normalized
within an interval of [-1, +1] through a logistic distributionfunction. This normalized user reputation score is: (i) utilized
for classification of users into honest, selfish and malicious;(ii) judicially disburse incentives based on both his degree ofparticipation (quantity) and quality of those contributions; and(iii) utilized for robust decision making.
Third, we propose a CPT inspired two-level decision making
scheme that exploits the reputation scores and other contextual
information to improve accuracy of publishing true events
while avoiding false (spam) events. In contrast with otherworks, a significant benefit of our scheme is that it can embed
heterogeneity that might exist among various CS providers interms of economic behaviors such as risk or loss aversion andavoid certain biases that negatively affect decision accuracy.
Finally, we conduct extensive performance to evaluate theproposed QnQ model using a vehicular crowdsensing system
as a proof-of-concept. We use some real data from Wazeand Epinions to parameterize the simulation environment. We
demonstrate that our approach outperforms Jøsang’s belief
and Dempster-Shafer (D-S) based reputation models in termsof classification, incentivization, and scalability. Experimental
results show that QnQ is able to give a reputation score, thatrewards both quality and quantity and saves significantly on
incentives in presence of dishonest users while maintaining
fairness. Furthermore, we show that our cumulative prospect
theoretic decision making scheme ensures better operational
accuracy compared to expected utility theory (EUT) basedmodels. We also present some recommendations for the systemparameters that show how QnQ can be adapted to any CS
application’s requirements.
The rest of the paper is organized as follows. Section II
summarizes the limitations of the existing literature. Section III
describes the system and threat models. Section IV proposes
the QnQ model for trust and reputation scores while Section V

3extends the QnQ model for trustworthy decision making.
Section VI presents results and performance. Section VII
discusses parameter recommendations and extensions undervarious CS systems while Section VIII offers conclusions andfuture research directions.
II. L IMITATIONS OF E XISTING W ORKThis section reviews the state-of-the-art research for QoI
and user reputation scoring models and certain important
limitations of existing literature for crowdsensing applications
under selfish and malicious users.
A. Quality of Information (QoI) Scoring Models
The QoI scoring aims at assessing the ‘veracity’ of theinformation contributed by the users. The veracity assessment
may be either on the individual reports or on the inferred
information statistic. Broadly, the QoI is assessed by modelingevidence obtained using (i) ground truth, (ii) similarity based
outlier detection, (iii) spatio-temporal provenance, (iv) priorreputation context, and (v) the rating feedback mechanism.
Such evidence is then mathematically modeled into a QoI.
However, the availability of ground truth is not immediate,
and often not guaranteed or feasible. Additionally, acquiring
ground truth often requires deployment of dedicated infrastructure, or agents thus obviating the relevance and benefitsof crowdsensing. Similarity based outlier detection [2], [18],[40] awards higher QoI to an event if most of the user’s
contributions agree in terms of event type, location, and timestamp. However, for fake events orchestrated by a group ofrogue users, high degree of similarity among contributions is
implicit, since honest users will not be reporting anything. SuchQoI scoring therefore, fails under orchestrated fake events.
Spatio-temporal provenance based schemes [38] assume existence of a prior and reliable reputation scores of each userand quantifies QoI based on prior reputation and similarity inreports. However, this does not address the cold start problem
and variability in user behaviors.
Some real CS applications, viz., FourSquare, Waze, YikYak,
Yelp, Ebay, etc. use a rating feedback mechanism, whereby
other consumers/agents of the service provide positive, negative
or neutral ratings on the published information. For example,
in case of a 5-star rating system, the ratings 4 and 5 correspondto positive, 1 and 2 as negative and 3 as neutral. The estimationof QoI is achieved based on the feedbacks received. The
benefits of using a feedback rating paradigm are that it iseasy, fast, and less expensive. Moreover, it really exudes theessence of a true mobile crowd sensing paradigm and do
not suffer from weaknesses of the other evidence modelingapproaches. In most cases, QoI scoring is done using variantsof Jøsang’s belief model [20] that compute the QoI based on
the ratio of positive feedback to the total feedback with somefixed weight to the ratio of uncertain feedbacks. Nevertheless,there exist threats such as ballot and obfuscation stuffing in

rating feedback paradigms. We observe the following inherent
weaknesses in Jøsang’s belief models:Confidence of the Feedback Community: Jøsang’s belief
model (details in Appendix A) fails to capture the differencesin confidence of the feedback community, thereby making theresultant expected belief (QoI in our case) more vulnerable
to manipulation by malicious raters who provide positive
ratings to false events (Ballot Stuffing attack) and vice-versa(Bad Mouthing attack). This may influence the QoI score offalse events in favor of the adversaries. As shown in Table I,each event is denoted as E : N, r, s, t, where N is the total
number of received ratings while r, s, t are positive, negative,
and uncertain ratings, respectively. For event E1, 3 out of 7
feedbacks are good, whereas for event E2, 30 out of 70 are
good. Jøsang’s belief model generates almost the same QoI in
both examples. From an adversary’s perspective, it is easy tocompromise or manipulate 3 good raters in E1 and maintainthe same fraction of positive ratings as E2. However, it isharder to maintain the same fraction when the crowd is large(as in E2), in which case the adversary has to manipulate 30
raters. Hence, given the same fraction of positive feedbacks,
any event with more feedbacks should be considered as
more trustworthy. If this feature is not incorporated, the QoI
becomes more vulnerable [8].
Table I: Limitations of Jøsang’s Belief Model
IssuesConfidence of Community
Not Null Invariant

ExamplesE1:7, 3, 2, 2
E2:70, 30, 20, 20
E3:105, 5, 0, 100E4:25, 5, 0, 20

Jøsang’s QoI
0.55
0.57
0.51
0.53

Not Null Invariant to Uncertainty: Jøsang’s belief models do
not offer null invariance property. This means that QoI ofan event can achieve unwarranted increased trustworthinessdue to high proportion of uncertain feedbacks, which may beeither intentionally generated (Obfuscation Stuffing attack), or
be a result of legitimate uncertainty. Either way, such event
should not unduly increase the trust (QoI) score. For example,
event E3 in Table I, has 100 uncertain feedbacks out of 105.
However, it achieves almost the same QoI as event E4 whichin contrast has only 20 uncertain ratings. For most services,it may be risky or unwise to give as high a QoI score to E3
as that of E4. Thus, the QoI scoring needs a mathematicalprovision for controlling the impact of high uncertainty on it.
B. User Reputation Scoring Models
Traditionally, reputation scoring models in crowdsensing use
either Beta (for binary evidence) or Dirichlet distributions (formultinomial evidence) as theoretical basis for probabilistic andevidence based reputation modeling [27]. Jøsang’s belief [20]and Dempster-Shafer (D-S) based trust models [42] provide
the state-of-the-art approaches that exploit either of these
distributions to model rating feedback based evidence into trustor reputation scores.

10

12000

8

10000

6

Frequency

Incentives
Ratings

4
2

8000
6000
4000
2000

0.
32
0. 04
64
0. 08
96
1. 12
28
1. 16
60
1. 20
92
2. 24
24
2. 28
56
2. 32
88
3. 36
20
3. 40
52
3. 44
84
4. 48
16
4. 51
48
4. 55
80
20 60
1
32 .85
2
32 .642.
9600Degree of Participation(a) Quality vs Quantity

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
63
0
10
0
10 7
08

Score

4

possible breaches loom large, if evidence suggest possible
threats or presence of considerable uncertainty. Unfortunately,
existing trust and reputation scoring models poorly capture theeconomic behavioral aspect linked to risk perceptions. Our
proposed model, on the other hand has provisions to adapt
according to different risk attitudes.

No. of Reports(b) Report Generation Frequency

Figure 1: Study on Waze Dataset
Recent works [2], [18], [27] have proposed a deterministic time varying reputation management system based on
Gompertz functions. However, rather than evidence based
scoring, they mainly investigate the evolution of trust overtime and do not actively assume threats. Most of the recent
works [2] [3], [32] [31] [13] [39] do not consider orchestrated
dishonest reports, consequent economic implications attachedwith the reputation dynamics, cannot unify quality and quantityof participation, has no provision to handle uncertainty, andcannot thwart the effect of rogue ratings. To our understanding,some limitations of these models are as follows:
Sacrificing Participation for Quality: D-S model [42] (as described in Appendix B) does not fairly capture the degree ofparticipation and quality together into the reputation score.
Table II illustrates this limitation. Although users 1 and 2
have the same reputation, the latter with 52 additional good
contributions ends up with a score almost similar to user 1. Thisis unfair and undermines the higher participation of users. If
the same high reputation is attainable with lower contributions,the users will not be motivated to participate, and hence theexistence of CS will be threatened.
Table II: Sacrificing Participation for Quality
User
1
2
3

Participation
9
61
20

Good
9
61
18

Bad
0
0
2

Dempster Score
0.99
1.00
0.99

Sacrificing Quality for Participation: In [5], we studied a real
data set from Waze and identified that quality may be sacrificed
for participation. Fig. 1b shows that the majority of the users
have generated around three reports over the span of one week.
However, there are a few users who have generated a verylarge number of reports (around 600 to 1000). Additionally, it
is evident from Fig. 1a that the incentive of the users gradually
increases with higher participation rate. Conversely, the ratings
assigned to the users with high participation are very low
and even drops to zero while maximum incentive is received.
Thus, the reputation score of a user needs to unify both degree(quantity) and quality of participation.
Lack of Adaptive Risk Modeling: The general notion of trust
and reputation has emphasized the need for incorporating therisk tolerance attitude of the defender. This is because twoentities may perceive the same evidence differently because theformer might have more to lose than the latter in case of an
unfavorable breach of trust. Before entering into a relationshipof dependence, the concerns over potential losses caused by

Figure 2: System Model
III. S YSTEM AND T HREAT M ODELSIn this section, we present the system and threat models andthe underlying assumptions.
A. System Model
Fig. 2 depicts the system model which consists of U users,
each equipped with a smartphone and subscribed to a vehicular
CS application. Two important aspects of this system are:
Report: A report is an alert furnished by a user after he perceives an incident (viz., accident, jam, road closure). However,
due to the presence of selfish and malicious users, there may
be reports generated for incidents that have never occurred.
Event: An event denoted by k ∈ E = {1, · · · , |E|} isa summarized information which is published after the CS
application receives a predefined number of “similar” reports.
Each k belongs to one out of a set of possible event types
denoted by E indexed as j ∈ {1, · · · , |E|}. If reports fromtwo different users indicate similarity in terms of location,
time epoch (t), and event type, they indicate the ‘same’ event.
As evident from Fig. 2, an event can be either true or false
and has a boundary within which subscribed users are liable
to participate. Event boundaries can be constructed throughvarious geo-spatial clustering methods from GPS stamps to
MGRS (Military Grid Reference System) conversion [18].
In the system model, there exists two types of users:
Reporter: A reporter is a user who has a propensity to generatereports and has reported at least one event. Any such user isliable to have a reputation score which reflects the overall quality of reports contributed, as well as the degree of participation.
Furthermore, to remove biases from the feedbacks, a reporter
is not allowed to rate a published event for which he himself
has generated a report.
Rater: A rater is a user who provides feedback on his perceived
usefulness of an event as: Useful (α), Not Useful (β), and Not
Sure (γ). For a published event, a rater is allowed to submit
only one rating.

5

In our system model, we view the act of providing ratingsas an obligation, and it is not rewarded. Hence, for majority
of normal users, there is no selfish incentive to provide false
ratings, although some false ratings could be motivated bymalicious intent.
The design principle of the system model entails collectionof as much feedbacks as possible to get a notion of truthfulnessof the event in absence of ground truth. This may be achievedby pushing a pop-up rating query to all active users navigating
through the event boundary (provided that that user has notreported that event under scrutiny). The rating query asks toclick one of three possible options: Useful, Not useful, and Notsure, to gather a subjective judgment about the published event.
B. Threat Model and Assumptions
Dishonest reporters may be selfish or malicious. A selfish
user is a legitimate user who generates true and false reports
intermittently, with certain probabilities for maximizing his
incentives. We consider two variations of selfish users. The first
type reports more true events than false events and the othertype reports more false events than true events. The rationaleof two selfish user subtypes is explained in Section VII-B.
In contrast, malicious users are either actual user devices
compromised and (or hardware emulators) controlled by anadversary that may intentionally act in collusion to cripple theCS platform. A part of these devices can be made to act incollusion as reporters to generate a false event while the restare ‘malicious’ raters who may act in the following ways:
Ballot stuffing: A rater submits positive feedbacks to an incorrect (false) published event (generated by dishonest reporters).
Obfuscation stuffing: A rater submits uncertain feedbacks to a(false) published event (generated by dishonest reporters).
Bad mouthing: A rater submits negative feedbacks to a legitimate published event (generated by honest reporters).
Note that, hardware emulators can further generate numerous
sybil interfaces to magnify the problem of false reports andratings. However, such sybil interfaces could be identified byexisting methods [37]. Hence, we assume that it is only the
hardware emulators or compromised devices which pose athreat of colluding attacks. In general, the adversary has a fixedattack budget by which it can compromise or manage a limited
number of reporters and raters. This is evident from [37], where
the authors generated 1000+ sybil (virtual) interfaces to collude
a Waze-like application, but were compelled to deploy only 10emulators (physical systems) due to budgetary constraints.
Hence, the adversary uses this finite attack budget that
effectively manipulates a fraction δmal of the reporters andraters in order to generate false reports and ratings. It will
be a significant fraction for areas with limited number oflegitimate users. However, in presence of a significant crowd of
independent users, δmal will be low, and it will not be possible
to sabotage the entire proportion of genuine feedbacks. Since
crowdsensing paradigms are more prevalent in urban spaces, it

Table III: Notations
SymbolNkα, β, γb, d, uwbwu
Ab , Au
Bb , Bu
ν
ϕ
max
wu
τk
Nthres
θ1
φ1
Qk
Si
Ri
j
Cj
v(Cj )
Nagg (j)Ragg (j)ρ
U + (z)θ2
φ2π+ , π−δ1 , δ2
pj

Meaning
Total Number of Ratings
Published Event ID
Rating Categories
Probability Masses for Rating Categories
Weighing Function for b
Weighing Function for u
Initial Asymptote for wb , wu
Growth Rate for wb , wu
Tipping Point for wb , wu
Kohlsrausch Relaxation Parameter for wu
Maximum benefit of doubt towards u
Expected Truthfulness of a k-th published event
Rating Mass where discounting of u starts
Gain Exponent for QoI value function
Loss Exponent for QoI value function
QoI Score of Event k
Aggregate Score of User i
Normalized Final Reputation Score of User i
Event Type ID
Confidence on event type j
Value function of Cj
Total ratings for the j-th event type
Total reputation for the users reporting j-th event type
Preference factor for Cj
Total set of active users in z-th region
Gain Exponent of v(Cj )Loss Exponent of v(Cj )Probability Weighing Functions
Steepness Exponent for π+ , π −Prior Likelihood of occurrence of j typemay be assumed that for majority of times, substantial numberof authentic raters are likely to be present in the vicinity ofan event, thus reducing the proportion of false ratings to thetotal number of feedbacks. In most cases, with larger rater
populations, the rating mechanism becomes less likely to getsabotaged. For any rating-based system, the number of raters isalways higher compared to the number of reporters generating
reports/reviews. Our study from the Epinions dataset [25]shows that the number of feedbacks is roughly three to fourtimes the number of reviews (reports) for any item.
IV. Q N Q: P ROPOSED R EPUTATION S CORING M ODELNow we present the modules of the proposed reputation
scoring model, called QnQ. The model is divided into 4 major
phases. First, the posteriori probability masses phase calculates
for each event published, the belief mass for each rating category using Bayesian inference. Second, the QoI scoring phasecalculates QoI for each event through a non-linear weighted
regression score (expected truthfulness) followed by a modified
Tversky-Kahnemann link function. Third, the user reputation
phase accumulates the QoI of events over a time window andassociates them with the users who generated them to calculate
an aggregate user reputation score. Fourth, the trustworthy
decision making phase utilizes the user reputation scores,
contextual evidence, etc., to make event publish decision moreaccurate and is presented in Section V.
A. Posteriori Probability Masses
The first step is to derive the expressions for the posteriori
probability masses associated with rating feedbacks: Useful,

P (F (N )|ω̄) =N!
ηω ηα ω β ω ηγ
ηα !ηβ !ηγ ! α β γ(1)The posteriori hypothesis of positive outcome of any event
based on the evidence vector and assumed prior is given as:
P (H(ω̄) = α|F (N )) =P (H(ω̄) = α, F (N ))P (F (N ))(2)

Similarly, the posteriori hypothesis of negative and uncertain
outcomes can be represented by replacing α with β and γ
respectively in Eqn. (2). Solving the above (see [6]), belief,
disbelief, and uncertainty probability masses of an event are+1= b,
derived as follows: P (H(ω̄) = α|F (N )) = ηNα+3
η +1
P (H(ω̄) = β|F (N )) = Nβ+3 = d, and P (H(ω̄) =η +1
γ|F (N )) = Nγ+3 = u, respectively. These are the posteriori probability masses for Useful, Not Useful, and Not Sure
feedbacks as perceived by the raters, respectively. Note that,when ηα = ηβ = ηγ = 0, all the possibilities are equiprobableunder no information (i.e., non-informative prior).
B. Expected Truthfulness of an Event
Since trustworthiness is related to choice under uncertainty and risk, it is natural that trustworthiness of an event
should account for uncertain evidence apart from the positive
evidence [11], [20]. Thus, we propose wb and wu as thecoefficients (or weights) of belief and uncertainty masses
respectively, where the weights control the extent to which
positive and uncertain probability masses contribute to thetruthfulness score of k-th event. The problem is modeled similar to a weighted regression approach where probability masses
are explanatory variables and the expected truthfulness is a
response variable. We apply Richard’s generalized curve [30]and Kohlrausch relaxation functions [4] to model wb and wu .
The expected truthfulness for any published event k is:τk = (wb ).b + (wu ).uwhere, 0 < {wb , wu } < 1. Hence, 0 < τk < 1.

(3)

1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0

wb

Not Useful, and Not Sure. The probability masses are estimated for each event k based on the available evidence(i.e., supporting each rating type), using a classical Bayesian
approach. Let ω̄ = {ωα , ωβ , ωγ } be the three tuple probability
parameter to be estimated. Here, ωα , ωβ , ωγ are the unknown
probabilities of observing a Useful, Not Useful, or Not Sure
feedback, respectively. We denoted H(ω̄) as the hypothesis,
such that it has three possibilities of either taking α , β or γ.
Formally, P (H(ω̄) = α|ω̄) = ωα , P (H(ω̄) = β|ω̄) = ωβ ,P (H(ω̄) = γ|ω̄) = ωγ . Let Fα , Fβ , and Fγ be the randomvariables denoting the number of feedbacks ηα , ηβ , and ηγ ,received for each feedback category, respectively, such that
Nk = ηα +ηβ +ηγ . For simplicity, we drop k from all the notations. The evidence vector, denoted as F (N ) = {Fα , Fβ , Fγ },should be modeled as a multi-nomial distribution given by:

Belief Weight (Wb)6

Bb = 0.04
Bb = 0.08
Bb = 0.24
0

20

40

60

80

100 120 140# of Ratings(a) Growth Rate:Bb

1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0

0

50

V = 0.5
V = 0.25
V = 0.0625
100
150

200

# of Ratings(b) Effect of Various ν

Figure 3: Parameters of Richard’s Equation
1) Design of Belief Coefficient: We mentioned earlier that
expected truthfulness should also consider the volume of the
feedbacks, i.e., how many feedbacks have been received foran event apart from the belief mass b. Intuitively, lesser N(total number of feedbacks/ratings) should have lower wb ,which in turn, contributes to a smaller expected truthfulness.
However, wb should gradually increase with N . Thus, to model
this nature of wb , we use a Generalized Richard’s Equation
normalized between 0 and 1 as:
1
wb =
(4)
−B
(1 + Ab e b N R )1/ν
where Ab > 0 but Ab = ∞ is the initial value of the coefficient,
Bb is the rate of growth, and ν = 0 is the parameter controlling
the point where the curve enters into exponential growth.
Physical Significance of wb : The wb is modeled by theRichard’s curve (refer to Fig. (3a)) and is motivated fromdeductive reasoning and learning studies in cognitive psychology [30], [11]. Intelligent humans are subconsciously rationalenough to know that the possibility of a biases negatively
affecting a belief inference is greater, if fewer number ofsources say the same thing (b in this case), as opposed to thesame endorsed by more sources. Hence, a Bayesian inference
backed by more people/sources carries more weight than thesame Bayesian inference backed by fewer people/sources.
This phenomena is modeled through incremental change processes [30], [11], that are characterized by a slower initial
phase followed by an inflection point where the learning rate
exponentially peaks in the face of increasing evidences andfinally saturates into a stationary phase where the learning rate
approaches an upper asymptote. Such provisioning enables the
CS application to better nullify the effects of ballot stuffing.
More details on the advantage of using richards’ curve isdiscussed in our preliminary work [8].
2) Design of Uncertainty Coefficient: In Eqn. (3), wu controls the contribution of uncertainty mass to the effective truthfulness. Intuitively, uncertainty is high if an incident has justoccurred, and the majority of users are uninformed. However, itgets reduced as more feedbacks are received. Thus, for smaller
values of N , we should have an increasing function for wu . As
this is also similar to growth curve, we model by a Richard’sfunction upper bounded at wumax . However, once N attains athreshold value, say N = Nthres , the coefficient should start
to decrease. The value of Nthres and wumax depend on theϕ = 0.2
ϕ = 0.4
ϕ = 0.8

20

40

60

80

100

# of Ratings(a) Kohlrausch Factor ϕ

120

0.6

wumax

0.5
0.4

Nthres = 25
Nthres = 50
Nthres = 100

0.3
0.2
0.1
0
0

50

100

150

200

250

# of Ratings(b) Nthres

Figure 4: Impact of Parameter Choices on wu
empirical data of relevant application scenario and risk attitude(as later discussed in Section VII-D.
Typically, Kohlsrausch relaxation function [4] is used tomodel the property of a system that evolves towards equilibrium after sudden perturbation or a trigger. In our proposedmodel, after N = Nthres , this function is used to capture
the discounting effect of uncertain ratings on trustworthiness.
Its parameter ϕ, where, 0 < ϕ < 1, controls the rate ofdiscounting of wu with N . The larger the value of ϕ, the moreis the decrease (refer to Fig. 4a). The following equation gives
the variation of wu w.r.t the number of received feedbacks:
⎧
max
wu
⎪
,
if N R < Nthres⎨ (1+Au e−B
u N R )1/ν
wu =
(5)
⎪
⎩ e−(N R−Nthres )ϕ ,if N R ≥ Nthres
where Au and Bu are respectively the corresponding asymptote
and growth parameters (as discussed in Eqn. (4)). Note that,0 < wumax < 1 is a fixed parameter controlling the maximum
allowable benefit of doubt for an event. Choice of wumax should
be guided by risk attitude or availability of trusted agents [32].
Physical Significance of wu : The concept of trust cannot
exist without a certain level of acceptance of uncertainty [10].
Especially for trusting some decisions that tend to be objective,
people tend to give some ‘benefit of doubt’ if uncertainty
is reported from a small number of people. But if the same
uncertainty mass occurs even as more people/sources haveparticipated, the effect of that uncertainty does not contribute to
the increase of trust, since the risk perception is magnified [11].
The uncertainty involves a trigger point (or knot point), aroundwhich there is a relatively brisk reorientation of the existingstate of ‘benefit of doubt’ into a qualitatively different stateof discounting the benefit of doubt. Such phenomenon indevelopmental learning theory is known as transformationalchange processes [30], which fit into a family of spline curves
and these phase transitions are modeled by multiple equations
around the knot point [10]. The nature of wu mimics such
effects on the modeling of uncertainty evidence. Appendix Dshows how this step increases resilience to obfuscation attacks.
C. QoI of Published Event
In Eqn. (3), τk is the expectation that the published event k
has actually happened. Now, the CS system needs to determine
the odds of k-th event being true or false which we model as

the QoI. When the response/predictor variables are categorical(true/false, yes/no, etc.), the error distribution is non-normal.
We need a link function to provide the relationship between thepredictor variable and the mean of the distribution defining the
QoI. Normally, under risk neutral case, a logit link function isused as in our preliminary work [8]. However, when it comesto trust relationships under risk and uncertainty given economic
objectives, another factor to be considered is the risk tolerance
attitude. Logit link function is inappropriate as it does not havea provision to embed such attitude while making decisions.
Therefore, we propose the use of cumulative prospect theory(CPT) inspired link function to embed such risk tolerance.
CPT [23] [36] is a descriptive model of how a decision makerperceives/interprets risky prospects that may lead to losses andgains. CPT properties relevant to our work are given below:
1. Reference Point: A decision maker judges a prospect basedon the potential gains or losses with respect to a reference
point, which acts as a neutral boundary about which gains orlosses of an outcome are visualized. In our case, τ = 0.5 isthe neutral point of the outcome variable.
2. Asymmetrical Value Function: A decision maker is by default risk or loss averse, and thus he strongly prefers avoiding
losses than achieving gains. As a result, the value function is
S-shaped and asymmetrical. Mathematically, it is concave forgains, convex for losses, and steeper for losses than for gains.
3. Principle of Diminishing Sensitivity: A decision makertends to over-react to smaller deviation from the reference point
and the sensitivity decreases at the boundary points.
The link between τk and the QoI of the event Qk is establishedby the following modified value functions from CPT:⎧if τk ≥ 0.5⎨ (τk )θ1 ,
(6)
Qk =
⎩
−λ1 (0.5 − τk )φ1 ,if τk < 0.5
where Qk has the value in the interval [−λ1 , 1], λ1 > 1, θ1 >0, and 0 < φ1 < 1. The variations of QoI with respect todifferent parameters are depicted in Fig. 5.

QoI

0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
0

Uncertainty Weight (wu)

Uncertainty Weight (wra)

7

1
0.5
0
Reference Point
-0.5
-1
-1.5
-2
θ1 = 2.5, φ1 = 0.6, λ1 = 3
θ1 = 0.88, φ1 = 0.88, λ1 = 2.5-2.5
θ1 = 1.25, φ1 = 0.3, λ1 = 3.5-3
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Expected Truthfulness (τ)

Figure 5: Risk Aware Modified Link Function
As evident from Eqn. (6), Qk has two value functions oneither side of the reference point, (τk = 0.5). The exponents θ1
and φ1 enable the CS administrator to control the rate of changeof QoI above and below the reference point respectively. The
values taken may vary according to the risk attitude. For
instance, if the administrator wants to prevent loss of revenue
and business goodwill of the application, he exhibits risk averse
attitude through a very gradual increment of QoI (with θ1 > 1)

8

w.r.t to increasing expected truthfulness (in Fig. 5, blue andgreen curves: convex nature) starting from the neutral point.
However, if the application can afford a ‘less’ risk averse
approach by setting θ1 < 1 (red curve: concave nature). The
choice of θ1 in Eqn. (6) can embed the contextual risk toleranceof various PS providers accordingly.
In contrast, the nature of QoI below the reference pointis always convex and its steepness depends on the exponent
φ1 . This is because, the CS administrator will always showloss aversion attitude and perceive steeper loss of quality with
τ < 0.5. The loss penalty parameter λ1 determines the lowest
value which the QoI can attain. Lower QoI will incur penalties
on the defaulters and the administrator will be satisfied if
it encourages rogue users to churn out of the system. This
will implicitly prevent loss of both revenue and operationalreliability of the application.

V. T RUSTWORTHY D ECISION M AKING S CHEME
The work until now was dedicated to build a reasonably
genuine user base via computation of QoI through feedbacks
received against published events. Once such an user base isformed, the event publishing step itself could be made more
dependable and accurate. In this section, we show how theQnQ reputation can be applied for trustworthy and dependable‘event publishing’ decisions. As discussed in Section III, avehicular CS (like Waze) application may receive reportsthat indicate either of the following event types, viz., jam,accident, road closure, weather hazard from a potential event
boundary, say z. Such an event boundary can be constructed
through a geospatial grid clustering of received GPS stamped
reports [18]. Intuitively, we should only consider reports fromusers with Ri > 0 for the decision making in z.

D. QoI-based User Reputation Score
For any reporter i, we match the reports he had generated
with the estimated QoI value of the corresponding events. We
sum up Qk for every unique event reported by reporter i, tocalculate the aggregate reputation score Si .
Si =
where,I(k, i) =|E|


Qk I(k, i)

(7)

k=1

1,
0,

If i reported event k
Otherwise(8)

E. Normalized User Reputation Score
The aggregated reputation score Si obtained from Eqn. (7)is a real number in the interval [−∞, +∞]. In order to make it
intuitive and consistent with the definition of trust metrics, we
use the logistic distribution function to map its values in theinterval [−1, +1]. Therefore, the final reputation score (Ri ) ofa reporter i is given as:
⎧ 
⎪
⎪
1
⎪ +
,
if Si > 0
⎪
+
⎪
S −μ
⎪
− i +s
⎪
C
1+e
⎪
⎪
⎪
⎨ 
Ri =
(9)
1
⎪
<
0
−
,
if
S
⎪
−
i
⎪
|Si |−|μs |
⎪
−
⎪
C−
1+e
⎪
⎪
⎪
⎪
⎪
⎩
0,
if Si = 0−
where μ+s and μs are the mean reputation scores for reporters withpositive and negative
Si respectively. Similarly,
√
√
3σs+
3σs−
+
−
and C =
where σs+ and σs− areC =ππ
the standard deviations for reporters with positive and negative
Si respectively. At t-th epoch, Rit , denotes the steady state
reputation at the t-th epoch. The reputation may be calculated
at the end of a predefined time window of multiple epochs.

Figure 6: Decision Making Scenarios
However, accurate decisions are still non-trivial in absence ofground-truth due to the following possibilities: First, potential
event boundary constructed from clustering step are not disjoint
as shown in Fig. 6. This can produce a mixture of relevant andirrelevant reports for a given event. Second, uncertainty looms
regarding the actual behavior of the new users (with no reputation) registered in the current time epoch. Third, some users
can build higher reputation first, and then start orchestrating
fake events. Fourth, although prior spatio-temporal likelihood
is often considered, we believe that event occurrence is still not
deterministic. Hence, rarer events may not be favored and neverget published, while malicious users can orchestrate a fakeevent in a location with high prior likelihood, both triggering
wrong decisions.
Wrong decisions (‘publishing’ fake events or ‘not publishing’ real events) will incur losses, while correct decisions(publishing events that occurred and not publishing when no
event occurs) will accrue gains. Therefore, the decision making
is confronted with the following four utility prospects: (i)‘publish’ given event type j ‘did not occur’ (P|e), incurring
a loss of l1 ; (ii) ‘not publish’ a given event type j that
actually occurred (P|e), incurring a loss l2 ; (iii) ‘publish’ given
event type j actually occurred (P|e), which incurs a gain of
g1 ; and (iv) ‘not publish’ given event type j did not occur(P|e), that incurs a gain of g2 . The gains are positive utilities(g1 , g2 > 0) while losses are negative utilities (l1 , l2 < 0),such that g1 > g2 > l1 ≥ l2 , is the ordering of the utility
prospects in terms of decreasing profits. Since, the real goal is9to publish an event when it occurs, g1 is strictly greater than g2 .
If an event has not occurred, but the CS system has received
supporting reports and still it successfully prevent itself frompublishing the false information, then it will be considered asa finite positive gain g2 > 0. In the loss front, l2 could begreater than l1 , if missing a true event is causing more loss
than publishing a fake event and vice-versa. However, for ourwork, we considered them equally bad and assumed l1 = l2 .
To solve this, we propose a two-level decision process thatmaximizes the gain corresponding to correct decisions whileaccounting for the losses for wrong decision depending on theapplication’s risk policy.
Decision Levels: The application needs to determine in nearreal-time: (D1) what event type to publish (i.e., the most likely
event type that has occurred), and (D2) whether to publish an
event (i.e., if sufficient evidence exists to suggest that the most
likely event has actually taken place).
The first decision (D1) is required since reports belonging
to more than one event type may be received at the same timeepoch in the event boundary z. Therefore, the first decision isto decide in runtime the most likely event type (winner event).
To achieve this, a confidence value for each reported event
type j is computed based on the relative quantity and quality
support for each event type j.
The second decision (D2) is required because, even if there
is a clear likely event type, there may not be strong overall
evidence to suggest that publishing this event will result ina benefit or gain. This is particularly true for preventing
orchestrated fake events, because honest reporters will notreport anything in the absence of any event. Such decision
problems are modeled as a decision tree [28].
Solution Methodology: We need to compute the final utilities
of both ‘publish’ util(P) and ‘not publish’ decisions util(P)and the event is published if util(P) > util(P). Most decision
trees are classically solved by the expected utility theory (EUT).
However, recent research in behavioral economics and decision
theory showed strong empirical evidence decision making
under risks and uncertainty does not follow EUT. In fact,
cumulative prospect theory (CPT) [23] [36] in recent decades
have become a stronger and realistic descriptive model for
decision making under risk and uncertainty particularly byhumans. While, CPT was originally proposed as descriptivemodel for human decision making, we propose a modified
application of the original theory which fits an automateddecision making system. The goal of such modification is topreserve the advantages of CPT while simultaneously avoiding
certain biases humans suffer from, that result in occasional
irrational choices. As an example, we show that our model ismore likely to publish true events with low prior likelihood of
occurrence, and avoid fake events at locations which have ahigh prior likelihood in contrast to existing approaches. To thebest of our knowledge, such an efforts have not been made inautomated trustworthy decision making for CS applications.

(a) Level 1(b) Level 2

Figure 7: Two Level Decision Scheme
The decision trees for the proposed two-level decision
scheme are given in Fig. 7a and Fig. 7b, respectively. For
the first decision level D1, we calculate the raw confidenceof each type j, adjust it with the risk attitude to a utility
value. The second decision level D2 adds another step that
handles the effect of prior likelihood of events for the decision
making. Then the final utilities of util(P) and util(P) arecomputed. The different steps of the proposed decision scheme
are explained below:
1) Raw Confidence of an Event Type: Let the CS application receive reports from different users from a common event
boundary z. Till the last time epoch, the reputation scoresof the users present within z are known. These reports may
or may not endorse the same event type. Let each particular
event type j receive Nagg (j) number of reports. Additionally,
let the aggregated reputation score of all users reporting thej th event type be denoted as Ragg (j). Let |U + (z)| be thetotal number of users currently present in z, each of themhas reputation Ri > 0. Thus, i∈U + (z) Ri denotes the total
reputation of all active users i ∈ U + (z). Therefore, for theN(j) Ragg (j)are the two evidences thatj th event, |Uagg+ (z)| andi∈U + (z) Ri
denote relative support for ‘quantity’ and ‘quality’ respectively.
Mathematically, we model the overall confidence on any event
type j (Cj ) using a weighted sum of quantity and quality
supporting each event type j.
Cj = ρ.

Nagg (j)+ (1 − ρ).
|U + (z)|Ragg (j)i∈U + (z) Ri(10)

where, 0 ≤ ρ ≤ 1 is the ‘preference factor’ associatedwith the evidence types, Nagg (j)/|U + )(z)| is the participation
fraction of event j. The preference factor ρ is controlled bycontextual information, such as spatio-temporal informationof a given type of event or risk policy. For example, if theevent occurs in a prior known inherently crowded location, theCS administrator may give higher weightage to the quantityN(j)support |Uagg+ (z)| . Conversely, for inherently sparsely crowded
locations where participation is lower, higher importance may
R(j). In the absence ofbe given to the quality support  aggi∈U + (z) Ri
any such information, ρ = 0.5.
2) Value Function of Event Confidence: This step embeds
the risk aversion policy into the event confidence. Now, weneed to determine the value of publishing event type j. Since
0 ≤ Cj ≤ 1, the midpoint where Cj = 0.5 is perceived

10

vmax = max(v(Cj ))
∀j∈E

(12)

In this level, if more than one event types yield the maximum
value, or if the largest value function is negative, then all thetypes are discarded and no event is published. The reason isthat the available evidence, in terms of the number of reports
and the aggregated reputation score supporting j th event, is
not sufficient to convincingly choose one particular winner
event type, and the uncertainty of publishing what still persists.
However, if we find only a specific event type with maximum
value, we move onto the second level. Formally, we define thefirst level decision process as:⎧
Select j,
iff vmax = v(Cj )⎪⎪⎨and vmax > 0
(13)
D1 =
⎪
⎪
⎩
No j is selected,
Otherwise4) Handling Bias of Prior Likelihood of Event Types:
Prior likelihood of any j occurring at a particular region andtime epoch biases decision making. In the second decision
D2, the CS administrator decides whether to publish a winner
event type j. However, if no j is chosen, D2 decision levelis not invoked. If we denote the occurrence likelihood of theevent type j to be the positive outcome, then its complement(i.e., non-occurrence) will be the negative outcome given a
publish decision. According to CPT, utility of a prospect isobtained by multiplying its value by a decision weight toobtain its utility [36]. These weights measure the impact of
events on desirability of the prospects, and not merely on theperceived likelihood and in general the decision probability
weighing functions should be concave near 0 and convex
near 1. Instead of one probability weighing function, we use

1
0.8

0.6

0.6
π

-

+

1
0.8

π

as the neutral reference point from the perspective of utility
value. Hence, all Cj > 0.5 and Cj < 0.5 instills the notion ofgains and losses respectively and accordingly perceived as lessrisky or more risky. Since a CS application is never absolutely
certain about the outcome, it can tune the interpretation of
Cj according to risk aversion policy or changing context.
In order to replicate a risk attitude while making decisions
under uncertainty, we again use the CPT value functions (CPT
property-2 in Section IV-C) for estimating the values:⎧
if Cj ≥ 0.5⎨ (Cj )θ2 ,(11)v(Cj ) =⎩−λ2 .(0.5 − Cj )φ2 ,if Cj < 0.5
The v(Cj ) ranges [−λ2 , 1], while the exponents θ2 > 0 and
0 < φ2 < 1 control the rates of growth of gains and loss
regions, respectively. The loss penalty parameter λ2 determines
the lowest value which v(Cj ) can attain. All λ2 > 1 imply risk
aversion.
3) First Level Decision (D1): At level D1 (refer to Fig.7a),the CS administrator needs to find the event type which yields
the maximum value. The event type with largest value is thewinner and is selected as a candidate for publishing. Thus:

0.4

0.4

0.2

0.2

0
0

0.2

0.4

0.6

0.8

1

0
0

0.2

0.4

p

0.6

0.8

1

(1-p)

(a) π + (pj ): Occurrence(b) π − (pj ): Non-Occurrence

Figure 8: Probability weighing functions
different decision probability weighing functions and theirsteepness of concavity and convexity are different for positive(prior likelihood of occurrence) and negative outcomes(complement of occurrence). Following equations give theweights for positive and negative outcomes:
π + (pj ) =π − (pj ) =

pδj 1
1

(pδj 1 + (1 − pj )δ1 ) δ1

,

(14)

1

(15)

(pj )δ2((pj )δ2 + (1 − pj )δ2 ) δ2

where pj and pj are likelihoods of the occurrences of event j
and its complement respectively, and 0 < δ1 , δ2 < 1 controls
the steepness of the decision weighing functions π+ (pj ) andπ − (pj ) respectively. The steepness determines how large π+
and π − will be for smaller values of pj and pj , and viceversa (see Figs. 8a and 8b). The weighing functions for
positive and negative outcomes are closer, although the former
is more curved than the latter (i.e., δ1 < δ2 ), such that strong
prior likelihoods underweigh the historical bias on the decisionfunction.
5) Final Utilities of Prospects: The sum of utilities to begained from publishing j is given as: util(P) = g1 ∗ v(Cj ) ∗π + (pj ) + l1 ∗ v(Cj ) ∗ π − (pj ). Similarly, the sum of utilities tobe gained from not publishing it is: util(P) = l2 ∗ v(1 − Cj ) ∗π + (pj ) + g2 ∗ v(1 − Cj ) ∗ π − (pj ). Here, v(1 − Cj ) evaluates
the value of discarding the confidence (Cj ) generated from theevidences received in the current time epoch.
6) Second Level Decision (D2): We formally define thesecond level decision as:⎧
if (util(P) − util(P)) > 0⎨ Publish j,
D2 =⎩Not Publish,
Otherwise(16)
VI. E XPERIMENTAL S TUDY
In this section, we evaluate the performance of QnQ andcompare them with Jøsang’s belief model [20] for expected
truthfulness (QoI), D-S model [42] for reputation scoring. We
also compare the performance of the proposed trustworthy
decision scheme against EUT.

11

A. Simulation Settings and Datasets
We simulated a realistic environment for vehicular crowdsensing system by extracting important simulation parametersfrom the Waze data set [5] and Epinions dataset [25]. The Waze
data comprises of reports for four major traffic event types: jam(ja), accident (ac), weather hazard (wh), and road closure (rc).
It has approximately 22,910 users, 71,505 reports, spanning
across 10 geographical regions adjacent to Boston, USA. In
each region, prior probabilities of occurrences (likelihood) ofdifferent event types have been computed from the dataset, assummarized in Table IV.
Table IV: Event Probabilities from Waze Dataset
Region
1
2
3
4
5
6
7
8
9
10

P(ja)
0.48
0.75
0.56
0.66
0.47
0.86
0.79
0.74
0.45
0.45

P(ac)
0
0.01
0.008
0.008
0
0.02
0.01
0.01
0.02
0.02

P(wh)
0.52
0.2
0.19
0.33
0.53
0.12
0.19
0.25
0.53
0.52

P(rc)
0
0.02
0.23
0
0
0
0
0
0
0

For simulation, we consider a city area of 20 X 20 sq.
miles as the region of interest. This area is partitioned into
ten rectangular grids to replicate regions from the dataset. The
system is initialized with U = 2400 number of active users,
among which Urp = 800 are reporters and Urt = 1600 are
raters. We extracted a realistic expected ratio of reporters toraters by studying an Epinions dataset collected from [25]since the Waze data did not offer this information. We assume
the presence of 520 dishonest devices out of which 120 (i.e.,
15% of total reporters) are used for generating false reports and400 (i.e., 25% of total raters) are used for false ratings. These
devices have been distributed uniformly in the simulated cityarea at the start of the simulation. The total simulation time isslotted into T = 240 number of epochs, each of which is ofduration 30 minutes.
We consider an event to have a fixed radius (5 miles) within
which all reporters and raters are liable to report or rate. Each
event has a tunable lifetime within which reports and feedbacks
are accepted. For example, if an event occurred in epoch t andthe duration of its lifetime is two epochs, then it can be reported
and rated until epoch t + 2. The probability of event type j ina particular region is extracted from Table IV.
We consider random paths along which a user moves withspeeds of 20-50 miles/epoch. We parameterize the number ofraters and ratings to account for all possible realistic combinations. However, we considered that users progressively leave
the region of interest mimicking a dense location becoming
sparse over time to capture effects of crowd movement.
For the reporters, we emulate honest, selfish, and malicious
behaviors in the following ways. 20% of the reporters areprogrammed as selfish, while 15% act as malicious and therest act honestly. Given that an event has occurred, an honest
reporter reports 99% of the time and has minuscule probabilityof generating a false report (simulating occasional wrong
perception). Malicious reporters within a randomly generated
location (chosen for false event) collude to generate fakereports of a fictitious event with high probability ≈ 100%.
One class of selfish users reports more true events (about 60%)
than false events, while the other class reports fewer true events(about 40%) than false events.
For the raters, the compromised raters give positive ratings tofalse events and negative ratings to true events, while the honest
raters provide genuine ratings with 5% legitimate uncertainty.
Note that a user reporting a particular event is prevented fromrating it. The percentages of compromised raters corresponding
to an event varies with the variation in the population size.
Since 400 out of 1600 raters are compromised, on average
the fake rating percentage for true and false events is about
25%. We have discussed its effects in the scalability analysis(see Section VI-F). Let the parameters of our system takethe following values: Ab = Au = 20, ν = 0.25, ϕ = 0.2,
Nthres = 60 and wumax = 0.5. Only the parameters Bb andBu are adjusted during runtime of QoI scoring. The growth rate
parameter is adjusted to Bb = Bu = 0.08 if low feedbacks arereceived for a particular event. For higher feedbacks received,
we keep Bb = Bu = 0.04. The value function parametersconsidered for simulation are θ1 = 2.5, φ1 = 0.6, and λ1 = 3.
An analytical study of effect of varying these parameters isprovided in Appendix D.
B. Expected Truthfulness (QoI) of Events
Fig. 9a illustrates a comparison between the expected truthfulness (QoI score) achieved by QnQ vs. Jøsang’s belief model
for a false event. We observe that QnQ refrains from giving an
undue high QoI score, unlike Jøsang’s model for low ratings.
As higher number of ratings are received, the confidence of thecrowd and the uncertainty discounting is taken into accountto converge to the true value, preventing malicious raters toharness an advantage. This is however not true for Jøsang’s
model, and false events end up getting higher scores even if
the number of ratings were smaller. In contrast, Fig. 9b shows
the the QoI score comparison for a true event. For QnQ, theQoI converges to the true value only after sufficient number ofratings are received, while for Jøsang’s model this aspect does
not matter. This is essential to prevent potential sabotaging by
an organized minority of rogue raters.
Note that QnQ will always assign low QoI to events receiving low feedbacks. When the number of ratings are limited,
there could be two possible options: (i) the published event
may not be significant enough and does not draw attention ofmajority of raters, resulting in low QoI and (ii) the place has aninherently low population, implying N is not very high. The
parameters Ab , Au , Bb , Bu and ν could be tuned to achieve
higher QoI score at comparatively lower number of ratings toadapt to contextual requirements (explained in Section VII-D).

20

40

60

80

100

# of Ratings

Score(a) False Event

1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-1
Qu120
ali 100

120

0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0

20

40

60

JosangQnQ
80 100

120

1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-1

0

1

Score

Josang
QnQ

Score

0.5
0.45
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
0

Exp. Truthfulness (τ)

Exp. Truthfulness (τ)

12

100

200

Malicious Users
Selfish Users
Honest Users
300 400 500 600

0.5
0
-0.5
-1
0

700

User ID# of Ratings

100 200 300 400 500 600 700

User ID(a) Proposed QnQ(b) True Event

Malicious Users
Selfish Users
Honest Users(b) D-S

Figure 11: Comparison: Reputation based User Classification

Figure 9: QoI Score Comparison
HonestSelfishMalicious
1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-1

ty 80
0
(Tr 6040 20
ue 40
80 60 vents)
Ev 20
100
ent 0160140120tity (Total Es)
Quan
Figure 10: User Reputation: Quality and Quantity

C. User Reputation Scores
We evaluate the performance of reputation scoring with agoal to unify quality and quantity, achieve fair classification ofuser behaviors, and compare with existing work.
1) Unifying Quality and Quantity: Fig. 10 shows how
QnQ is able to reflect both quantity (i.e., total number of events
participated) and quality (i.e., the number of events found to
be true) of participation in the resultant user reputation score.
The first observation is that three distinct user groups emerged.
The lowest group corresponds to malicious, the middle groupto selfish, and the top group to honest users. Another key
observation is that selfish and malicious users cannot increase
their reputation by boosting up only participation. Since selfish
users intermittently contribute true and false events, their scoresare higher than malicious but lower than honest users.
2) Classification of Users with Fairness: We considered
two different types of selfish users: (i) those who report
more true events than false events, and (ii) those who report
more false events than true events. Intuitively, selfish users
with higher number of genuine contributions should havehigher scores than others from the same class. However, it
is noteworthy that the penalty factor of reporting a fake event
is higher (λ1 = 2) than the reward for generating a true one.
Hence, the majority of both categories of selfish users end
up with negative score. This aspect is evident in Fig. 11a.
However, only few selfish users (around six out of 160) owing
to their participation in true events (with higher QoI) manage tohave positive scores. Likewise, very few honest and malicious
users end up having negative and positive scores, respectively.

These are the outliers to our user behavior classification.
Table V compares the reputation scores of various userclasses and their outliers. Here ni is the number of events forwhich user i has generated reports. Honest user #1 has very
low event participation compared to that of honest user #2, andhence has a lower score. Although selfish user #1 has reported
more true events than #2, both have reported a large numberof false events, leading to negative scores. Evidently, malicious
users (malicious #1 and #2) reporting majority of false events
have negative scores.
Our3) Comparison with Dempster Shafer (D-S) Model:
model exhibits better performance in terms of accuracy andfairness than D-S based reputation score as shown in Fig. 11b,
where many selfish users end up with very high scores.
Table V: Comparative Reputation Scores
Type
Honest #1
Honest #2
Selfish #1
Selfish #2
Malicious #1
Malicious #2

ni
3
100
41
37
4
102

True #
3
99
26
13
0
2

False #
0
1
15
24
4
100

Score
0.245
0.842
-0.346-0.462-0.299-0.919

D. Reducing Incentive Losses
QoI-aware incentive mechanisms account for quality of eachsensing report before making incentive/reward assignments.
The QoI metrics can be broadly classified into two categories: (i) reputation scoring based micropayments [19], (ii)satisfaction index-based involving data quality in terms ofsampling rate, accuracy, similarity, and timeliness [29], [35].
In particular, [29] proposes an Expectation Maximization (EM)algorithm to estimate "effort matrix" for the participants, which
captures the goodness of reports in terms of temporal proximityof the reported data with the time interval of ground truth
occurrence. A scalar function maps the effort matrix to a
QoI score which forms the basis of a reward mechanism that
achieves both individual rationality and profit maximization.
Beside this, game-theoretic (auction-based) incentive mechanisms exist but some of their limitations include: (a) Rationality of agents: Consider human users to be perfectly
rational agents and absence of malicious participants in thesensing task [17], [41]; (b) System/Computational inefficiency:
Consider incentivization as a maximum coverage problem
which is essentially NP-hard [44]. Therefore, the system andDempster-Shafer

E. Trustworthy Decision Making Accuracy

ProposedHonest Users

Dempster-Shafer

Selfish Users(a) Honest(b) Selfish

Figure 12: Protecting Undue Incentive Leakage
computational efficiency of the mechanism are not guaranteed;(c) Static tasks and users: Few works have assumed incentive
task and the number of users in the system to be static [12],[15]. However, for real mobile crowdsensing applications, suchassumptions may not hold.
Based on the above discussions, we argue the incentive
mechanism in mobile crowdsensing applications should havethe following features: (i) computationally inexpensive, (ii)dynamic, and (iii) maintain fairness. Dynamism entails thatover time the reward for users should change and fairness isguaranteed if participants are incentivized based on both thechanging quality and quantity of contributions. Incidentally,
the reputation score generated by QnQ encompasses these
attributes. Furthermore, consideration of a simpler reputation
score based reward function will ensure a lightweight incentive mechanism, suitable for real-time systems like vehicular
crowdsensing applications. Thus, we adopt the reputation based
incentive function presented in [32] as our choice. In [32], theincentive received by user i at the end of tth time epoch is:
Rt
Iit = U +it
k=1 Rk

.

B.U +
U

(17)

where Rit is the reputation score of user i as computed after
tth time epoch, U + is the number of users in the systemwith positive reputation score, B is the total incentive budget
allocated in time epoch t, and U is the total number of users inRj
the system. The fraction U +i j acts as a discounting factor
k=1 Rk+any user can gain.
to the maximum possible incentive B.U
U
Thus, the user with relative reputation on the higher side will
yield less discount and ends up getting handsome reward andvice versa. Fig. 12a shows that QnQ offers a larger variationof incentives disbursed to the honest users according to thevariations in quality and quantity. However, the D-S model
gives higher incentives since it only awards quality but not
quantity. Hence, users with lower participation also end up witha high score and hence a higher incentive. In contrast, Fig. 12bshows that mean QnQ-based incentives for selfish users is 50%
that of honest ones and is three times smaller than that yieldedby D-S based reputation model. Unlike D-S model, QnQ candistinguish between honest and selfish behaviors, and penalize
the latter with low rewards thus preventing loss of revenue due
to false contributions.

As mentioned, the simulator was run for 240 time epochs togenerate a history of occurrences of events, and a set of eligible
reporters with reputation score greater than 0. Following this,we again run the simulator for another 240 epochs to evaluate
the operational accuracy of the proposed decision scheme.
In practice, eligible reporters may get compromised at thecurrent time epoch (zero-day attack) or experience wrong
perception of a true event. Moreover, given that an eligible
reporter generates a false event, he generates the correct event
type with probability 1.
We evaluate the performance of our CPT-inspired decision
scheme against EUT-based model by computing (i) successrate, i.e., the fraction of true events successfully publishedamong of all true events that actually occurred (ii) detection
rate, i.e., fraction of true events published among all publishedevents. The objective of our decision scheme is to ensure thatrare events with sufficient quality and quantity support has ahigher chance of getting published.
1
0.8

1

CPT
EUT

Success rate

Proposed

2.6
2.4
2.2
2
1.8
1.6
1.4
1.2
1
0.8
0.6

Success rate

2.6
2.4
2.2
2
1.8
1.6
1.4
1.2
1
0.8
0.6
0.4

Incentives per user

Incentives per user

13

0.6
0.4

0.6
0.4
0.2

0.2
0

0.8

0.2

0.25

0.3

0.35

0.4

0.45

Likelihood of (rare) events(a) Success Rate vs pj

0.5

CPT
EUT
0
0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1

Frac. of genuine participation(b) Success Rate vs Fg (U + )Figure 13: Performance Evaluation of Decision Scheme
The values of various parameters are considered as follows:θ2 = φ2 = 0.88, λ2 = 2.25, ρ = 0.5, g1 = 2, g2 = 1,
and l1 = l2 = −1. For performance analysis of CPT versus
EUT, we analyze the success rates against the two following
parameters: (i) pj as the prior likelihood of occurrence of anevent (rare) and (ii) Fg (U + ) as the fraction of users reporting
correctly among all eligible users (with Ri > 0) (termed as
fraction of genuine participation).
Publishing Low Likelihood Events: Fig. 13a shows that thesuccess rates of the proposed CPT-based model is significantly
better than EUT for publishing true events whose prior likelihood of occurrences are very low (pj < 0.5). The reason
is CPT uplifts the likelihood of occurrences of rare events(less than 0.3), and thereby increases the publishing utilities.
This ensures that the rare events do not remain unpublished
if it generates higher confidence in the current time epoch.
To realize this scenario, we have considered the probability ofreporting the accurate event type for a true event as 0.75.
Fraction of Genuine Participation Fg (U + ): As shown in
Fig. 13b, CPT yields notable improvement in success rateover EUT for any fraction of genuine participation greaterthan 0.5. This is because, unlike EUT, CPT produces anenhanced value for the confidence of true events, and thereby
increases the number of published true events. Consequently,

F. Scalability and Robustness of Performance

0

Score
200

400

Malicious Users
Selfish Users
Honest Users
600
800 1000

1200

1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-1

0

200

Malicious Users
Selfish Users
Honest Users
600
800 1000

400

User ID

1200

User ID(a) Risk Averse(b) Risk Neutral

Figure 14: Scalability with U = 3600
-0.3
-0.5

Score

Score

-0.4
-0.6
-0.7
-0.8
-0.9
-1

0

50

100

Malicious Users
Selfish Users
Honest Users
150 200 250 300

350

1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-1

0

50

100

Malicious Users
Selfish Users
Honest Users
150 200 250 300

User ID

350

User ID(a) Risk Averse(b) Risk Neutral

Figure 15: Scalability with U = 1200
we study through Figs. 16a, 16b and 17b. Fig. 16a, shows theclassification accuracy when the attack budget is fixed (520fake devices) and the rating population varies, showing thatlarger crowd populations are more robust. On the other hand,Fig. 16b, shows the effect of varying attack budgets undera fixed population size of 3600. It is evident from Fig. 16b,that attackers need to control a sizeable number of the crowd(around 52%) for the classification to fail completely. However,
52% for a crowd of 3600 is about 1800 devices that attacker
needs to control which is very expensive.
Additionally, in most crowdsensing systems, the volume ofratings are always higher than reporters. We verified this claim
from Epinions dataset across different items in the dataset(See Fig. 17a above). Many other works have studied Yelp
and Amazon datasets and found similar observations. In mostparadigms including Waze, there is no incentive to provide
ratings on reviews/reports, but the volume of ratings remainhigh due to the relative ease of the pop-up feedback. Fig. 17b,
clearly shows that missed detection rates are minimal and false
alarm rates become miniscule when the raters are about 4times the reporters. In general as the ratio of raters to reporters
increases the performance only improves. To show conservative
results we have assumed a much lesser rater to reporter ratio
for most results.
Misclassification Rate (%)

Unlike all prior plots with total number of users U = 2400,
let us we consider U = 1200 and U = 3600 (fake devices
inclusive), under the same population 520 dishonest devices.
For U = 1200 (low population scenario), malicious users form
about 43% of the total population, and there is presence ofa very low proportion of genuine raters (only 45% of all
raters). Although the chances of such a scenario is rare, it
is still possible and some conservative systems may want tounderstand the performance limits of the defense model. Now
we examine the scalability in the context of risk aversion andrisk neutral attitudes. Fig. 15a shows the reputation scoresfor a risk averse system for U = 1200. Note that, we still
succeed to keep all malicious and selfish users in the lower
reputation tier with negative scores. Interestingly, our model
misclassifies all the honest users too due to the presence ofvery low proportion of genuine raters (only 45% of all raters).
Since our system follows a protective risk averse approach, thissacrifice is significant. For the risk neutral approach, Fig. 15bshows that we still manage to put all malicious and selfish users
in the lower reputation tier but the misclassification of honest
into the malicious tier is much smaller. However, as and when
the crowd increases (under U = 3600), the reputation of allhonest users are improved for both risk averse and risk neutral
approaches, reinforcing the significance of the crowd.
Fig. 14a is the reputation score distribution for the risk averse
system (with value function as link function) while Fig. 14b,shows the same for a risk neutral system (with classical logit
link function (used in our preliminary work)). Here, we seethat the risk aversion embedded by the value function is betterat keeping selfish users regardless of their subtype in the lower
reputation tier, while for risk neutral systems, all selfish users
with more true events than false events are in the positiveside on the reputation scale. For conservative systems, thesystem may want to keep all kind of selfish users from being
considered for any sort of decision making. Thus, it is evident
that the risk tolerance attitude is not only related to losses, gains
and uncertainty but also to the scalability aspect. We intend to
study in our future work, how a system can perform betterwith fewer misclassification of honest users than the current
case when the population is low, system is risk averse withbad mouthing attacks.
Classification accuracy is expressed in terms of whether anhonest user and dishonest user is accurately inferred or not. If
a legitimate selfish and malicious user is classified as honest
then it is a missed detection while if an honest user is classified
as anything else it is a false alarm. Missed detections andfalse alarms are an index of classification accuracy that is
affected by varying population sizes and attack budgets which

1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-1

25

100

False Alarm
Misdetection Rate

20
15
10
5
0
1200

1500

2400

3600

Varying Population Size(a) Fixed Attack Budget

Misclassification Rate (%)

the success rate for CPT gradually reaches 1 with increase in
the fraction of genuine participation. Similarly, a comparisonof the detection rates of our CPT inspired decision scheme isshown to outperform EUT in Appendix C.

Score

14

4800

80

False Alarm
Misdetection Rate

60
40
20
0
0.17

0.25

0.33

0.42

0.5

0.58

0.66

Perc. of Bad Raters(b) Varying Attack Budget

Figure 16: Robustness against Various Attack Budgets andRating Sample Sizes

# unique Reviews/Feedbacks

16000
14000

Reviews #Feebacks #

12000
10000
8000
6000
4000
2000
0

1 2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829

Object ID(a) Real Rating Distribution

Misclassification Rate (%)

15

1.4
1.2
1
0.8
0.6
0.4
0.2
02

Note that, this is a selfish user with more false events than true
events.

False Alarm
Misdetection Rate

C. Filtering of Rogue Raters
3

4

Ratio of Raters to Reporters(b) Varying Reporter-Rater Ratios

Figure 17: Additional Illustrations
VII. D ISCUSSIONSIn this section, we present various discussions on possible
extensions of QnQ and parameter recommendations undervarying assumptions of rating systems, risk attitudes, availability of trusted agents.
A. Extension to Multinomial or Real Valued Rating
The QnQ framework can be easily extended under crowdsensing systems with more than 3 rating levels or have realvalued ratings. For a 5-point rating mechanism viz., Bad,
Mediocre, Average, Good, Excellent (proposed in [22]), thelevels bad and mediocre may correspond to the disbelief mass,
while good and excellent form the belief mass, and averagecorrespond to the uncertainty mass. In IMDB like 10 pointgrading mechanisms levels 1-4 will correspond not useful ornegative feedbacks, 5,6,7 are uncertain and 8,9,10 are usefulor positive feedback categories. In case of real valued ratings,
a membership function may be used that discretizes the realvalued rating space into discrete rating levels.
B. Rationale of Two Selfish User Subtypes
The rationale of two sub-types of selfish users are inspired
from certain real world incidents. For example, in Los Angeles
California, news reports [45] surfaced that residents in aparticular uptown residential area generated fake reports suchthat Waze would not reroute traffic through their neighborhood.
Of course, fake reports are legitimately possible when suchusers are located in that particular area. Suppose an user goesout for work in the downtown area where she spends 10 hoursof her day. She could only false report during the evening when
her location is in this target uptown residential area. On theother times of the day, she has no selfish incentive to reportfalse events. Conversely, a user who works from or stays athome throughout; for her it makes complete sense to generate afake report on jam because these reports are auto-GPS stamped.
Occassionaly, when she goes out she does not produce fakereports since she does not any incentive to produce false reportat other locations. Thus for the same selfish objective in thesame area, different users could have different quantities oftrue and false reports. Conversely, there may be users whose
selfish objective is in terms of maximizing her incentives. In
such case, this user will often have the urge to report someevent, even if she is in a place that is relatively less eventful.

The report and rating are different roles that may be performed by the same physical user (interface). Over several
time epochs, this assignment of the malicious roles to physical interfaces have two possible alternatives. First, the same
physical (user) interface can act as a fake reporter on certain
time epochs and fake rater at other time epochs, dependingon their location/convenience. Second, each physical interface
controlled by an adversary has a fixed role that it only act as a
fake reporter or a fake rater across all time epochs. In the firstcase, the false rater is a role attached to the physical interface,
just like a false reporter. When our QnQ framework identifies
and isolates the false reporting users, they implicitly isolate
the fake raters too. Hence, a separate reputation mechanism is
not required under the first case. For the second case, where
each malicious interface has fixed roles; once the initial false
reporters in malicious team are identified by our reputation
mechanism, there will be no option left for the adversary but
to use some of the fake rater interfaces as fake reporters.
Else, there will be no interfaces left for fake events, hence
the purpose of having fake raters will be defeated. In such acase, our proposed mechanism will eventually detect all thesemalicious interfaces incrementally.
D. Parameter Choices under Risk Attitudes
Parameters need to be adjusted according to risk attitudes
for following functions: (i) coefficients of belief mass wb(Ab , Bb , ν), and (ii) coefficient of uncertainty mass wu(Au , Bu , Nthres , ϕ, wumax ). (iii) coefficients of value function (θ1 ,θ2 ,φ1 ,φ2 ,λ1 ,λ2 ), (iv) coefficients of weighing function(δ1 ,δ2 ). A provider could be risk averse in terms of the QoI and
reputation scoring by having a conservative increase of scores.
It could be risk neutral or risk seeking in terms of the QoI and
reputation scoring with more liberal increase of scores withevidence. Any instance of availability of trusted agents meansthat the provider may afford to lessen its risk aversion.
1) Choice of Ab , Bb and ν: The Ab is the base value ofthe weight given to belief mass when no rating is received. If
the system is not restrictive, then a higher initial weight wb
is required, and hence a lower value of Ab is recommended.
In contrast, for a conservative system, the initial weight of wb
should be very low, to ensure that it should acquire a sufficientnumber of ratings before attaining a substantial weight.
Fig. 3a shows the effect of Bb that controls the number ofratings N required to attain the maximum possible value of
wb once it enters the exponential phase. For example, if theconcerned area is inherently crowded and higher N is expected,
then Bb should be kept low such that the full weight to wb isawarded only after a sufficient number of ratings is received.
If the system is less restrictive, it can lower the value of Bb .

16

The parameter ν controls the value of N at which the curve
first enters into the exponential growth phase. A lower value ofν is preferred if the CS system expects receipt of false ratings,
or if the location historically receives lower number of ratings.
Fig. 3b shows the different values of ν. To conclude, the morerisk averse a provider, the smaller is the ν, the smaller is the
Bb and larger is the Ab , to be chosen in the belief coefficient.
This is because more evidence in terms of ratings are required
for wb to attain a higher weight which controls how positive
ratings contribute to the final trust values.
2) Choice of ϕ, Nthres and wumax : The Kohlrausch factor
ϕ determines how quickly wu discounting effect reaches minimum after Nthres is reached. Fig. 4a shows the effect ofvarious choices of ϕ. A CS system chooses a higher valueof ϕ if the proportion of uncertainty needs to be immediately
discounted or vice versa. Effects of Au and Bu to wu aresimilar to that of Ab and Bb to wb .
A small Nthres would prevent wu to reach its maximum
value, before the uncertainty discounting starts. This is true
for more conservative systems and is evident from Fig. 4b. A
low wumax may be required when the CS administrator comesto know about the ground truth (from other sources such as
mobile trusted participants [32]), and does not want uncertainty
mass to obtain higher weights. To conclude, the more risk
averse a provider, the smaller is the wumax and larger is theϕ parameter, and smaller Nthres . If the trusted agents, thenwe should have a smaller wmax and a smaller Nthres , andlarger ϕ.
3) Choice of θ1 , θ2 , φ1 , φ2 , λ1 , λ2 :Theparameters{θ1 , θ2 } < 1 gives us a risk seeking system, while {θ1 , θ2 } > 1
gives a risk averse system. A larger φ gives a risk seeking
system, while a smaller φ is risk averse. A larger λ is a
penalty factor that is high if the system is more risk averse.
The problem of being risk averse in scoring is that someusers may end up being demotivated due to lesser scores.
For example, if the provider already has a decent user base,{θ1 , θ2 } > 1 is recommended.
4) Choice of δ1 , δ2 : The parameters δ1 and δ2 control thecurvatures of weighing functions at their endpoints (finite
asymptotes). If most of the events in a region are non-recurringin nature, then intuitively high prior likelihoods should not be
accounted for decisions on event publishing. In such cases, ahigh δ1 values is required and lower δ2 is required. If a system
has events that show recurrence or periodicity, the δ1 value
should be lower and δ2 should be higher.
VIII. C ONCLUSIONSIn this work, we addressed the issue of quality of information
and reputation scoring in crowdsensing (i.e., vehicular CS
application) and propose a regression-based reputation model,
QnQ, which is resilient to rogue contributions and nullinvariance. The model assesses the QoI for a publishedevent by incorporating the cardinality of rating feedback,proportion of positive support, and uncertainty in ratings.

The QoIs of relevant events are aggregated to generate thefinal reputation score of a user. The resultant reputation
score provides a clear segregation among honest, selfish andmalicious users, and implicitly guarantees fairness withineach segregated group without sacrificing either participationor quality. Further, we propose CPT-based decision scheme
which takes the generated reputation score as input andsupports publish/not publish decisions, and implicitly ensures
operational reliability of the CS application. Extensive
analytical and simulation study was carried out to establish
the efficacy of the proposed approach in terms of scalability,
fairness, and decision accuracy. Finally, we present therecommendations on system parameters to enable QnQ adapt
under varying conditions of risk and uncertainty. In future,
we will study incentive and classification trade-offs under risk
averse and risk seeking systems for varying crowded locations.
Acknowledgements: The work has been supported by thefollowing NSF grants: CNS-1818942, CNS-1545037, CNS1545050, and DGE-1433659. A major portion of the work wascompleted at Missouri S & T, where Dr. Shameek Bhattacharjee was a post-doctoral fellow during 2015-2018, Dr. Nirnay
Ghosh was a post-doctoral research fellow during 2016-2017,
and Vijay Shah was a PhD student until August 2017.
R EFERENCES[1] H. Amintoosi and S. S. Kanhere, “A Trust-based Recruitment Framework for Multihop Social Participatory Sensing”, IEEE DCOSS, pp. 266-273, 2013.
[2] H. Amintoosi and S. S. Kanhere, “A Reputation Framework for Social Participatory
Sensing Systems”, Springer Mobile Networks and Applications, vol. 19, no. 1, pp.
88-100, 2014.
[3] H. Amintoosi, S. S. Kanhere, and M. Allahbakhsh, “Trust-based Privacy-aware
Participant Selection in Social Participatory Sensing”, Journal of Information
Security and Applications, vol. 20, pp. 11-25, 2015.
[4] R. S. Anderssen, S. A. Husain, and R. Loy, “The Kohlrausch Function: Properties
and Applications”, Anziam Journal, vol. 45, pp. 800-816, 2004.
[5] R. P. Barnwal, N. Ghosh, S. K. Ghosh, and S. K. Das, “Enhancing Reliability of
Vehicular Participatory Sensing Network: A Bayesian Approach”, IEEE SMARTCOMP, pp. 1-8, 2016.
[6] S. Bhattacharjee and M. Chatterjee, “Trust based Channel Preference in Cognitive
Radio Networks under Collaborative Selfish Attacks”, IEEE PIMRC, pp. 15021507, 2014.
[7] S. Bhattacharjee, N. Ghosh, V. K. Shah, and S. K. Das, “W2Q: A Dual Weighted
QoI Scoring Mechanism in Social Sensing using Community Confidence”, IEEE
PerCom Workshops, pp. 375-380, 2017.
[8] S. Bhattacharjee, N. Ghosh, V. K. Shah, and S. K. Das, “QnQ: A Reputation
Model for Securing Mobile Crowdsourcing Systems from Incentive Losses”, IEEE
Communications and Network Security, October 2017.
[9] J. A. Burke, D. Estrin, M. Hansen, A. Parker, N. Ramanathan, S. Reddy, and M. B.
Srivastava, “Participatory Sensing”, Center for Embedded Network Sensing, 2006.
[10] R. Cudeck and K.J. Klebe, “Multiphase Mixed-Effects Models for RepeatedMeasures Data" Psychological Methods, vol 7, no. 1, pp. 41-63, 2002.
[11] J.R. Eiser and M.P. White, “A Psychological Approach to Understanding how
Trust is Built and Lost in the Context of Risk",Conference on Social Contexts
and Responses to Risk, 2005.
[12] Z. Feng, Y. Zhu, Q. Zhang, H. Zhu, J. Yu, J. Cao, L.M. Ni, “Towards Truthful
Mechanisms for Mobile Crowdsourcing with Dynamic Smartphones" IEEE ICDCS,
pp. 11-20, 2014.
[13] S. Ganeriwal, L. K. Balzano, and M. B. Srivastava, “Reputation-based Frameworkfor High Integrity Sensor Networks” ACM Trans. on Sensor Networks, vol. 4, no.
3, pp. 15:1-15:37, 2008.
[14] R. K. Ganti, F. Ye and H. Lei, "Mobile crowdsensing: current state and futurechallenges," in IEEE Communications Magazine, vol. 49, no. 11, pp. 32-39,
November 2011.
[15] L. Gao, F. Hou, J. Huang “Providing Long-Term Participatory Incentive inParticipatory Sensing", IEEE INFOCOM, pp. 2803-2811, 2015.
[16] R. D. Gupta and D. Kundu, “Generalized Logistic distributions”, Journal of Applied
Statistical Science, vol. 18, no. 1, 2010.

17[17] B. Hoh, T. Yan, D. Ganesan, K. Tracton, T. Iwuchukwu, J. Lee. “TruCentive: A
game-theoretic incentive platform for trustworthy mobile crowdsourcing parking
services, Proc. of ITSC, pg. 160–166, 2012.
[18] K. L. Huang, S. S. Kanhere, and W. Hu,“Are You Contributing Trustworthy
Data?:The Case for a Reputation System in Participatory Sensing”, ACM MSWiM,
pp. 14-22, 2010.
[19] K. L. Huang, S. S. Kanhere, W. Hu, “On the Need for a Reputation System in
Mobile Phone based Sensing”, Elsevier Ad Hoc Networks, vol. 12, pp. 130-149,
2014.
[20] A. Jøsang,“An Algebra for Assessing Trust in Certification Chains”, NDSS, 1999.
[21] A. Jøsang and R. Ismail, “The Beta Reputation System”, Bled eConference, pp.
41-55, 2002.
[22] A. Jøsang, J. Haller, “Dirichlet Reputation Systems", IEEE Conf. on Availability,
Reliability and Security, pp. 112-119, 2017.
[23] D. Kahneman and A. Tversky, “Prospect Theory: An Analysis of Decision under
Risk”, Econometrica, pp. 263-291, 1979.
[24] I. Koutsopoulos, “Optimal Incentive-driven Design of Participatory Sensing Systems”, IEEE INFOCOM, pp. 1402-1410, 2013.
[25] P. Massa and P. Avesani, “Trust-Aware Bootstrapping of Recommender Systems”,
ECAI Workshop on Recommender Systems, pp. 29-33, 2006.
[26] P. Mohan, V. N. Padmanabhan, and R. Ramjee, “Nericell: Rich Monitoring ofRoad and Traffic Conditions using Mobile Smartphones”, ACM SenSys, pp. 323336, 2008.
[27] H. Mousa, S. Mokhtar, O. Hasan, O. Younes, M. Hadoud, and L. Brunie “TrustManagement and Reputation Systems in Mobile Participatory Sensing Applications:
A survey" Computer Networks, vol. 90, pp. 49-73, 2015.
[28] R. Neopolitan, “Learning Bayesian Networks", Prentice Hall, 2003.
[29] D. Peng, F. Wu and G. Chen, “Data Quality Guided Incentive Mechanism Design
for Crowdsensing," IEEE Trans. on Mobile Computing, vol. 17(2), pp. 307-319,
Feb. 2018.
[30] N. Ram, K. Grimm, “Handbook on Child Psychology, Developement Science andMethods: Growth Curve Modeling and Longitudinal Factor Analysis", Wiley, pp.
758-785, 2015.
[31] S. Reddy, D. Estrin, and M. Srivastava, “Recruitment Framework for Participatory
Sensing Data Collections”. International Conference on Pervasive Computing(Springer), pp. 138-155, 2010.
[32] F. Restuccia and S. K. Das, “FIDES: A Trust-based Framework for Secure User
Incentivization in Participatory Sensing”, IEEE WoWMoM, pp. 1-10, 2014.
[33] F. Richards, “A Flexible Growth Function for Empirical Use”, Journal of experimental Botany, vol. 10, no. 2, pp. 290-301, 1959.
[34] M. B. Sinai, N. Partush, S. Yadid, and E. Yahav, “Exploiting Social Navigation”,
arXiv preprint arXiv:1410.0151, 2014.
[35] C. Tham and T. Luo, “Quality of Contributed Service and Market Equilibrium for
Participatory Sensing" IEEE Trans. on Mobile Computing, Vol. 14(4), April 2015.
[36] A. Tversky and D. Kahneman, “Advances in Prospect Theory: Cumulative Representation of Uncertainty”, Journal of Risk and Uncertainty, vol. 5, no. 4, pp.
297-323, 1992.
[37] G. Wang, B. Wang, T. Wang, A. Nika, H. Zheng, and B. Y. Zhao, “Defending
Against Sybil Devices in Crowdsourced Mapping Services”, ACM MobiSys, pp.
179-191, 2016.
[38] X. O. Wang, W. Cheng, P. Mohapatra, and T. Abdelzaher, “Enabling Reputation andTrust in Privacy-Preserving Mobile Sensing”, IEEE Trans. on Mobile Computing,vol. 13, no. 12, pp. 2777-2790, 2014.
[39] Q. Xiang, J. Zhang, I. Nevat, and P. Zhang, “A Trust-based Mixture of GaussianProcesses for Robust Participatory Sensing”, ACM AAMAS, pp. 1760-1762, 2017.
[40] L. Xiao, Y. Li, G. Han, H. Dai and H. V. Poor, "A Secure Mobile Crowdsensing Game With Deep Reinforcement Learning," in IEEE Trans. on Information
Forensics and Security, vol. 13, no. 1, pp. 35-47, Jan. 2018.
[41] C. Yi, S. Huang and J. Cai, “An Incentive Mechanism Integrating Joint Power,
Channel and Link Management for Social-Aware D2D Content Sharing andProactive Caching," IEEE Trans. on Mobile Computing, vol. 17(4), pp. 789-802, 1
April 2018.
[42] B. Yu and M. P. Singh. “An Evidential Model of Distributed Reputation Management”, ACM AAMAS, pp. 294-301, 2002.
[43] R. Yu, R. Liu, X. Wang, and J. Cao, “Improving Data Quality with an Accumulated
Reputation Model in Participatory Sensing Systems”, Sensors Journal, vol. 14, no.
3, pp. 5573-5594, 2014.
[44] D. Zhao, X. Li, and H. Ma, “Budget-Feasible Online Incentive Mechanisms for
Crowdsourcing Tasks Truthfully" IEEE/ACM Transactions on Networking, Vol. 99,
2015.
[45] [Online] (2014, Nov.) Irate Homeowners Are Spoofing Waze, Available:
https://jalopnik.com/irate-homeowners-are-spoofing-waze-to-reroute-la-traff1660192011.
[46] [Online] (2015, Feb.) Sending fake data to Waze, Available:
https://www.yahoo.com/news/miami-cops-sending-fake-data-waze-stop-people210552132.html

Shameek Bhattacharjee is an Assistant Professor
at the Department of Computer Science at WesternMichigan University, USA. He received his Ph.D. and
M.S. from the University of Central Florida, Orlando
in 2015 and 2011 respectively and his B.Tech from
West Bengal University of Technology, India, 2009.
Between 2015-2018, he worked as a post-doctoral
researcher at the Missouri S & T at Rolla, MO, USA,
where he is also affiliated as an Adjunct Faculty. His
current research interests include information security in cyber-physical systems, wireless and social
networks, particularly in topics such as anomaly detection, trust models, secure
crowd-sensing, and dependable decision theory. He is a receipient of Provost
Fellowship and IEEE PIMRC Best Paper Award.
Nirnay Ghosh is a research fellow at the iTrust
Research Center for Cyber Security, in the Singapore University of Technology and Design (SUTD).
Prior to this, he worked as a postdoctoral fellow
at the Missouri S & T (MST), Rolla, USA. Dr.
Ghosh completed his masters (MS-by research) and
PhD from the Department of Computer Scienceand Engineering, IIT Kharagpur, in 2010 and 2016,
respectively. His research interest and experience
include graph theory applications to security, secure
cloud computing, participatory sensing, and internetof-things. He is receipient of TCS Research Fellowship and IEEE ADCOMBest Paper Award.
Vijay K. Shah is a Ph.D. student at the Departmentof Computer Science, University of Kentucky, USA.
From Jan. 2015-Aug. 2017, he was a PhD student at
Missouri S & T. His Ph.D. advisors are Dr. Simone
Silvestri and Dr. Sajal K. Das. He has authored/coauthored over a dozen scientific papers in top-tier
networking journals and conferences, including ACM
TOSN, IEEE TMC, ACM BuildSys, IEEE INFOCOM, and IEEE CNS. His research interests primarily include Wireless Networked Systems, Spectrum
Sharing, 5G Networks, Smart Cities, Internet ofThings.
Sajal K. Das is a professor of Computer Scienceand the Daniel St. Clair Endowed Chair at the Missouri University of Science and Technology, where
he was the Chair of Computer Science Dept. during 2013-2017. His research interests include cyberphysical security and trustworthiness, wireless sensor
networks, mobile and pervasive computing, crowdsensing, cyber-physical systems and IoTs, smart environments (e.g., smart city, smart grid and smart
health care), cloud computing, biological and social
networks, applied graph theory and game theory. He
has published over 700 research articles in high quality journals and refereed
conference proceedings. Dr. Das holds 5 US patents and coauthored 4 books– Smart Environments: Technology, Protocols, and Applications (John Wiley,2005), Handbook on Securing Cyber-Physical Critical Infrastructure: Foundations and Challenges (Morgan Kauffman, 2012), Mobile Agents in Distributed
Computing and Networking (Wiley, 2012), and Principles of Cyber-Physical
Systems: An Interdisciplinary Approach (Cambridge University Press, 2018).
His h-index is 83 with more than 28,000 citations according to Google Scholar.
He is a recipient of 10 Best Paper Awards at prestigious conferences like ACM
MobiCom and IEEE PerCom, and numerous awards for teaching, mentoring
and research including the IEEE Computer Society’s Technical Achievement
Award for pioneering contributions to sensor networks and mobile computing,and University of Missouri System President’s Award for Sustained Career
Excellence. He serves as the founding Editor-in-Chief of Elsevier’s Pervasiveand Mobile Computing Journal, and as Associate Editor of several journals
including the IEEE Transactions on Dependable and Secure Computing,IEEE Transactions on Mobile Computing, and ACM Transactions on Sensor
Networks. Dr. Das is an IEEE Fellow.

Appendix for:
QnQ: A Quality and Quantity Unified Approach for Secure andTrustworthy Crowdsensing
1

Appendix A

2

Success Rate

Expected Bayesian belief (E J ) is given as: E J = b + a.u, 3
Appendix C
r+1
s+1
t+1
where b = r+s+t+3; d = r+s+t+3; u = r+s+t+3; and r, s,
and t denote the number of positive, negative, and uncer- The following Fig. 1, shows that our better performancetain ratings. The value a = 0.5 is the relative atomicity during decision making is preserved over time compared towhich is equal to the reciprocal of the cardinality of infer- the EUT performance.
ence state space {true, f alse}.
1

Appendix B

0.9
0.8

0.7
D-S based reputation model uses ternary feedback-based
evidence space. For a particular agent, each witness as0.6
CPT
EUT
signs probability values to the three elements (viz. trust0.5
120 140 160 180 200 220 240 260
ful, distrustful, uncertain) in the state space, thus forming
Time Epochs
the belief, disbelief, and uncertainty masses, respectively.
The probability masses obtained Lfrom different sources are
Figure 1: Success Rate vs Time Epochs
combined by an orthogonal sum ( ) operator and then the Figure 2: Decision Model Comparison over Time Epochs
difference between overall belief and disbelief masses gives
the agent’s reputation score. The orthogonal sum operator 4
Appendix D
is defined as follows:
Let Bel1 and Bel2 be the belief functions of two sources The generalized logistic curve is parameterized by a variover a frame of discernment Θ, with the belief mass assign- able N , is given as:
ments m1 and m2 respectively. If the focal elements be
U −Lgiven as x1 , x2 , . . . , xk and y1 , y2 , . . . , yl respectively,(2)w(N ) = L +L then−BN )1/ν
1,2
1
2
(1
+
A.e
the combined belief mass function mΘ = mΘ mΘ :2|Θ| 7→ [0, 1] is defined by:where, L is the lower asymptote, U is the upper asymptote,
m1,2 (Φ) = 0 P
B is the growth rate, ν is the parameter that controls them1 (xi ).m2 (yj )value of N at which the curve enters the exponential growthLi,j,xi ∩yj =xP
m1,2 (x) = m1 (x) m2 (x) = 1−phase, and A is the initial value of the coefficient at w(0).
m1 (xi ).m2 (yj )i,j,xi ∩yj =φ

5

As the underlying principle of both QnQ and D-S reputation model is based on classical Bayesian, we considered
the latter for fair comparison. Mathematically, Dempstershafer theory proposes an orthogonal sum operator ⊕ which
combines evidences from multiple sources to generate cumulative belief, disbelief and uncertainty functions.
For example, if Bel1 Bel2 · · · are the belief functions obtained from different
L sources,
L the cumulative belief is given
as: Bel(i) = Bel1 Bel2 ·L· · and the L
cumulative disbelief is: DisBel(i) = DisBel1 DisBel2 · · ·Then, the reputation of agent i will be given by
Rep(Ai ) = Bel(Ai ) − DisBel(Ai )Appendix E

In this section, we compare the QoI score generated by
QnQ with Jøsang’s expected truthfulness (E J ), which isequivalent to τk in our approach (because the scale of both
metrics have to be between 0 and 1 for fair comparison).
To better explain, the five important aspects of QoI
and reputation scoring in QnQ, we consider two different
scenarios: (i)sparsely-crowded location, and (ii) denselycrowded location. Let the total number of ratings received
be 50 and 300 for the sparse and the dense locations respectively. The event published by the CS administrator basedon the reports can be either true or false. Thus, there could(1)1Table 2: Case-b: False Event at a Dense Location
be four possible scenarios: (a) false event/sparse location,(b) false event/dense location, (c) true event/sparse location, and (d) true event/dense location. The densities keepchanging all the time. Under the four cases, let the adversary have a budget to manage a fixed number of raters (say20), in every location, who can pose the following four typesof threats: (i) false ratings to true events (bad mouthing);(ii) true ratings to false events (ballot stuffing); (iii) deliberate undecided ratings to boost up expected truthfulnessof false events (obfuscation stuffing); and (iv) combine false
and undecided ratings (mixed attack). Let the parametersof our system take the following values: Ab = Au = 20,
ν = 0.25, ϕ = 0.2, Nthres = 60 and wumax = 0.5. Only
the parameters Bb and Bu are adjusted in the runtime of
QoI scoring. For cases (a) and (c), which correspond toa sparse location that has a low number of feedbacks, thegrowth rate parameter is adjusted Bb = Bu = 0.08. For
cases (b) and (d), which correspond to a dense location,
Bb = Bu = 0.04. Note that the only difference is in thegrowth parameter B which mostly depends on the expectation that more ratings will be available over time.
For cases (a) and (b), where the event is false, we do
not consider the bad mouthing attack as it is not relevant,as such attacks are meant to manipulate a true events into
false by deliberate fake (negative) ratings. We represent the
rating distributions for each threat barring bad mouthing
by a tuple hr, s, ti. The expected truthfulness scores generated by Jøsang’s model and QnQ are depicted in Table
1. It is clearly evident from Table 1, for sparse location,
Table 1: Case-a: False Event at a Sparse Location
ThreatNo threat
Ballot stuffing
Obfuscation
Mixedhr, s, ti
h5, 42, 3ih25, 22, 3ib/d/uJøsang QnQ
0.11/0.81/0.075 0.128
0.036
0.49/0.43/0.07 0.525
0.147

h5, 22, 23ih15, 22, 13i

0.11/0.43/0.45 0.335
0.3/0.43/0.26 0.43

ThreatNo threat
Ballot stuffing
Obfuscation
Mixedhr, s, tih25, 260, 15ih45, 240, 15ib/d/u
Jøsang QnQ
0.085/0.86/0.052 0.111
0.087
0.15/0.79/0.052 0.176
0.15h25, 240, 35i
h35, 240, 25i

0.085/0.79/0.12 0.205
0.12/0.79/0.085 0.163

0.09
0.12

ues computed by the two models in these two cases. For
Table 3: Case-c: True event at a Sparse Location
ThreatNo threatBadmouthing
Obfuscation
Mixedhr, s, tih30, 15, 5ih30, 35, 5ib/d/u
0.58/0.3/0.11
0.42/0.49/0.08

Jøsang QnQ
0.635
0.177
0.46
0.128

h30, 15, 25ih30, 25, 15i

0.42/0.22/0.36
0.42/0.36/0.22

0.6
0.53

0.156
0.164

case (c), it is clearly evident that Jøsang’s model assigns
high QoI even . However, QnQ refrains from assigninghigher truthfulness value even to true events unless it receives a substantial number of ratings. The truthfulness
value given by QnQ is lowest under bad mouthing attack, which shows that our model is less robust if rogue
raters give deliberate false ratings to true events. The values of the coefficients computed here are wb = 0.28 and
wu = 0.14(normal), 0.2(other threats). Unlike the other
Table 4: Case-d: True event at a dense location
ThreatNo threatBadmouthing
Obfuscation
Mixed

0.093
0.12

hr, s, tib/d/u
Jøsang QnQ
h180, 90, 30i 0.597/0.3/0.102 0.648
0.596h180, 110, 30i 0.56/0.34/0.09 0.605
0.558h180, 90, 50i 0.56/0.28/0.16
h180, 100, 40i 0.56/0.31/0.13

0.64
0.625

0.562
0.56

three cases, the truthfulness scores assigned in (d) by bothmodels are on the higher side and are at par with eachother. This is because, the event is true and substantial
the truthfulness score assigned by QnQ to false events are
ratings have been received, which has lead to the generamuch less compared to that given by the Jøsang’s model.
tion of high scores.
Moreover, our model can readily detect the obfuscation atSummarizing the results depicted in Tables 1, 2, 3, and 4,
tack, assigning it lowest value.
we draw three important observations: (i) QnQ is resilient
For case (b), the truthfulness comparison is presented to ballot stuffing attack by preventing false events to be
in Table 2. If the location is densely populated, the QoI portrayed as a true ones for both low and high number ofscore assigned by Jøsang’s model is relatively less compared ratings; (ii) unlike Jøsang’s model, QnQ is completely nullto sparse locations. However, it is still on the higher side invariant and thwarts the threat of obfuscation; and (iii) thecompared to the scores generated by QnQ, which computed proposed model is robust against bad mouthing attacks, if
the coefficients as wb = 0.99 and wu = 0.05. Like (A), here substantial number of ratings are available.
also QnQ is particularly able to be severe on obfuscation
attack.
For cases (c) and (d), as the event is true, ballot stuffing
attack is not practical since that is meant to manipulate
a false event to true by deliberate fake (positive) ratings.
Tables 3 and 4 give the comparison of truthfulness val2