The 8th International Workshop on Information Quality and Quality of Service for Pervasive Computing 2017

W2Q: A Dual Weighted QoI Scoring Mechanism inSocial Sensing using Community Confidence
Shameek Bhattacharjee, Nirnay Ghosh, Vijay K. Shah, and Sajal K. Das
Department of Computer Science,
Missouri University of Science and Technology, Rolla, MO
Email: {shameek, ghoshn, vksr38, sdas}@mst.edu.

Abstract—A significant vulnerability in social sensing based
services is false notifications from sensing agents, thereby resulting in inaccurate published information that induces loss ofrevenue and business goodwill. Existing popular schemes utilize
rating feedbacks (over the published information) to quantify theperceived usefulness (quality) of the information. However, theseschemes do not reward the confidence of the feedback communityand lacks provision to regulate the impact of uncertain feedbacks(ratings), and hence can be easily manipulated. In this paper, we
propose a model, called 𝑊 2𝑄, to mathematically evaluate theQuality of Information (QoI) as a function of the proportionof positive ratings, total number of ratings, and amortizedproportion of uncertain ratings. The proposed model exploits
Bayesian inference, and a dual weighted regression model tocompute the QoI of any published information. We evaluate theproposed model through an experimental study assuming a crowd
sourced-urban application as a proof of concept. Experimental
results show that compared with the state-of-the-art Jøsang’s
belief model, the resultant QoI score is less susceptible to rogue
ratings and captures subtle differences between true and false
information.
Index Terms—Trust, Crowd Sourcing, Pervasive Computing,Information Quality

I. I NTRODUCTION
Recent years have witnessed a splurge in commercial services leveraging “social sensing” paradigm to publish summarized statistics (information). The sensing agents in thisparadigm could either be automatic sensors, called auto sensing (AS), or voluntarily participating human users, called
crowd sensing (CS), who use their smart mobile devices
to send notifications. Each sensing agent monitors its local
environment and sends alert message to a remote serverwhenever it perceives an event of interest. Such notifications(possibly of different modalities) sent by either AS or CS are
aggregated by the service provider to generate a summarized
information. This information is then broadcasted to otherusers (consumers) through different media to support better
decision making and thus, improve the quality of service.
One prominent example of AS-based social sensing is the
Total Traffic & Weather Network (TTWN)1 which has developed massive sensing infrastructures in major cities across the
USA to capture traffic, transit, and weather data. Commuters,
drivers, and other users access such information in real-timefrom the Twitter accounts maintained by TTWN. Followersof those accounts are eligible to like, share, reply or retweet
1 http://www.ttwnetwork.com/

978-1-5090-4338-5/17/$31.00 ©2017 IEEEagainst the generated messages. Similarly, some CS-based
applications worth mentioning are Waze2 and Nericell, which
receive notifications on road conditions to improve navigation
experiences through dynamic route planning. Other CS-based
application like FourSquare, Yelp, and YikYak recommend ideal
point of interests in geographical proximity.
A major drawback with both the social sensing mechanisms
is the generation of false notifications. In case of AS, incorrect
data may be generated if one or more sensing devices becomefaulty or are compromised by adversaries. For example, a false
traffic jam or weather hazard may be notified, making theservice provider believe that such event has really occurred. If
the provider persists with incorrect (or false) information for
longer periods of time, the consequences may be serious.
In contrast, CS allows human users (contributors) to voluntarily participate (typically in exchange of incentives) andgenerate notifications on perceived incident. However, the“open” nature of the CS paradigm exposes the applications
to erroneous and dishonest contributions [4]. Several CS
applications use credit based incentive schemes [11] [10] tomotivate users to be active participants and sustain a steadyflow of contributions. In most incentive schemes, the degree ofparticipation (i.e., how much the users contribute), rather thanthe correctness of the submitted data, is used as the criterion
for rewards. Consequently, dishonest users misuse such reward
mechanisms to generate false notifications and gain unfair
incentives [13]. In many instances a group of sensing agents
may collude [15] to make the false information believable tothe provider. If the false published information is not detected
quickly, consequent impacts could be drastic [14].
In [1], the authors have studied real data sets of Waze
application which confirms the presence of bogus notifications
and their negative impact on the incentive mechanism. Thus,
irrespective of reasons for generating false notifications in both
AS and CS, loss of revenue and trustworthiness are inflicted on
the service provider. Therefore, there is a need to estimate thequality of information (QoI) generated by the sensing agents.
Some existing literature [4], [13] dealing with real systems
use the rating feedback paradigm to gain an estimated perception of truthfulness or QoI of a published information. The
underlying trust models [6], [7] measure QoI as the fractionof positive ratings. However, these approaches do not account
2 www.waze.com

The 8th International Workshop on Information Quality and Quality of Service for Pervasive Computing 2017

for the confidence of the community (total number of ratings),
null invariance (high uncertainty or inconclusive association
should not be allowed to influence correlations), and fractionof uncertain ratings that may negatively influence the QoI.
In this paper, we propose a novel QoI scoring model, termed
as 𝑊 2𝑄, which unifies both the proportion of positive ratings
and the confidence of the community, while accommodating
the provision to regulate uncertainty using amortized proportion of uncertain ratings. Consideration of both of these
attributes nullifies the effect of possible rogue ratings on theexisting QoI measures and sparse ratings. We use Bayesianinference to calculate the probability masses for different
rating categories viz., Useful, Not Useful, Not Sure. Then we
calculate an expected truthfulness of the information based on
regression with dual weights that capture the number of ratings
and the amortized proportion of uncertainty mass dependingon the risk attitude of the provider. The expected truthfulness ismapped into a QoI using a link function. Experimental results
establish that compared to Jøsang’s belief model, the resultant
QoI score is less susceptible to rogue ratings and capturessubtle differences between true and false information.
The rest of the paper is organized as follows. Section II
presents related works and their limitations to draw motivations. Section III describes the system and threat models. Section IV presents the proposed QoI scoring model. Extensive
simulation results are presented in Section V while conclusions
are drawn in Section VI.
II. R ELATED W ORK AND M OTIVATION
Existing works on quality of information (QoI) in social
sensing paradigms primarily deals with its improvement. The
following literature review is primarily based on CS paradigm.
The QoI improvement has been mainly attempted throughdesign of incentive mechanisms and selection of sensing
agents. The primary objective of incentive mechanisms is tomotivate the users to contribute with correct sensing data. It
has two variants: online and offline. For online type, the system
is trained by the first batch of participants, and tested on thesubsequent batches [18], [19]. Conversely, for offline type,the incentivization system receives apriori information about
the participants, and then uses reverse auction mechanism forbidding and winner selection [5], [3]. Selection of humansensing agents is based on their location proximity to theevent [3].
The fundamental differences between our proposed QoIscoring model, 𝑊 2𝑄 and existing works are as follows. First,
many existing works such as [18], [19] focus on the selectionof trusted sensing agents, and incentives mechanisms that
motivate participation to improve the QoI. However, once
agents had generated the notifications, assessing the extent of
truthfulness of the summarized information is largely missing.
In contrast, 𝑊 2𝑄 focuses on quantifying the QoI, given thepossibilities of false notifications from sensing agents. Secondand most importantly, our work proposes a mathematicalquantification of QoI that ameliorates the following limitations

existing in the state-of-the-art (Jøsang’s belief and its variants)
approaches of evaluating QoI from the ratings:
Confidence of Rating Community: Jøsang’s belief model,
and its variants such as Beta reputation [8] uses several
variants of the expected ratio between number of positive
ratings to the total number of ratings as the QoI metric.
However, such models do not capture a fundamental aspect
which we term as confidence of the community. This feature, if
not incorporated makes QoI vulnerable to manipulation when
dishonest raters provide positive ratings to useless and bogus
information (known as ballot stuffing) and negative ratings totrue or authentic information (known as bad mouthing).
This is illustrated in Table I. Each usecase in the table isrepresented as a four tuple 𝑈 : ⟨𝑁, 𝑟, 𝑠, 𝑡⟩, where 𝑁 is thetotal number of received ratings, while 𝑟, 𝑠, and 𝑡 respectively
denote the number of positive, negative, and uncertain ratings.
The details of Josang’s QoI model is provided in Section V-A.
The first row in Table I shows that usecase 𝑈 1 has 4
good ratings out of 8 total ratings, whereas 𝑈 2 has 40 suchratings out of 80. Jøsang’s belief model measures almost same
QoI for both usecases. From an adversary’s viewpoint, it iseasy to manipulate 4 good ratings by compromising 4 raters
and maintain the same fraction of positive ratings as 𝑈 2.
However, it is harder to maintain the same when the crowd islarge (as in 𝑈 2), where the adversary has to compromise 40
raters. Hence, given the same fraction of positive ratings, theinformation with a higher number of ratings (i.e., confidenceof the community) should be much more trustworthy.
Controlling the Effect of Uncertainty: As aforestated, the
QoI of a false information can also be increased due to many
undecided ratings which may be deliberately faked or a resultof legitimate uncertainty in the rating feedback mechanism.
Jøsang’s belief model or closely related Dempster Shafer based
reputation cannot differentiate between scenarios with highand low uncertainties. An illustrative example is shown insecond row of Table I. Usecase 𝑈 3 has 105 ratings out of
which 100 are uncertain; however it achieves almost same QoI
as 𝑈 4 which in contrast has only 20 uncertain ratings. Hence,
this QoI measure suffers from null-invariance. For certain
conservative services, it may be risky to give such a high QoI
to 𝑈 3, as opposed to 𝑈 4. Thus, the QoI measure needs aprovision for controlling the impact of higher uncertainty on
the QoI.
III. S YSTEM AND T HREAT M ODELS
A. System Model and Design
As depicted in Figure 1, our system model captures a particular urban area which may consist of both auto-configured
sensors and smartphone users, to collect various sensing data
viz., air and sound pollution, traffic, and so on. Two importantcomponents of this system are:Notification: A notification is an alert generated by a sensing
agent (sensor or human) in response to its perception of
an incident of relevance. However, due to malfunctioning or
adversarial attack on one or more sensing devices, or presenceof dishonest users, there may be false notifications.

The 8th International Workshop on Information Quality and Quality of Service for Pervasive Computing 2017

TABLE IL IMITATIONS OF J ØSANG ’ S B ELIEF M ODEL

IssuesConfidence of Rating Community
Not Null Invariant

Use Cases
U1: ⟨8, 4, 2, 2⟩
U2: ⟨80, 40, 20, 20⟩
U3:⟨105, 5, 0, 100⟩
U4:⟨25, 5, 0, 20⟩

Jøsang QoI
0.59
0.61
0.51
0.53

Information: An information 𝐼𝑘 , 𝑘 ∈ {1, ⋅ ⋅ ⋅ , 𝐾} is a message
which is published in some form (e.g., on live map) after theservice provider receives a pre-defined number of “similar”
notifications. 𝐾 is the total number of such information
published over a given time span.
In the system model, there exists two types of actors:
Sensing agent: A sensing agent is either a sensor or humanuser who has a proclivity to generate notifications. Furthermore, to remove biases from the ratings, a sensing agent is
not allowed to rate a published information for which it has
generated a notification.
Rater: A rater is a human user who provides a rating basedon his utility of a published information through one of thesecategories: Useful (𝛼), Not Useful (𝛽), and Not Sure (𝛾).
For a published information, only one rating can be submitted by each rater. We consider the act of providing rating as a
choice, and there is no mechanism to incentivize users. Hence,for majority of the neutral users, there is no selfish incentive
to provide false ratings. However, raters could be motivatedin a malicious manner as discussed in the threat model in thenext subsection. In general, we consider the number of ratings
to be higher than the number of notifications which is also
evident from the Epinions dataset [12].
The primary design principle of the proposed 𝑊 2𝑄 scoringmodel should be to collect higher number of ratings to get anotion about the truthfulness of the information in absence ofthe ground truth. This may be achieved through auto-activationof a pop-up rating query during an active rater’s transit throughthe vicinity of the published information.

Comment
No significant difference in expected truthfulness. Does notaward confidence of the community
Expected truthfulness is fairly high in 𝑈 3even though majority ratings are “undecided”

behave dishonestly in the following ways: (i) intermittently
act as malicious agents to gain undue incentives, and (ii)persistently act as dishonest agents, either through collusion orby generating several fake logical interfaces (sybils) to tarnish
the goodwill of the application.
For raters, we assume that they can also collude among
themselves to provide false ratings as follow:
Bad mouthing: The rater provides negative ratings to an authentic published information.
Ballot stuffing: This attack arises when the rater provides
positive ratings to an incorrect (false) published information.
IV. 𝑊 2𝑄: P ROPOSED Q O I S CORING M ODELThe proposed QoI scoring model consists of three major
modules. The first module derives the posterior probability
masses for each rating category. The second module calculates the expected truthfulness of a published information
by estimating appropriate weights (coefficients) of belief anduncertainty masses using logistic and stretched exponentialfunctions, respectively. Finally, the third module calculates ofa real valued QoI for each published information using a logit
link function. A flow diagram of the different modules alongwith the inputs and outputs are shown in Figure 2.

Fig. 2.

𝑊 2𝑄: Flow Diagram

A. Derivation of Posteriori Probability Masses

Fig. 1.

System Model

B. Threat Model and Assumptions
As mentioned earlier, sensing agents can be either autoconfigured sensors (auto sensing) or human users (crowd
sensing). The sensors can generate false notifications either
due to hardware/software fault or adversarial manipulations.
If one or more sensors are compromised, they can be madeto act dishonestly. On the contrary, human users may also

The first step is to derive the posteriori probability masses
associated with the published information being ‘Useful’, ‘Not
Useful’ and ‘Not Sure’. The probability masses are calculated
for each information 𝐼𝑘 . For better readability, 𝐼𝑘 is removed
from all the notations. The posteriori probability mass of eachrating category should be based on the available evidence (i.e.,
support for each rating type). A classical Bayesian approachhas been considered for estimation of probability masses.
Let 𝜃¯ = {𝜃𝛼 , 𝜃𝛽 , 𝜃𝛾 } be the three tuple probability parameter to be estimated. Here, 𝜃𝛼 , 𝜃𝛽 , 𝜃𝛾 are the unknown
probabilities of observing a Useful, Not Useful, or Not Sure¯ as the hypothesis, suchratings, respectively. We denote 𝐻(𝜃)that it has three possibilities corresponding to 𝛼, 𝛽, and 𝛾.
Formally the corresponding probabilities are given by:

The 8th International Workshop on Information Quality and Quality of Service for Pervasive Computing 2017

¯ = 𝛼∣𝜃)¯ = 𝜃𝛼𝑃 (𝐻(𝜃)¯¯ = 𝜃𝛽𝑃 (𝐻(𝜃) = 𝛽∣𝜃)¯ = 𝛾∣𝜃)¯ = 𝜃𝛾𝑃 (𝐻(𝜃)(1)

Let 𝐹𝛼 , 𝐹𝛽 , and 𝐹𝛾 be the random variables denoting thenumber of ratings 𝜂𝛼 , 𝜂𝛽 , and 𝜂𝛾 , received for each possible
rating category, respectively, such that 𝑁 = 𝜂𝛼 + 𝜂𝛽 + 𝜂𝛾 . The
evidence vector, denoted as 𝐹 (𝑁 ) = {𝐹𝛼 , 𝐹𝛽 , 𝐹𝛾 }, should bemodeled as a multinomial distribution given by:¯ =𝑃 (𝐹𝛼 = 𝑛𝛼 , 𝐹𝛽 = 𝑛𝛽 , 𝐹𝛾 = 𝑛𝛾 ∣𝜃)
𝑁
!
𝑛
¯ =
𝜃𝑛𝛼 𝜃 𝛽 𝜃𝑛𝛾
𝑃 (𝐹 (𝑁 )∣𝜃)𝑛𝛼 !𝑛𝛽 !𝑛𝛾 ! 𝛼 𝛽 𝛾(2)The posteriori hypothesis based on the evidence vector andassumed prior is given as:¯¯ = 𝛼∣𝐹 (𝑁 )) = 𝑃 (𝐻(𝜃) = 𝛼, 𝐹 (𝑁 ))𝑃 (𝐻(𝜃)𝑃 (𝐹 (𝑁 ))¯ = 𝛽, 𝐹 (𝑁 ))𝑃(𝐻(𝜃)¯ = 𝛽∣𝐹 (𝑁 )) =(3)
𝑃 (𝐻(𝜃)𝑃 (𝐹 (𝑁 ))¯¯ = 𝛾∣𝐹 (𝑁 )) = 𝑃 (𝐻(𝜃) = 𝛾, 𝐹 (𝑁 ))𝑃 (𝐻(𝜃))𝑃 (𝐹 (𝑁 ))First, we calculate¯¯ = 𝛼∣𝐹 (𝑁 )) = 𝑃 (𝐻(𝜃) = 𝛼, 𝐹 (𝑁 ))𝑃 (𝐻(𝜃)𝑃 (𝐹 (𝑁 ))(4)

The denominator 𝑃 (𝐹 (𝑁 )) of Eqn. (4) is the marginal¯ The numerator
probability, conditioned on all possibilities of 𝜃.
is a combination of likelihood and the prior 𝑓 (𝜃). Since 𝜃¯ iscontinuous, the denominator of Eqn. 4 can be written as:∫

𝑃 (𝐹 (𝑁 )) =¯ (𝜃)𝑑(¯ 𝜃)¯𝑃 (𝐹 (𝑁 )∣𝜃)𝑓(5)¯
𝐹 (𝑁 )(𝜃)Similarly, the numerator of Eqn. (4), can be written as:∫¯¯ = 𝛼, 𝐹 (𝑁 )∣𝜃)𝑓¯ (𝜃)𝑑(¯ 𝜃)¯𝑃 (𝐻(𝜃) = 𝛼, 𝐹 (𝑁 )) =𝑃 (𝐻(𝜃)¯𝐹 (𝑁 )(𝜃)(6)
Eqns. (5) and ( 6) can be solved and plugged into Eqn.( 4).
The details of this procedure is provided in [2]. Following that,it can be shown:¯ = 𝛼∣𝐹 (𝑁 )) = 𝜂𝛼 + 1 = 𝑏
𝑃 (𝐻(𝜃)(7)
𝑁 +3
For other hypothesis in Eqn.( 3), it can be similarly shownthat¯ = 𝛽∣𝐹 (𝑁 )) = 𝜂𝛽 + 1 = 𝑑
𝑃 (𝐻(𝜃)(8)𝑁 +3¯ = 𝛾∣𝐹 (𝑁 )) = 𝜂𝛾 + 1 = 𝑢
𝑃 (𝐻(𝜃)(9)
𝑁 +3
Eqns. (7), (8), and (9) are the posteriori probability masses for
Useful, Not Useful, and Not Sure as perceived by the raters,
respectively. The masses are referred as belief (𝑏), disbelief (𝑑),and uncertainty (𝑢) masses. For verification of correctness, it
can be shown that 𝑏 + 𝑑 + 𝑢 = 1. Additionally, when 𝜂𝛼 =𝜂𝛽 = 𝜂𝛾 = 0, all the possibilities are equiprobable under noinformation (non-informative prior).

B. Expected Truthfulness of Published Information
For computing the expected truthfulness, we apply the logistic function and the stretched exponential function to estimate
the weights of belief and uncertainty masses, respectively. The
problem is modeled similar to a weighted regression approach
where probability masses are independent (predictor) variables
and the expected truthfulness is a dependent (response) variable. Thus, if 𝑤𝑏 and 𝑤𝑢 are the coefficients (or weights)
of belief and uncertainty masses respectively, the expected
truthfulness for any published information 𝐼𝑘 will be givenas:𝜏𝑘 = (𝑤𝑏 ).𝑏 + (𝑤𝑢 ).𝑢(10)1) Estimation of the Belief Coefficient: Apart from theprobability masses, the expected truthfulness should also consider the confidence of the community, i.e., how many ratings
have been received for the concerned piece of information.
Intuitively, lesser 𝑁 (total number of ratings) should have low
𝑤𝑏 , which in turn, contributes to a lesser expected truthfulness.
However, 𝑤𝑏 should gradually increase as more ratings areavailable. Thus, to model this nature of 𝑤𝑏 we use the logistic
function in the following way:1
(11)
(1 + 𝐴𝑏 𝑒−𝐵𝑏 𝑁 )where 𝐴𝑏 is the initial value of the weight, and 𝐵𝑏 is the rate of
growth. More conservative (or high risk applications) systems
will have less 𝐴𝑏 and 𝐵𝑏 . This provision gives control to thesystem administrator to thwart the extent of ballot stuffing
and bad mouthing. This is because QoI based on only belief
masses usually get influenced in favor of the adversaries if
there are limited number raters in the system. However, if
a large crowd of independent raters is present, it may not be
possible to sabotage the entire proportion of positive ratings in
the adversary’s favor. Hence, the administrator should choose
𝐵𝑏 such that 𝑁 is large enough to prevent sabotaging.
2) Estimation of the Uncertainty Coefficient: In Eqn. (10),𝑤𝑢 controls the contribution of uncertainty to the effective belief. In general, if an incident has just occurred, and sufficient
notifications have not been received to establish a consensus,
uncertainty will be increasing. As this increase is also similar
to growth curve, we estimate the coefficient in terms of logistic
function. However, as more ratings are available over time,
this uncertainty in the perceived truthfulness of the gained
information gets reduced. Thus, once the number of ratings
reaches a threshold value, say 𝑁 = 𝑁𝑡ℎ𝑟𝑒𝑠 , the coefficient
should start to gradually decrease.
A stretched exponential function has been used to model the
decay aspect of the uncertainty and its parameter 0 < 𝜑 < 1
controls the diminishing rate. Hence, after 𝑁 = 𝑁𝑡ℎ𝑟𝑒𝑠 , thisfunction is used to amortize the 𝑤𝑢 over 𝑁 . The larger thevalue of 𝜑, the quicker is the decrease. Eqn.( 12) gives thevariation of 𝑤𝑢 with respect to the number of received ratings:⎧
1
if 𝑁 < 𝑁𝑡ℎ𝑟𝑒𝑠⎨ (1+𝐴𝑢 𝑒−𝐵𝑢 𝑁 ) ,
(12)
𝑤𝑢 =
⎩ −(𝑁 −𝑁𝑡ℎ𝑟𝑒𝑠 )𝜑𝑒,if 𝑁 ≥ 𝑁𝑡ℎ𝑟𝑒𝑠
𝑤𝑏 =The 8th International Workshop on Information Quality and Quality of Service for Pervasive Computing 2017

where 𝐴𝑢 and 𝐵𝑢 are respectively the corresponding asymptote and growth parameters, as discussed in Eq. (11).
C. QoI of Published Information
In Eqn. 10, 𝜏𝑘 is the expectation that the published information 𝐼𝑘 has actually happened. Now, the system needs
to perform a regression to determine the odds of 𝐼𝑘 beingtrue or false which we model as the QoI. We have used thegeneralized linear model (GLM) for this purpose. When the
response/predictor variables is categorical (true/false, yes/no,etc.) with non-normal error distribution, we need a link function to provide the relationship between the predictor variable(linear) and the mean of the distribution (explanatory) defining
the QoI. Thus, if 𝑄𝑘 is the response and 𝜏𝑘 is the mean,
the link between them is established by the following logit
function:)(𝜏𝑘
𝑄𝑘 = ln(13)
1 − 𝜏𝑘Here, 𝑄𝑘 is the QoI of the published information 𝐼𝑘
which has value in the interval [−∞, +∞]. The logit function
gives monotonically decreasing QoIs to all 𝜏𝑘 < 0.5, andmonotonically increasing ones to the rest.
V. P ERFORMANCE S TUDY
In this section, we present analytical and simulation resultsof 𝑊 2𝑄 and compare it with Jøsang’s belief model.
TABLE IIU SE CASES : C OMPARATIVE Q O I

Usecases⟨8, 4, 2, 2⟩⟨80, 40, 20, 20⟩⟨105, 5, 0, 100⟩
⟨25, 5, 0, 20⟩⟨25, 18, 2, 5⟩⟨100, 72, 8, 20⟩

Jøsang’s Model
0.59
0.62
0.54
0.58
0.79
0.81

Model 𝑊 2𝑄
0.05
0.54
0.27
0.16
0.25
0.75

A. Analytical Results
Let us compare our QoI scoring model with Jøsang’s
expected truthfulness (𝐸 𝐽 ) which is equivalent to 𝜏𝑘 in our
approach (because the scale of both metrics have to be between
0 and 1 for fair comparison). Jøsang’s belief model is given by
the following: 𝑏 = 𝑟 + 1; 𝑑 = 𝑠 + 1; 𝑢 = 𝑡 + 1; 𝐸 𝐽 = 𝑏 + 𝑎.𝑢,where 𝑎 = 0.5. The amortization factor in Jøsang’s belief
model known as relative atomicity is fixed as the reciprocalof the cardinality of inference state space (i.e. true or false
information) [7].
Table II presents a comparative analysis of the QoI generated by two approaches. Two important observations are: (i)𝑊 2𝑄 rewards those scenarios which has the higher community confidence, and (ii) unlike Jøsang’s model, 𝑊 2𝑄 does
not suffer from the risk of null-invariance.
B. Simulation Results
This section depicts the details of simulation settings, empirical comparison with Jøsang’s belief model, and efficiency
of 𝑊 2𝑄 in terms of distinguishing true and false information.

1) Simulation Settings: For experiment, an urban sensing
application (viz., air pollution, traffic, etc.) which receives
crowd-sourced notifications, has been considered. We simulate
a city with an area of 100×100 sq. units as the sensing region.
Let there be 𝑀 sensing agents and 𝑁 raters in the region,
where 𝑁 ≥ 2𝑀 as extrapolated from Epinions dataset [12].
The region is equally divided into four sub-regions, each of𝑁
which is initialized with 𝑀
4 sensing agents and 4 raters. Here,we consider 𝑀 = 800 and 𝑁 = 1600.
Out of 𝑀 sensing agents, 85% are considered to act ashonest agents whereas the rest as dishonest. An honest agent
sends notification for perceived true information with atleast
90% probability whereas a dishonest agent behaves the samefor a fake incident (that has not actually happened). It isnoteworthy that a dishonest agent may also send notification
for true information with probability between 5% and 40%.
Out of 𝑁 raters, we consider at least 75% will rate thepublished information correctly if any such rater lies in thesensing boundary (< 10 sq unit) of the published information.
Whereas, the rest of the raters are dishonest by nature. They
give positive ratings to false information and negative ratingsto true information. These considerations are made to replicate
the aforementioned threat model (section III-B).
The total sensing duration is uniformly divided into 𝑇
discrete number of time spans, where each span represents
a fixed time interval unit (such as 30 minutes). In each timespan, there is a pre-specified likelihood of occurrence of true
and/or false incident. Within a certain radius (10 sq. unit) of
the incident, each sensing agent is likely to send notifications
for it. Similarly, each rater is liable to rate the publishedinformation for the same.
For human mobility model, we consider a simple random
waypoint model where each human agent including rater
would have non-zero probability to move in any direction.
However, auto-configured sensors are fixed.
Unless otherwise stated, we consider following parametervalues for the simulation: 𝐴𝑏 = 𝐴𝑢 = 20, 𝐵𝑏 = 𝐵𝑢 = 0.08,
𝜑 = 0.2 and 𝑁𝑡ℎ𝑟𝑒𝑠 = 40.
2) Comparative Analysis with Jøsang’s Belief Model:
Fig. 3(a) shows the comparison between QoI achieved by
𝑊 2𝑄 and Jøsang’s belief model for false published information. We observe that 𝑊 2𝑄 does not provide undue highQoI to the published information with low number of ratings(< 60) unlike Jøsang’s belief model. However, with increasing
number of ratings, our model considers the confidence of thecommunity and converges to true QoI value.
Fig. 3(b) compares the QoI achieved by 𝑊 2𝑄 against
Jøsang’s belief model for true published information. It isnoteworthy that in 𝑊 2𝑄, the QoI converges to the true value
only after gaining a sufficient support from the community,
whereas in Jøsang’s belief model, that support (or community
confidence) does not matter.
3) QoI Analysis: Fig. 4(a) shows that 𝑊 2𝑄 is able toefficiently classify true and false published information. A true
information may end up having low QoI in scenarios with very
few ratings (i.e., low community confidence) as shown in Fig.

The 8th International Workshop on Information Quality and Quality of Service for Pervasive Computing 2017

0.8

Josang
W2Q

0.7
0.6
0.5

QoI

QoI

0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0

0.4
0.3
0.2

20

40

60

80

100

0.1
0

120

20

40

# of Ratings

120(b) True Information
QoI Comparison

1

0

0.5

-0.5

ACKNOWLEDGMENT
The work is partially supported by the NSF grants under award numbers CNS-1545037, CNS-1545050, and IIS1404673.
R EFERENCES

True Information
False Information

-1

0

QoI

QoI

Josang
W2Q
80 100# of Ratings(a) False Information
Fig. 3.

True Information
False Information

-0.5

-1.5
-2

-1

-2.5

-1.5
0

-3
0

20

40

60

80

100

120

20

Information ID

Fig. 4.

60

40

60

80

100

120

Information ID(a)(b)
Classification: (a) Normal Ratings, (b) Very Low Ratings

4(b). This is because of the service provider’s conservative
approach. Thus, 𝑊 2𝑄 takes both fraction of positive support
and community confidence to evaluate QoI.
4) Decision Parameter Analysis: In this section, we discuss
how the service provider may judiciously choose the suitablevalues for important parameters: 𝐵𝑏 , 𝐵𝑢 , and 𝑁𝑡ℎ𝑟𝑒𝑠 .
Effect of 𝐵𝑏 : Fig. 5(a), shows the effect of 𝐵𝑏 that controls
the number of ratings 𝑁 , required to attain the maximum
possible value of 𝑤𝑏 . For example, if the concerned area isinherently crowded and higher number of ratings are expected,
then 𝐵𝑏 should be kept low such that 𝑤𝑏 attains maximum
value after a sufficient number of ratings are received. However, if the system is less conservative, it can have low 𝐵𝑏 .
Similar is the case with 𝐵𝑢 .
0.6
0.5
0.4

Nthres = 25
Nthres = 50
Nthres = 100

wu

wb

1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
00

0.3

20

40

60

Bb = 0.04
Bb = 0.08
Bb = 0.24
80 100 120 140

0.2
0.1
00

50

# of Ratings

100

150

200

thwart the deterioration of the information quality, while being
more resilient to dishonest raters and null invariance. As a partof the future work, we intend to propose a reputation model
which estimates the reputation score of the sensing agentsbased on their quality of participation. Such score will enable
the application provider to segregate honest from rogue agentsso that fair incentives and service levels can be provided.

250# of Ratings(a)(b)
Fig. 5. (a) Effect of 𝐵𝑏 ; (b) Effect of 𝑁𝑡ℎ𝑟𝑒𝑠

Effect of 𝑁𝑡ℎ𝑟𝑒𝑠 : A small 𝑁𝑡ℎ𝑟𝑒𝑠 would prevent 𝑤𝑢 , to reachits maximum value, before the uncertainty discounting starts
as depicted in Fig. 5(b). Selection of an ideal 𝑁𝑡ℎ𝑟𝑒𝑠 depends
on the application type and the service provider’s risk attitude
which is an open issue.
VI. C ONCLUSION
In this work, we propose a novel model, called 𝑊 2𝑄, toevaluate the QoI of published information in social sensing.
𝑊 2𝑄 incorporates the total number of ratings, proportionof positive ratings and uncertain ratings. The resultant QoI
enables the application provider to decide whether to persist
with a published information or not, taking suitable actions to[1] R. P. Barnwal, N. Ghosh, S. K. Ghosh, and S. K. Das. Enhancing
Reliability of Vehicular Participatory Sensing Network: A BayesianApproach. Proc. of IEEE SMARTCOMP, pp. 1-8, 2016[2] S. Bhattacharjee and M. Chatterjee. Trust based Channel Preference in
Cognitive Radio Networks under Collaborative Selfish Attacks. Proc. ofIEEE PIMRC, pp. 1502-1507, 2014.
[3] Z. Feng, Y. Zhu, Q. Zhang, L. M. Ni, and A. V. Vasilakos. TRAC:
Truthful Auction for Location-aware Collaborative Sensing in Mobile
Crowdsourcing. Proc. of IEEE INFOCOM, pp. 1231-1239, 2014.
[4] K. L. Huang, S. S. Kanhere, and W. Hu. Are You Contributing
Trustworthy Data?: The Case for a Reputation System in Participatory
Sensing. Proc. of ACM MSWiM, pp. 14-22, 2010.
[5] L. G. Jaimes, I. Vergara-Laurens, and M. A. Labrador. A Location-based
Incentive Mechanism for Participatory Sensing Systems with Budget
Constraints. Proc. of IEEE PerCom, pp. 103-108, 2012.
[6] A. Jøsang. An Algebra for Assessing Trust in Certification Chains. Proc.
of NDSS, 1999.
[7] A. Jøsang. Trust-based Decision Making for Electronic Transactions. In
Proc. of NORDSEC, pp. 496-502, 1999.
[8] A. Jøsang and R. Ismail. The Beta Reputation System. Proc. of BledeConference, pp. 41-55, 2002.
[9] R. Kawajiri, M. Shimosaka, and H. Kashima. Steered Crowdsensing:
Incentive Design towards Quality-oriented Place-centric Crowdsensing.
Proc. of ACM Ubicomp, pp. 691-701, 2014.
[10] I. Koutsopoulos. Optimal Incentive-driven Design of Participatory Sensing Systems. Proc. of IEEE INFOCOM, pp. 1402-1410, 2013.
[11] Q. Li and G. Cao. Providing Privacy-aware Incentives for Mobile
Sensing. Proc. of IEEE PerCom, pp. 76-84, 2013.
[12] P. Massa and P. Avesani. Trust-Aware Bootstrapping of Recommender
Systems. Proc. of the ECAI Workshop on Recommender Systems, pp.
29-33, 2006.
[13] F. Restuccia and S. K. Das. FIDES: A Trust-based Framework for Secure
User Incentivization in Participatory Sensing. Proc. of IEEE WoWMoM,
pp. 1-10, 2014.
[14] M. B. Sinai, N. Partush, S. Yadid, and E. Yahav. Exploiting Social
Navigation. arXiv preprint arXiv:1410.0151, 2014.
[15] G. Wang, B. Wang, T. Wang, A. Nika, H. Zheng, and B. Y. Zhao.
Defending against Sybil Devices in Crowdsourced Mapping Services.
Proc. of ACM MobiSys, pp. 179-191, 2016.
[16] X. Wang, W. Cheng, P. Mohapatra, and T. Abdelzaher. ARTSense:
Anonymous Reputation and Trust in Participatory Sensing. Proc. ofIEEE INFOCOM, pp. 2517-2525, 2013.
[17] X. Wang, K. Govindan, and P. Mohapatra. Collusion-resilient Qualityof Information Evaluation based on Information Provenance. Proc. ofIEEE SECON, pp. 395-403, 2011.
[18] X. Zhang, Z. Yang, Z. Zhou, H. Cai, L. Chen, and X. Li. Free Market of
Crowdsourcing: Incentive Mechanism Design for Mobile Sensing. IEEE
TPDS, 25(12):3190- 3200, 2014.
[19] D. Zhao, X.-Y. Li, and H. Ma. How to Crowdsource Tasks Truthfully
without Sacrificing Utility: Online Incentive Mechanisms with Budget
Constraint. Proc. of IEEE INFOCOM, pp. 1213-1221, 2014.