Bayesian inference based decision reliability underimperfect monitoring
Shameek Bhattacharjee and Mainak Chatterjee

Kevin Kwiat and Charles Kamhoua

Electrical Engineering & Computer Science
University of Central Florida
Orlando, Florida 32816
Email: {shameek, mainak}@eecs.ucf.edu

Air Force Research Laboratory
Information Directorate
Rome, NY 13441
Email: {kevin.kwiat,charles.kamhoua.1}@us.af.mil

Abstract—Reliability of a cooperative decision mechanism iscritical for the proper and accurate functioning of a networked
decision system. However, adversaries may choose to compromise
the inputs from different sets of components that comprise thesystem. Often times, the monitoring mechanisms fail to accurately
detect compromised inputs; hence cannot categorize all inputs
into polarized decisions: compromised or not compromised.
In this paper, we propose a Bayesian inference model basedon multinomial evidence to quantify reliability for a cooperativedecision process as a function of beliefs associated with observations from the imperfect monitoring mechanism. We propose two
reliability models: an optimistic one for a normal system and aconservative one for a mission critical system. We also provide
an entropy measure that reflects the certainty or uncertainty
on the calculated reliability of the decision process. Throughsimulation, we show how the reliability and its corresponding
entropy changes as the accuracy of the underlying monitoringmechanism improves. 1

I. I NTRODUCTION AND M OTIVATION
Oftentimes, reliability of a cooperative decision in a networked system depends on how well the individual components perform and how reliable they are [1]. The individualcomponents could be a piece of hardware or software, or a linkconnecting two devices– all of which work together enabling
the system as a whole to perform its tasks. To improvereliability of a decision making process in the presence ofpossible component failures, redundancy and voting schemes
are often used [6]. Usually voting schemes are associated witha fusion rule (e.g., majority or plurality voting) which dictates
how individual votes are combined to decide the final output.
A malfunction or a security breach due to an adversary
might result in a faulty input (vote) to the central decision
making entity. Such inputs may potentially have an adverse
effect on the reliability of the decision process– the extent of
which depends on the inputs in question. A failure monitoringmechanism is supposed to detect such faults and providea ‘feedback’ on each component. However, this monitoringmechanism might not have the ability to identify such faults
with certainty due to inherent imperfect temporal and spatial
factors. Hence a binary decision on whether a fault occurredor not is not always possible for all inputs. With the adversary
1 Approved for Public Release; Distribution Unlimited: 88ABW-2014-4001,
Dated 26 Aug 2014.

employing a wide variety of attacks to compromise a dynamic
set of inputs (i.e., deciding the number of inputs and whichones), and imperfect monitoring conditions, it becomes difficult for a failure monitoring mechanism to quantify reliabilityof a collective decision based on inputs from individualcomponents.
In the presence of adversaries i.e., intentional cyber-attacks,
the compromised inputs may induce incorrect results in a cooperative decision making process. The set of inputs attacked
usually vary over time. The damage is evident when some
central entity of the system using some fusion rule fuses the
inputs from all components– compromised or not. A simple
example of a cooperative decision process could be a voting
system where components vote to produce a binary outcome. If
majority of the inputs from the components are compromised,
then the simple majority voting rule may produce a wrong
result [4].
In this paper, we investigate the quantification of reliabilityof a cooperative decision process based on the inputs fromvarious components. We assume, each component provides asingle input, all of which are prone to attacks. The underlying
imperfect failure monitoring mechanism produces varying
feedback over time. We consider that the outcome of the monitoring mechanism can be placed into three categories: those weknow have not been compromised, those we know have been
compromised, and those which cannot be inferred either way.
Given these, we compute the reliability of a decision process
in making a decision i.e., how reliable its output is. In thisregard, we conceptualize the outcome of monitoring the input
over time as a multinomial hypothesis of a Bayesian inference
model with three parameters. We build a Bayesian inference
based reliability model, where we assign a value to a decisionthat indicates how reliable the outcome of the decision is. The
way reliability is computed also depends on how much risk asystem can tolerate. For example, a mission critical system
may need to have a more strict reliability model because
the associated risks are too high. Therefore, we propose
two ways of computing the reliability– first is an optimistic
one and the second is a conservative one. The optimistic
model could be applied to systems where some tolerance
for wrong decisions are allowed. However, for a mission
critical system where there is almost no room for erroneousdecisions, the conservative model could be used. To accountfor the confidence associated with the reliability computation,
we propose an entropy based uncertainty value to represent
how certain or uncertain the computed reliability is. A lower
uncertainty associated with reliability value is an indication of
being more confident about that value. We conduct extensive
simulation experiments and show how reliability varies undera variety of system factors like attack intensity and inaccurate
detection. We observe that with more inputs compromised, thereliability over the fused decision reduces. Low reliability may
also be caused by temporal or initial lack of evidence due touncertainty. However, as time evolves, the system adjusts itself
towards more accurate monitoring and the reliability improves.
The rest of the paper is organized as follows. In section II,
we present the system model and state all the assumptions. In
section III, we provide the Bayesian model for reliability. We
present the reliability models in section IV. The simulation
model and results are discussed in section V. Conclusions are
drawn in the last section.
II. S YSTEM M ODEL AND A SSUMPTIONS
We consider a time-slotted system comprising N voting
components
each of which provides only one input (i.e., the vote) on
each time slot. The nature of the decision is generic; it could
be as simple as a binary voting or it could be some complex
decision metric. A centralized controller fuses all votes fromeach component through a fusion scheme (e.g., majority orplurality voting rule) to arrive at a global decision.
•Adversarial model: We assume that all the inputs fromeach component are exposed to an adversary whose goal is
to disrupt the voting process at the central controller. The
adversary has some predefined attack resources and can choose
to attack different sets of inputs over time and also attack
varying number of inputs in each time slot. However, it
maintains a long term average of the fraction of the inputs
it attacks which we call the probability of attack and denoteas Pa . For example, Pa = 0.6 means that the adversary
compromises 60% of the inputs over a large period of time.
Hence a single observation (over one time slot) is not sufficientfor characterizing the behavior of the adversary.
•Imperfect component failure monitoring: We assume that
there is a component failure monitoring or anomaly detection
mechanism in place that infers whether the input from eachcomponent has been compromised or not. Oftentimes, themonitoring mechanism cannot infer anomaly with certainty.
Thus, it classifies the inputs into three categories: i) compromised, ii) not compromised, and iii) undecided. All three area function of environmental parameters that may be dynamic
over time. Note, the monitoring occurs over a period of time.
Also system transients and noisy environments may increase
temporal uncertainty. Therefore, reliability is computed overtime– a larger time window of observation allows a moreaccurate estimation of the actual reliability.
•Uniformly distributed prior inference: Since there is nobias over any of the three possible outcomes of the monitoringprocess, we assume that the initial probabilities of each isequal.
•Probability of detection: We define the probability ofdetection as the percentage of ‘components’ inputs that canbe accurately inferred as compromised or not compromisedand denote it as Pdet . Let us further illustrate the meaningof Pdet using Fig. 1 that shows an input in reality could beeither compromised or not compromised. If compromised, it
can be inferred as either as compromised with a probability(say a1 ) or undecided with probability 1−a1 . Note, we assume
that there is no way a compromised input will be inferred
as not compromised. This assumption is because we argue
that ‘undecided’ is inferred in absence of credible evidence
else there will be a compromised feedback only when sure.
Similarly, if an input is not compromised, it can be inferred
as either ‘not compromised’ with a probability (say b1 ) or
‘undecided’ with probability 1 − b1 . Again, there is no way a
not compromised component will be inferred as compromised.
Thus, for the two real cases, detection occurs with probabilities
a1 and b1 . If an input has equal chances of being compromised1
and not compromised, then Pdet = a1 +b
2 . Else, a1 and b1 will
have to be weighted with their corresponding probabilities. For
all practical purposes, we consider Pdet to be at least 0.5.
Real

Inferreda1Compromised
Input

Compromised

1 − a1
1 − b1

Undecided

Not compromisedb1
Not compromised

Fig. 1.

Inference possibilities for detection probability

The above features make the problem of computing the reliability of the output decision a probabilistic concept. Hence,
we compute the reliability as a continuous process based on
observations over time slots. If the adversary uses the sameattack strategy, then the reliability will converge sooner. On theother hand, if the adversary changes its attack strategy (i.e.,
dynamic attack strategy), the reliability will oscillate even forlarge time windows.
III. R ELIABILITY BASED D ECISION M AKING
Suppose that the three feedbacks of the monitoringmechanism– ‘not compromised’, ‘compromised’, or ‘undecided’ be denoted by α, β and µ respectively. Let nα represent
the number of inputs that have ‘not’ been compromised,nβ represent the number of inputs observed to have been
compromised, and nµ is the number of inputs for which we
do not know either way. Of course, nα + nβ + nµ = N . Since
the values of nα , nβ and nµ change over time, we represent
these observations at time t as nα (t), nβ (t) and nµ (t).
Since the system’s underlying parameters of cooperative
voting behavior are unknown, we propose to use Bayesianinference to update corresponding probability estimate for ahypothesis that the decision process is correct with a certainTABLE I
N OTATIONS
Symbolα, β, µnαnβ , nµNθα , θβ , θµ
θ̄
X(θ̄)
X̂(θ̄
D(N )
Pdet
Rα , Rβ , Rµ
Rso
Rsc
Es

Meaning
Not compromised, compromised or undecided events
Number of inputs detected as ‘not compromised’
No. of inputs det. as ‘compromised’ and ‘undecided’
Total number of voting components
Unknown probability for observing event α, β, µBayesian probability parameters of the three events
Hypothesis of a event
Posterior hypothesis or belief
Random vector denoting data hyperparameter
Probability of detection
Posterior Bayesian belief of the three events
Reliability of an optimistic system
Reliability of an conservative system
Entropy or uncertainty associated with the system

reliability. The system is only as reliable as the individual
inputs are. Therefore, we have to calculate the posterior
probabilities associated with encountering each of the threefeedbacks. The final decision reliability will be a function ofthese posterior probabilities which are also known as belief
estimate in Bayesian inference.
To begin with, an uniform belief over the three possibilities
is assumed as there is no initial information. As time progresses, we update the belief estimate based on the observed
values of α, β, and µ which increases the accuracy of theestimate of the belief associated with each category.
We define θα , θβ , and θµ as the probabilities for an input
being ‘not compromised, ‘compromised’, and ‘undecided’
respectively. Of course, θα + θβ + θµ = 1, since the outcomesare exhaustive and mutually exclusive. We define X(θ̄) as the
hypothesis described by these underlying unknown Bayesian
probability parameters where θ̄ = {θα , θβ , θµ }.
Let Dα , Dβ , and Dµ represent the random variables that
represent the number of times the outcomes α, β and µ occur.
The observation data can be represented as random observation vector D(N ) = {Dα , Dβ , Dµ } having a multinomial
distribution also known as concentration hyperparameter ofthe underlying 3-tuple probability parameter described by θα ,θβ , and θµ . The commonly used notations are tabulated inTable I.
A. Bayesian Inference
As mentioned earlier, there are N independently monitoredcomponents of a system whose parameters for voting behavior
are unknown due to changing adversarial attack strategies andthe imperfect monitoring mechanism. Given this, we calculate
the Bayesian belief associated with ‘not compromised’. Similarly, we will model Bayesian posterior belief for the othertwo cases as well viz. compromised and undecided.
We use the observation counts from the sequential observations over time to calculate the posterior Bayesian estimate ofeach of the parameters. Our objective is to estimate and update
the probability parameters in X(θ̄), viz. θα , θβ , and θµ basedon observation evidence D(N ) and prior information on the
hypothesis parameter, θ̄, itself.
Since there is no information about θ̄ initially, we consider
the prior parameters of θ̄ to be uniformly distributed a-priori.
Subsequent observations will decide how these parameters are

updated. Our first step is to calculate the Bayesian estimate ofθ̄.
First, we show the case of estimating belief that a ‘notcompromised’ occurs (θα ). Since in Bayesian inference, theassumption is that prior and posterior probability have thesame distribution, we can formally define the probability
parameters as:
P (X(θ̄) = α|θ̄) = θαP (X(θ̄) = β|θ̄) = θβP (X(θ̄)) = µ|θ̄) = θµ(1)This assumption is due to the well known fact that a Dirichlet
distribution acts as a conjugate prior to multinomial distributions [7]. Hence prior and posterior preserve the same form.
The observations data D(N ) can be treated as a multinomial
distribution with probability parameter θα , θβ , and θµ, where
the probability mass function is given by:
P (Dα = nα , Dβ = nβ , Dµ = nµ |θ̄) = P (D(N )|θ̄)N!
nθnα θ β θnµ=nα !nβ !nµ ! α β µ(2)Given this we can use Bayes theorem to calculate theposterior belief estimate on the event of a positive interactionX̂(θ̄) = α, given observation data D(N ) as:P (X̂(θ̄) = α|D(N )) =P (X̂(θ̄) = α, D(N ))P (D(N ))(3)

Denominator of the above equation is the marginal probability that can be conditioned or marginalized on all possible
outcomes for θ̄ and since probabilities are continuousZ
P (D(N )) = P (D(N )|θ̄)f (θ̄)d(θ̄)(4)
D(N )(θ̄)

Since there is no prior information on θ̄ (before any observations) in Eqn. (4), we can assume it to be uniformly distributed
such that f (θ̄) = 1 and we can put Eqn. (2) in Eqn. (4), and
get
Z
N!
n
θαnα θβ β θµnµ dθα dθβ dθµ (5)P (D(N )) =nα !nβ !nµ !
D(N )(θα ,θβ ,θµ )For simplicity, let

R

n

n

θαnα θβ β θµ µ dθα dθβ dθµ = I1

D(N )(θα ,θβ ,θµ )To solve for I1 we use the multivariate generalization of theEulerian integral of first kind. Note that D(N )(θα , θβ , θµ )denotes a space and we know that a space of m(= 3)parameters has only m − 1(= 2) degrees of freedom due tothe additivity constraint θα + θβ + θµ = 1. Therefore when
we integrate over this space, the integration has m − 1 = 2
dimensions. Hence
Z 1Z 1−θα −θβ(n +1)−1θα(nα +1)−1 θβ β
(1 − θα − θβ )(nµ +1)−1 dθα dθβ
I1 =
0

0

(6)
Eqn. (6) is a known form for the multivariate extension of
the Beta function which in this case is defined as B(nα +1, nβ + 1, nµ + 1). The proof can be found in Lemma 2.4.1
of [3]. In general B(α1 , · · · , αm )
Z
m−1
X
1 −1
xi )αm −1 dx1 ...dxm−1=xα...(1 −1i=1
D(x1 ,··· ,xm−1 )QmΓ(α )Γ(α1 ) · · · Γ(αm )i=1
Pm i ==
Γ(α1 + · · · + αm )Γ( i=1 αi )(7)Using the above result, we can write Eqn. (6) as
=

I1

=

B(nα + 1, nβ + 1, nµ + 1)
Γ(nα + 1)Γ(nβ + 1)Γ(nµ + 1)
Γ(nα + 1 + nβ + 1 + nµ + 1)

(8)

Putting Eqn. (8) in Eqn. (5) we get:P (D(N )) =N!
Γ(nα + 1)Γ(nβ + 1)Γ(nµ + 1)
(9)nα !nβ !nµ ! Γ(nα + 1 + nβ + 1 + nµ + 1)

Since the parameters in gamma functions nα + 1 etc. are allnon-zero positive values, we can use the result Γ(z) = (z −1)!
to calculate Eqn. (9) asP (D(N )) =N!
(N + 2)!

(10)

Assuming conditional independence between the X̂(θ̄),
D(N ) and θ̄, we calculate the numerator of Eqn. (3),P (X̂(θ̄) = α, D(N )), as:Z=
P (X(θ̄) = α, D(N )|θ̄)f (θ̄).d(θ̄)
D(N )(θ̄)=

Z

P (X(θ̄) = α|θ̄)P (D(N )|θ̄)d(θ̄)

D(N )(θ̄)=

N!
nα !nβ !nµ !

=N!
nα !nβ !nµ !

Z

We propose two reliability models and also show how entropy can capture the uncertainty associated with the reliability
calculation due to the undecided inputs.
A. Decision Reliability for an Optimistic System
For a system, we assumed that adversary has uniformly
chosen the inputs it chooses to attack i.e., there is no reason
for preferential attack on a certain component’s input. Hence
we can account for the undecided Rµ by splitting it in theratio of Rα : Rβ , and adding it to the value Rα to provide
the optimistic reliability denoted by Rso . Of course, when the
proportion of Rµ is high, we may not be as confident on thereliability value than when we have lower values of Rµ . Thus,
Rso is computed as:
RαRµRα + Rβ(14)

B. Decision Reliability for a Conservative System

D(N )(θα ,θβ ,θµ )nθαnα +1 θβ β θµnµ dθα dθβ dθµ(11)D(N )(θα ,θβ ,θµ )The above integral has the same form as Eqns. (5), (6), and(7). Hence this integral portion of Eqn. (11) can be rewritten as
Z Z1 1−θα −θβ(n +1)−1θα(nα +2)−1 θβ β
(1 − θα − θβ )(nµ +1)−1 dθα dθβ=0 0which can be solved using Eqn. (7).
Using the above result, Eqn. (11) can be simplified as
P (X̂(θ̄) = α, D(N )) =IV. R ELIABILITY M ODELS

Rso = Rα +nθα θαnα θβ β θµnµ dθα dθβ dθµZposterior belief of ‘not compromised’, ‘compromised’, and‘undecided’. To simplify the notations of belief estimates of
the three categories, we rewrite them as Rα , Rβ , Rµ respectively. Of course, it can be verified that Rα + Rβ + Rµ = 1.
The above equations also satisfy the Cromwell’s rule [5],which suggests that no prior belief unless logically impossible,
should be assigned zero probability even if no events in thatcategory has occurred so far.
We mentioned that in Bayesian inference posterior probabilities may be used as priors for future calculations. The
Cromwell’s rule leaves open the probability however small,to experience an event that has not occurred yet but may
happen in future. Hence Bayesian estimates should have nonzero priors for an event that has not occurred yet. From thederived equation it is evident that even if nβ = 0, Rβ 6= 0, is
a very small number but not zero when N is large.

N !(nα + 1)
(N + 3)!

(12)

Thus, Eqn. (3), can be solved by dividing Eqn. (12) byEqn. (10), which gives
P (X̂(θ̄) = α|D(N )) =nα + 1
N +3
n +1

(13)

Similarly, P (X̂(θ̄) = β|D(N )) = Nβ+3 and P (X̂(θ̄) =n +1µ|D(N )) = Nµ+3 . These equations are the expressions for

Unlike the optimistic approach, where the undecided ones
are split in a ratio, the conservation model treats the undecided
ones as if they were compromised. In other words, only the‘not compromised’ event (Rα ) is used for computing thereliability. Hence,
Rsc = Rα(15)This conservative way of computing the reliability is more
appropriate for mission-critical systems where the decisions
can only be made based on the ‘not compromised’ inputs. No
risk is taken on the undecided inputs even if there could besome that were not compromised.
C. Uncertainty associated with Reliability
System reliability as computed by Eqn. (14) can yield thesame value for different sets of Rα , Rβ , and Rµ . For example,
consider two scenarios.
Scenario 1: Rα = Rβ = 0.5, and Rµ = 0
Scenario 2: Rα = Rβ = 0.3, and Rµ = 0.4
For both scenarios, the optimistic decision reliability as given

Scenario
1
2
3

Rα
0.5
0.3
0.2

Rβ
0.5
0.3
0.4

Rµ
0.0
0.4
0.4

Rso
0.5
0.5
0.33

Es
1.00
1.57
1.51

by Eqn. (14) is 0.5 as shown in Table II. It can be notedthat though scenario 2 has higher Rµ than scenario 1, Rso
for both are the same. However, intuitively we ought to trust
scenario 1 more than scenario 2 because more certain decisions
have been made when Rµ is less. We know that higher valuesof Rµ reduces the chances of being closer to the real valueof reliability. As shown in the two scenarios, the optimistic
reliability can yield the same value for different sets of Rα ,Rβ , and Rµ
In order to illustrate the uncertainty associated with the Rµ ,we use entropy which is a measure of uncertainty inherent ina system. Usually, entropy of a system uses the steady state
probabilities that the system could be in. We define the entropyof the decision reliability as:

A. Optimistic Decision Reliability: Instantaneous and Average
In Fig. 2, we plot both the instantaneous and averagedecision reliability for the optimistic reliability model when
the adversary launches attacks with Pa = 0.5. We observethat the decision reliability fluctuates over time. As expected,
with sufficient observations, the moving average of decision
reliability converges to a steady state reliability equal to 1−Pa
which in this case is 0.5.
0.7
Instantaneous Reliability
Moving Average Reliability

0.65
0.6

Optimistic Reliability

TABLE IIR ELIABILITY-E NTROPY TUPLE ; N=1000

0.55
0.5
0.45
0.4
0.35

0

100

200

300
400
Time Slot

500

600

700

Fig. 2. Instantaneous and average decision reliability with Pa = 0.50 and
Pdet = 0.8

Es = −[Rα log2 (Rα ) + Rβ log2 (Rβ ) + Rµ log2 (Rµ )] (16)We simulate a generic centralized system with 100 components. Inputs from all components are monitored by animperfect monitoring mechanism that produces three possible
outcomes. The probability of detection, Pdet , is varied tocapture its effects on decision reliability.
An adversary attacks and compromises different sets of
inputs over time. The number of inputs compromised varyover time slot; although the long-term average of the numberof inputs compromised, denoted by Pa , remains the same. We
study the decision reliability for different values of Pa , andplot instantaneous and moving average of decision reliability.
We first present the results for an optimistic system and thenfor a more conservative system.

In Fig. 3, we plot steady state decision reliability withincreasing Pa and for two different values of Pdet . The plotsshow a steady decrease of decision reliability values for Pdet
0.5 and 0.9. Recall, with different Pdet the values of averagedecision reliability may differ but the relation between Rso
and Pa does not change with change in Pdet unlike Rsc .
This is because inputs chosen by the adversary is uniformly
random. The conservative reliability Rsc , also falls linearly withincreasing Pa , but the slope or rate of this change varies forthe different values of Pdet . This is because the conservativemodel does not account for the undecided ones. Hence a lower
value of Pdet yields a lower reliability value than a higher
value of Pdet .
Furthermore, Fig. 4 shows the entropy values of the system
under the same set of Pdet . This shows that a system withhigher Pdet has a lower uncertainty on the decision reliability
estimate for all values of Pa . This is useful when we use
optimistic system reliability which uses a fraction of undecided
ones to contribute to the final reliability value.
0.9

0.9
0.8
0.7

0.8

o
: P
= 0.50
s
det
c
R : P
= 0.50
s
det

R

0.6
0.5
0.4
0.3

0.5
0.4
0.3

0.1

0.1

Fig. 3.

0.6

0.2

0.2

0
0

o
: P
= 0.90s
detc= 0.90
s
det

R

R : P
0.7

Decision Reliability

V. S IMULATION M ODEL AND R ESULTS

B. Decision Reliability and Entropy

Decision Reliability

The entropy, Es , captures the uncertainty which is shownin Table. II. Scenario 1 with Rµ = 0 has an uncertainty
measure of 0.69 which is lower than scenario 2 with 1.08.
Thus, the reliability of scenario 1 can be trusted more thanthat of scenario 2.
Now, let us consider scenario 3 which has the same Rµ as
of scenario 2; however, Rβ is higher than scenario 2. Hence
scenario 3 has lesser reliability value than scenario 2. Since
scenario 2 has been established to be less trustworthy than1, we can also verify that from Rso of scenario 3 is alsoless than that of scenario 1. However, these arguments holdtrue only when at least 50% of the observations are accurate
observations, i.e., Rµ < 0.5. This is a reasonable assumption
as it is impractical to have a detection mechanism whichcannot detect or decide majority of the time.

0.5
Probability of Attack

1

0
0

0.5
Probability of Attack

1

Optimistic and Conservative Reliability over Pa for different Pdet

1.6

1
0.9

1.5

Instantaneous Reliability
Moving Average Reliability

0.8

1.4
0.7

Reliability

Entropy

1.3
1.2
1.1

Entropy: Pdet=0.5

0.5
0.4
0.3

Entropy: Pdet=0.9

1

0.6

0.2

0.9

0.1

0.8

0
0

0.1

0.2

0.3

0.4
0.5
0.6
Probability of Attack

0.7

0.8

Fig. 6.
Entropy over different values of Pa

In Fig. 5, we show how entropy changes with increasing
probability of attack. As time evolves and Pdet improves,
the uncertainty (i.e., entropy) associated with the decision
reliability decreases indicating increased confidence on thereliability values.

Detection Accuracy

Fig. 4.

Detection Accuracy

1
Detection Accuracy
0.8
0.6
100

200

300

400
Time Slot

500

600

700

800

400
Time Slot

500

600

700

800

Entropy

1.6
1.4
1.2

Moving Entropy

1
0.8
0

Fig. 5.

100

200

300

200

300

400

500
Time

Conservative decision entropy with incremental increase of Pdet

D. Conservative Reliability Model
In Fig. 7, we plot the changes in conservative decision reliability, over time with gradual increase in Pdet . As mentioned
earlier, if the system evolves into more accurate monitoring,the decision reliability also improves, although Rs 6= 1 − Pa .
Thus, having a conservative decision reliability is not unfair,given that risk associated with it is high, and there is scopefor improving reliability when detection accuracy increases.

Conservative Reliability

600

700

800

900

1000

Decision Reliability under Non-Uniformly distributed Pa = 0.50

1Detection Accuracy
0.8
0.6
0

C. Decision Reliability with Time Variant Pa
Some adversaries may choose to attack with short-term low
or high attack probability; however, maintaining the long-term
average value of Pa . For example, an adversary attacks lessinitially conserving its attack resources for future and eventually attacking more (under favorable conditions). In Fig. 6, weinvestigate how the proposed model behaves in such cases. We
consider an adversary with Pa = 0.5. The first 500 slots areattacked less and next 500 slots are compensated by attacking
more. We observe that decision reliability progressively moves
towards the expected reliability, although no strict convergenceon the decision reliability is achieved.

0

100

0.9

100

200

300

400
Time Slot

500

600

700

800

500

600

700

800

0.8
0.6

Instantaneous Reliability

0.4
0.2
0
0

Fig. 7.

100

200

300

400
Time Slot

Conservative decision reliability with increasing Pdet

VI. C ONCLUSIONSIn this paper, we presented a Bayesian inference basedmodel to translate multinomial observations for any voting
system under imperfect monitoring. We computed the posterior belief for the decision reliability based on the numberof compromised, not compromised, and undecided inputs.
We proposed two models for decision reliability– optimistic
and conservative. Through simulation experiments, we showed
how the decision reliability changes with attack probability
which also affects the detection accuracy of the underlying
monitoring mechanism. Using entropy, we provided a way toevaluate the certainty of the reliability calculation. Finally, we
showed how the decision reliability decreases when the attack
probability increases.
R EFERENCES[1] A. Avizienis, J.-C. Laprie, B. Randell, and C. Landwehr, “Basic concepts
and taxonomy of dependable and secure computing,” IEEE Trans.
Dependable Secure Computing, vol. 1, no. 1, pp. 11–33, Jan. 2004.
[2] D. Lindley “Making Decisions”, Wiley 2nd Edition, pp. 104, 1991.
[3] K. Fang and Y.T. Zhang, “Generalized Multivariate Analysis, SpringerVerlag, pp. 47-49, 1990.
[4] K. Kwiat, A. Taylor, W. Zwicker, D. Hill, S. Wetzonis, and S. Ren,
“Analysis of binary voting algorithms for use in fault-tolerant and securecomputing,” ICCES, 2010, pp. 269-273.
[5] D. Lindley “Making Decisions”, Wiley 2nd Edition, pp. 104, 1991.
[6] D. P. Siewiorek and R. S. Swarz, Reliable Computer Systems: Design
and Evaluation, 3rd Edition, A. K. Peters/CRC Press, 1998.
[7] Y.L. Sun, W. Yu, Z. Han and K.J.R. Liu, “Information Theoretic
Framework of Trust Modeling and Evaluation for Ad-Hoc Networks,
IEEE J. Sel. Areas in Commn, Vol. 24, Issue 2, 2006.