8 The dev elopmen t and v alidation of metho ds for fault detection and iden ti ca- tion in w astew ater treatmen t researc h to da y relies on t w o imp ortan t assump- tions: (i) that sensor faults app ear at distinct times in dieren t sensors and (ii) that an y giv en sensor will function near-p erfectly for a signi can t amoun t of time follo wing installation. In this w ork, w e sho w that suc h assumptions are unrealistic, at least for sensors built around an ion-selectiv e measure- men t principle. Indeed, long-term exp osure of sensors to treated w astew ater sho ws that sensors exhibit imp ortan t fault symptoms that app ear sim ulta- neously and with similar in tensit y . Consequen tly , our w ork suggests that fo cus of researc h on metho ds for fault detection and iden ti cation should b e reorien ted to w ards metho ds that do not rely on the assumptions men- tioned ab o v e. This study also pro vides the v ery rst empirically v alidated sensor fault mo del for w astew ater treatmen t sim ulation and w e recommend its use for eectiv e b enc hmarking of b oth fault detection and iden ti cation Corresp onding author, Email kris.vil lez@e awag.ch Pr eprint submitte d to W ater R ese ar ch De c emb er 19, 2018 metho ds and adv anced con trol strategies. Finally , w e ev aluate the v alue of redundancy for the purp ose of remote sensor v alidation in decen tralized w astew ater treatmen t systems. Keywor ds: data qualit y, drift, fault detection and iden ti cation, 9 ion-selectiv e electro des, predictiv e main tenance, w astew ater 10 1. In tro duction 11 By sev eral accoun ts, the lac k of online sensor data qualit y p oses a long- 12 standing c hallenge for b oth the adv ancemen t of en vironmen tal science and 13 engineering practice (Rieger et al., 2005, 2006; Rosén et al., 2008; Rieger 14 et al., 2010; Haimi et al., 2013; Corominas et al., 2018). It is therefore not 15 surprising that considerable time and energy has b een in v ested in metho ds 16 for automated qualit y assessmen t and qualit y con trol of online measuremen t 17 devices (e.g., Thomann et al., 2002; Thomann, 2008; Corominas et al., 2011; 18 Spindler and V anrolleghem, 2012; Alferes et al., 2013; Spindler, 2014; Villez 19 and Hab ermac her, 2016; Le et al., 2018). 20 Metho ds that are nding their w a y in to practice to da y mainly consist of 21 sanit y c hec ks. In the authors' exp erience, these w ork rather w ell to detect 22 and classify a subset of commonly recognized fault symptoms, including out- 23 liers, spik es, stuc k, and out-of-range v alues. F or sensor faults that lead to 24 more subtle symptoms, curren t practice relies primarily on regular on-site 25 sensor main tenance, e.g. once ev ery one or t w o w eeks, to coun ter suc h subtle 26 faults. F or unstaed w astew ater treatmen t plan ts, on-site main tenance ma y 27 2 b e feasible economically only if this is limited to once p er y ear. This practical 28 constrain t to the adoption of qualit y assessmen t and con trol practices forms 29 the primary motiv ation for this study . 30 The literature suggests that data-analytical tec hniques can enable auto- 31 mated and remote detection of sensor faults. Without exception, suc h tec h- 32 niques rely on redundan t relationships and can therefore b e categorized b y 33 the t yp e of redundancy that is used. A rst category consists of tec hniques re- 34 lying on reference measuremen ts and computing a deviation b et w een online 35 sensor signal and the reference signal. A second category relies on hard- 36 w are redundancy b y placing m ultiple online sensors, p ossibly built around 37 a distinct measuremen t principle, in the same lo cation and then computing 38 deviations b et w een them. A third category relies on temp oral redundancy , 39 essen tially assuming that meaningful c hanges in the sensor signal can only b e 40 smo oth when measured with a su cien tly high frequency . Finally , the fourth 41 category relies on spatial redundancy , relating signals pro duced at distinct 42 lo cations or for dieren t measured v ariables. Examples of this last cate- 43 gory include b oth metho ds based on rst principles, e.g. balance equations, 44 as w ell as metho ds ro oted in statistical practice, e.g. principal comp onen t 45 analysis. Imp ortan tly , eac h of these adv anced metho ds require tuning to 46 maximize the n um b er of true alarms and to ensure suitable qualit y con trol 47 eorts while sim ultaneously minimizing the n um b er of false alarms and fu- 48 tile main tenance actions. In v ariably , suc h tuning is obtained b y means of 49 a historical, fault-free data set from whic h acceptable limits for computed 50 3 residuals are deriv ed. Consequen tly , this means that these metho ds rely on 51 the a v ailabilit y of represen tativ e data of an acceptable qualit y . In addition, 52 the use of most tec hniques implies that sensor fault symptoms can b e as- 53 sumed to app ear indep enden tly from eac h other, i.e. the probabilit y that 54 t w o faults start at the same time is assumed to equal zero. 55 The prev alence of faults in actuators, sensors, and pro cesses as w ell as 56 the complexit y of the fault detection and iden ti cation (FDI) task, has led 57 to a plethora of metho ds that exploit one or more of the t yp es of redundancy 58 discussed ab o v e. In fact, the w ealth of literature as w ell as the n um b er 59 of reviews on this or related topics (V enk atasubramanian et al., 2003c,a,b; 60 Haimi et al., 2013; Corominas et al., 2018) suggest that the science and 61 practice of FDI is all but settled, an observ ation also supp orted b y no free 62 lunc h theorems (W olp ert, 1996). 63 Despite the tremendous amoun t of researc h on FDI metho ds, little is ac- 64 tually kno wn ab out the cause-and-eect relationships b et w een sensor ageing, 65 the o ccurrence of sensor faults and failures, and the pro duction of fault y data. 66 This is explained b y the fact that the a v ailabilit y of information describing 67 the exact circumstances under whic h faults o ccur or fault y data is pro duced, 68 i.e. meta-data, is usually sev erely limited. This is the secondary motiv ation 69 of this study . 70 T o facilitate p erformance ev aluation of FDI to ols, the form ulation of sim- 71 ulation b enc hmarks has b een an accepted practice in engineering sciences 72 (Bart y et al., 2006; Do wns and V ogel, 1993). Similarly , the Benc hmark Sim- 73 4 ulation Mo del No. 1 w as conceiv ed as a w a y to test and compare inno v ativ e 74 FDI and con trol strategies (Jeppsson et al., 2007). T o da y , it is primarily used 75 as a starting p oin t for a family of plan t-wide mo dels of w ater resource reco v- 76 ery facilities (Nop ens et al., 2009; V olc k e et al., 2006). A ctual b enc hmarking 77 of FDI metho ds has b een limited to one study so far (Corominas et al., 2011). 78 The BSM family includes a set of sensor mo dels whic h include sensor faults 79 and this allo ws the user to add realism to the sensor signals. The sim ulated 80 sensor faults alw a ys start at a time that is substan tially later than the start 81 of the sim ulated time. This pro vides ideal conditions for FDI metho d tuning 82 as high-qualit y sensor data are alw a ys presen t in the rst sections of the sim- 83 ulated data set. Moreo v er, a sim ulated fault alw a ys app ears indep enden tly 84 of an y other sensor fault, i.e. no t w o sensor faults are sim ulated to start at 85 the same time or with the same direction or magnitude. W e exp ect that the 86 situation in real-w orld conditions is v ery dieren t. W e th us h yp othesize that 87 t ypical fault symptoms will app ear at the same time and with similar direc- 88 tions and magnitudes when exp osed to the same harsh medium, esp ecially 89 when the same measuremen t principle is applied. Ev aluating the merit of 90 this h yp othesis is the tertiary motiv ation of this study . 91 The follo wing paragraphs are fo cused on the results and conclusions 92 dra wn directly from exp erimen tal data obtained during a long-term sensor 93 exp osure exp erimen t. A dditional insigh t is ho w ev er obtained b y studying a 94 v ariet y of dynamic mo dels to describ e our measuremen ts. 95 5 2. Materials & Metho ds 96 2.1. The or etic al and r e al-world b ehavior of the ion-sele ctive ele ctr o des for pH 97 me asur ement 98 The ion-selectiv e measuremen t principle for pH measuremen t is under- 99 sto o d rather w ell. A ccording to the Nernst equation (W estcott, 2012) one 100 measures an electric p oten tial E (in m V), whic h is related to the activit y of 101 the protons, [H+] , in the measured medium in steady state: 102 E=E0+RT Fln H+ (1) whereE0is the reference p oten tial, F is the F arada y constan t ( 96485:33289Cmol1, 103 T a ylor et al., 2007), [H+] is the proton activit y in the reference cell, R is the 104 molar gas constan t ( 8:3144598Jmol1K1, T a ylor et al., 2007), and T is 105 the temp erature measured in Kelvin. The pH is de ned as log [H+] (Buc k 106 et al., 2002) so that S(T) is the temp erature-sp eci c sensitivit y , whic h can 107 b e computed as: 108 S(T) =RT Flog (e)(2) Most t ypically , pH sensors are designed to deliv er 0 m V at pH 7 so that 109 E0is theoretically 0 m V. Similarly , the theoretical sensitivit y at standard 110 6 am bien t temp erature and pressure (SA TP) th us is S(298:15) = 59:1593 m V 111 p er pH unit. Because the actual v alues of these parameters tend to deviate 112 from their theoretical v alues, it is common to iden tify their v alues through 113 a 2-p oin t calibration pro cedure. A t the engineering departmen t at Ea w ag, 114 the most common practice is to use buered calibration media with pH 4:01 115 and7:00 for v alidation, follo w ed b y calibration when the absolute deviations 116 b et w een the pro duced pH measuremen ts and the kno wn pH v alues exceed a 117 predetermined threshold. The data end user sets this threshold. Dep ending 118 on the application, this ranges from 0:1 to0:4 pH units. The theoretical 119 p oten tial at pH 4:01 and SA TP is 177:0 m V. 120 2.2. Studie d sensors 121 A total of 12 pH sensors are pro duced b y Endress+Hauser (Reinac h, 122 Switzerland). These sensors consist of 5 sensor t yp es (T1-T5) whose exact 123 t yp e cannot b e rev ealed due to a con den tialit y agreemen t. The rst eigh t 124 sensors consist of pairs of four commercially a v ailable sensor t yp es (T1-T4) 125 whic h are t ypically sold with a one-y ear w arran t y agreemen t. The rst (sec- 126 ond) sensor in eac h pair is designated with an a ( b ), e.g. T1a, T1b. The last 127 4 pH sensors are replicates of a recen tly dev elop ed sensor protot yp e (T5) and 128 are referred to as T5a, T5b, T5c, and T5d. 129 The rst three sensor pairs (T1-T3) ha v e b een in use throughout a long- 130 term exp osure exp erimen t whic h lasted for 731 da ys (Oct. 4th, 2016  Oct. 131 4th, 2018). An o v erview of this exp erimen t is giv en in Fig. 1. The 4th pair 132 7 (T4) has b een in use during the rst half y ear and w as replaced with the 133 5th pair (T5) on April 3rd, 2017 (da y 182) as (i) the T4 sensors exhibit a 134 long resp onse time (not sho wn) and (ii) the opp ortunit y arose to test the T5 135 protot yp es. The T5a sensor stopp ed pro ducing a meaningful signal on June 136 30th, 2017 (da y 270) while T5b b ecame fault y (details b elo w) on August 137 31st, 2017 (da y 332). These sensors w ere replaced with another sensor of 138 the same protot yp e (T5) on Oct. 2nd, 2017 (da y 364). In this last pair, one 139 sensor (T5d) failed within 1 da y (da y 365) while the other (T5c) has b een 140 fully functional un til the end of the exp erimen t. 141 2.3. L ong-term exp osur e exp eriment 142 The sensors are exp osed to the con ten ts of a reactor used primarily to 143 study adv anced con trol strategies for nitrite accum ulation prev en tion in a 144 urine nitri cation pro cess (Thürlimann et al., Submitted). T o this end, the 145 nitri ed urine is pump ed through a closed tub e made from PV C with a o w 146 rate of 43 L/h. The design of this tub e equipp ed with sensor-holding lo c ks 147 is sho wn in the Supplementary Information (Se ction B) . 148 The treated urine is from an throp ogenic origin during the whole exp eri- 149 men tal p erio d. The treated urine w as collected from male la v atories in the 150 F orum Chriesbac h building at Ea w ag, with exception of the p erio d from da y 151 April 30th, 2018 to June 21st, 2018 (da y 574-625), when it w as collected from 152 female la v atories in the same building. F rom Octob er 4th, 2017 to No v em b er 153 24th, 2017 (da y 366 to 417), the reactor w as additionally fed with a nitrite 154 8 0 60 120 180 240 300 360 420 480 540 600 660 720 Time [d]T5dT5cT5bT5aT4bT4aT3bT3aT2bT2aT1bT1aFigure 1: Ov erview of the complete exp erimen tal campaign. The p erio ds of sensor exp osure are indicated b y rectangles. The p erio ds during whic h the sensors pro duced meaningful data are mark ed blac k. sto c k solution. During the exp erimen tal p erio d, the measured concen tra- 155 tions of nitrogen sp ecies in the nitri ed urine ranged b et w een 1180 and 2730 156 mgN/L (mg atomic nitrogen p er liter) for total ammonia, 0 and 82 mgN/L 157 for nitrite, and 1290 and 2720 mgN/L for nitrate. These measuremen ts are 158 copied from Th ürlimann et al. (Submitted) and are sho wn in the Supple- 159 mentary Information (Se ction C) . The pH v alue of the nitri ed urine, as 160 9 measured b y t w o indep enden t and regularly calibrated pH sensors installed 161 directly in the reactor, ranged b et w een 5.7 and 7.3. 162 2.4. Sensor char acterization tests 163 A t regular in terv als, the sensors w ere remo v ed from their normal p osition 164 and exp osed to other media for sensor c haracterization. This w as executed 165 47 times in total. The exact times of these sensor c haracterization tests are 166 listed in the Supplementary Information (Se ction G.1) . T w o pairs of tests 167 w ere executed on the same da y to ensure acceptable exp erimen tal repro- 168 ducibilit y (da y 70: tests 11-12; da y 351: tests 29-30). The selected media 169 include (C4) pH 4.01 calibration solution (CPY20-C10A1, Endress+Hauser, 170 Reinac h, Switzerland); (C7) pH 7.00 calibration solution (CPY20-E10A1, 171 Endress+Hauser, Reinac h, Switzerland); (U4) nitri ed urine at pH 4; (U7) 172 nitri ed urine at pH 7; and (W) tap w ater. F or the presen t w ork, only the 173 exp osure to W, C4, and C7 is relev an t. This o ccurs in v e distinct phases 174 (P0-P4), eac h lasting at least 5 min utes and exp osing the sensors to W, C4, 175 C7, C4, and W in this order. Exemplary results are sho wn in Fig. 2 and 176 discussed in detail b elo w. 177 Ra w p oten tial measuremen ts recorded during P1, P2, and P3 are used 178 to compute the oset ( ~E0) and t w o measuremen ts of the sensitivit y ( ~SD and 179 ~SR ). In line with (Carr, 1993), the follo wing steps are applied for ev ery sensor 180 and ev ery sensor c haracterization test: 181 1. Compute the median v alue among the p oten tial measuremen ts collected 182 10 11:00 11:05 11:10 11:15 11:20 11:25 11:30 11:35 11:40 11:45 11:50 11:55-100-50050100150200P0 (W) P1 (C4) P2 (C7) P3 (C4) P4 (W) 0 mV 177 mVFigure 2: Exemplary sensor c haracterization test. Ra w data obtained in the rst sensor c haracterization test with sensor T1a. The measured p oten tial deca ys during P0, P2, and P4, while it increases during P1 and P3. Steady state is reac hed quic kly in P1, P2, and P3. The theoretical p oten tial v alues for P1, P2, and P3 are indicated with dashed horizon tal lines. Grey shading indicates the data used to obtain the p oten tial measuremen ts (2 to 1 min ute b efore phase c hange). The selected median p oten tial v alues are sho wn with red crosses. in P1, P2, and P3 b et w een 2 and 1 min utes b efore the start of the next 183 phase (P2, P3, and P4). Refer to these v alues as EP1,EP2, andEP3184 2. The sensor oset is de ned as ~E0=~EP2. 185 3. The deca y p oten tial sensitivit y is de ned as ~SD=~EP1~EP2 7:004:01=~EP1~EP2 2:99. 186 4. The deca y p oten tial sensitivit y is de ned as ~SR=~EP3~EP2 7:004:01=~EP3~EP2 2:99. 187 These steps are demonstrated b elo w with a practical example. 188 2.5. Drift mo del 189 The results sho wn b elo w indicate that the oset signi can tly v aries o v er 190 time while the sensitivit y remains remark ably stable in all studied sensors. 191 11 W e describ e the observ ed drift of the oset b y means of t w o mo dels. 192 2.5.1. Mo del 1 - Constant tr end fol lowe d by line ar tr end 193 F or the rst mo del, w e apply a mo di ed v ersion of the excessiv e drift 194 mo del prop osed for the BSM family (Rosén et al., 2008). This mo del sim u- 195 latesE0(t) , the sensor oset, as: 196 E0(t) =do+rdH(ttf) (3) withdo the initial oset, rd the drift rate parameter, H() the Hea viside 197 function (H(a) = 1 ifa0 ,H(a) = 0 otherwise), t the time since sensor 198 installation, and tf the time of the drift onset. The applied mo di cation con- 199 sists of adding the parameter do . T o t this mo del, the oset measuremen ts, 200 ~E0(th) , collected at discrete time instan ts th , are assumed to exhibit inde- 201 p enden tly and iden tically distributed measuremen t errors, h , dra wn from a 202 normal distribution with zero mean and standard deviation,  : 203 ~E0(th) =E0(th) +h; hN(0;) (4) V alues for the 4 parameters do ,tf ,rd , and are obtained indep enden tly 204 for all sensors through maxim um lik eliho o d estimation (MLE). Once cali- 205 brated, the mo del is used to obtain the estimated mean and p oin t-wise stan- 206 12 dard deviations for the sensor oset, 1(t) =E(E0(t)) and1(t) , while using 207 the estimates of tf and as xed h yp erparameter v alues. 208 2.5.2. Mo del 2 - Inte gr ate d Br ownian motion for a single sensor 209 In mo del 2, w e assume instead that the recorded oset measuremen ts 210 are generated b y an in tegrated Bro wnian motion. This is a con tin uous-time 211 sto c hastic pro cess, whic h re ects that the drift rate is sub ject to unmeasured 212 disturbances: 213 _rd(t) = (t)dt; rd(0) =rd;o; (t)N(0; ); (5) _E0(t) =rd(t)dt; E (0) =do; (6) ~E0(th) =E0(th) +h; hN(0;) (7) This mo del also includes 4 parameters: the initial drift rate ( rd;o ); the 214 initial oset ( do ); an input noise standard deviation con trolling the rate b y 215 whic h the drift rate c hanges (  ); and an output noise standard deviation 216 ( ). As with mo del 1, parameter v alues are obtained through MLE. This is 217 ac hiev ed b y form ulating the ab o v e pro cess as a Gaussian pro cess (Rasm ussen 218 and Williams, 2006). This also enables to compute exp ected v alues and 219 asso ciated p oin t-wise standard deviations, 2(t) =E(E0(t)) and2(t) , with 220 the estimates of  and no w used as xed h yp erparameter v alues. 221 13 2.5.3. Mo del 3 - Inte gr ate d Br ownian motion for multiple sensors 222 A third mo del is deriv ed from Eqs. 5-7 b y considering that t w o sensors 223 of the same t yp e ma y b e c haracterized b y distinct initial conditions ( rd;o , 224 do ) but the same noise parameters (  , ). This lead to a mo del with six 225 parameters ( da o ,db o ,ra d;o ,rb d;o , , ), instead of t w o mo dels with 4 parameters 226 eac h. Their v alues are again obtained via MLE and used to obtain calibrated 227 predictions ( 3(t) =E(E0(t)) ,3(t) ) , once again using the estimates of  228 and as xed h yp erparameter v alues. 229 2.5.4. Mo del evaluation 230 The prop osed mo dels are ev aluated through visual insp ection of the mea- 231 suremen ts, predictions, and residuals b et w een the measuremen ts and predic- 232 tions. In the presen t case, suc h a visual insp ection is considered su cien t to 233 select a suitable mo del. 234 2.5.5. Implementation 235 All data collected during the sensor c haracterization tests and all co de 236 necessary to repro duce our results is added in the Supplementary Information 237 (Se ction A) . 238 3. Results 239 3.1. Sensor char acterization tests: Example 240 Fig. 2 sho ws the data obtained in the rst sensor c haracterization test 241 with sensor T1a on Oct. 6th, 2016 (da y 3). The ra w p oten tial measuremen t 242 14 decreases during P0, increases to a steady v alue in P1, decreases to a steady 243 v alue in P2, increases to a steady v alue in P3, and decreases again in P4. 244 The time in terv als used for computation of ~EP1,~E0, and ~EP3(in calibration 245 medium, pH = 4, 7, and 4) are indicated b y grey shading. One can see 246 that the measured oset ~E0is sligh tly b elo w 0 m V (1:30 m V). The v alues 247 for~EP1and ~EP3are sligh tly lo w er than their ideal v alue ( 171:9 and172:4 248 m V). The measured rise and deca y sensitivities are therefore ~SD= 57:73 and 249 ~SR= 57:90 m V p er pH unit. The results of ev ery sensor c haracterization 250 test are visualized in the Supplementary Information (Se ction G.2) . 251 3.2. L ong-term tr ends in the oset me asur ements within the warr anty p erio d 252 Fig. 3 displa ys the measured osets in all sensors throughout the exp er- 253 imen tal p erio d. The recorded v alues collected within the w arran t y p erio d 254 (1 y ear) range from appro ximately 0 m V (no oset) to roughly 70 m V. 255 All commercially a v ailable sensors (T1-T4) pro duce a deca ying trend in the 256 osets. The rstly recorded osets for the T1-T3 sensors are small in magni- 257 tude and concen trate around 0 m V. In con trast, the T4 sensors oset v alues 258 indicate a sho c k eect pro ducing a shift of 20 and45 m V (T4a, T4b) 259 within da ys from installation. This is explained b y the man ufacturer as an 260 eect of the high ammonium concen tration in the medium and should only 261 b e exp ected for this sp eci c t yp e of sensors. The accum ulated drift in the 262 T1 sensors is at most 25 m V after one y ear while the T2 and T3 sensors 263 exhibit an oset of 75 m V after one y ear. Without calibration, this means 264 15 the T1 sensors can pro duce a pH v alue as high as 7:4 when the true pH is 7 . 265 The T2 and T3 sensors will pro duce a pH v alue as high as 8:3 in the same 266 circumstances. Due to failure of T5d, no osets could b e measured for this 267 sensor. The remaining protot yp es (T5a/b/c) do not pro duce a signi can t 268 oset at an y time, except for T5b whic h pro duces a dramatic shift in the 269 oset during three sensor c haracterization tests executed prior to replace- 270 men t. A detailed insp ection of the T5b measuremen ts rev ealed that the rst 271 symptoms of sensor degradation can b e observ ed on August 31st, 2017 (da y 272 332). This is ho w ev er only ob vious when comparing these measuremen ts 273 with the sim ultaneous T1b/T2b/T3b measuremen ts (see the Supplementary 274 Information, Se ction D ). In all cases, except for the T4 and T5a/b pairs, the 275 dierence b et w een osets in sensors of the same t yp e remains rather small 276 with 1 y ear of installation, with a maximal dierence of 16:7 m V recorded 277 with the T2 sensors. T aking the 0:1 pH threshold discussed ab o v e as a 278 guideline, one could prop ose to v alidate and calibrate the sensors when their 279 p oten tial measuremen ts are 5:9 m V apart. This happ ens for the rst time 280 for the T1, T2, and T3 sensors on da y 127, 79, and 309. By these times, 281 the absolute osets are already larger than this accepted threshold so that 282 the relativ e dierence b et w een sensors of the same t yp e is unlik ely a go o d 283 measure to trigger sensor main tenance. 284 Fig. 4 sho ws osets for the sensors T1a, T3a, and T3b collected in the rst 285 y ear of the exp erimen t as a function of the dierence in the oset b et w een 286 T1a and T3a (left panel) and T3b and T3a (righ t panel). The left panel 287 16 0 60 120 180 240 300 360 420 480 540 600 660 720-90-80-70-60-50-40-30-20-10010T1a T1b T2a T2b T3a T3b T4a T4b T5a T5b T5c T5dFigure 3: Oset in all studied sensors as a function of time. V ertical lines indicate a c hange of installed sensors (see Fig. 1). Grey bands indicate a c hange of reactor medium (see Section 2.3). The commercially a v ailable sensors (T1-T4) exhibit drift from the start of installation while the protot yp es (T5) exhibit close to no drift when otherwise functioning prop erly . A signi can t sho c k eect is observ ed for the T4 sensors at the start of the exp erimen t but not for an y other sensor. suggests that oset dierence b et w een sensors can b e predictiv e of the oset 288 in an individual sensor. The righ t panel sho ws that this is less lik ely to b e 289 successful for sensors of the same sensor t yp e, as also describ ed ab o v e. This 290 is considered an imp ortan t opp ortunit y for further researc h, whic h w e discuss 291 further b elo w. 292 3.3. L ong-term tr ends in the oset me asur ements b eyond the warr anty p erio d 293 The oset measuremen ts obtained after the w arran t y p erio d expired ex- 294 hibit t w o phenomena that are surprising (Fig. 3). The rst phenomenon is 295 the rise of the oset of the T1a sensor after 480 da ys of exp osure and a similar 296 17 -10 0 10 20 30 40 50 60-80-70-60-50-40-30-20-100 T1a T3a -10 0 10 20 30 40 50 60-80-70-60-50-40-30-20-100 T3a T3bFigure 4: Oset measuremen ts as a function of relativ e deviations in the oset measuremen ts. L eft p anel: Osets of sensor T1a and T3a as a function of the dierence of these osets. These data are suggestiv e of a close to linear relationship b et w een sensor osets and the oset dierence. R ight p anel: Osets of sensors T3a and T3b relativ e to the dierence of these osets. The dierence in oset remains small and there is no ob vious relationship in this case. rise of the oset of the T1b sensor after 630 da ys of exp osure. Considering 297 that this app ears at distinct times in the lifetime of the T1 sensors, this can- 298 not b e explained as a direct eect of medium comp osition c hanges. Based on 299 information pro vided b y the sensor man ufacturer, this t yp e of drift rate sign 300 rev ersal is unique for the T1 sensors and is unlik ely to b e observ ed with an y 301 other sensor t yp e co v ered in this study . It is the opinion of the authors that 302 the time for this rev ersal is di cult to predict in adv ance. F or this reason, 303 this phenomenon is b est handled as an unmeasured pro cess disturbance. 304 The second phenomenon consists of the rather at to increasing pro le of 305 the oset measuremen ts in the T2 and T3 sensors b et w een da y 360 and da y 306 18 480. Before and after this p erio d, the drift rate in these sensors are visually 307 similar. Giv en the sync hronicit y of this eect b et w een 4 pH sensors, it is 308 h yp othesized that this c hange in the drift rate is in uenced b y the delib erate 309 addition of nitrite in the form of NaNO2 salt to the reactor con ten ts from da y 310 366 to 417. The nitrite addition aected the biomass concen tration and the 311 concen trations of all dominan t nitrogen sp ecies (ammonia, nitrite, nitrate, see 312 Supplementary Information, Se ction C ) and ma y also ha v e aected the ion 313 strength and conductivit y of the reactor con ten ts. Due to this com bination 314 of eects, the a v ailable data only oers an incomplete understanding of the 315 complete c hain of causes and eects b et w een the nitrite addition and the 316 observ ed c hanges in the sensor drift rates. F or this reason, the eects of 317 c hanging media comp osition on the sensor drift rate is b est also considered 318 an unmeasured pro cess disturbance. 319 3.4. L ong-term tr ends in the sensitivity me asur ements 320 Fig. 5 displa ys the computed sensitivit y measuremen ts for the p oten tial 321 rise ( ~SR ) during the complete exp erimen tal p erio d. These measuremen ts 322 do not exhibit strong trends in an y particular direction. The sensitivit y 323 measuremen ts fall b et w een 54:9 and62:1 m V p er pH unit. This means that 324 one can exp ect to measure a pH v alue b et w een 5:95 and6:08 when (i) the 325 true pH v alue is 6 and (ii) an y oset is corrected for. The same graph also 326 sho ws the theoretical v alue of the sensitivit y according to (2) and the recorded 327 temp erature. This pro le is v ery similar to the recorded sensitivit y pro les 328 19 and explains most of the v ariations in the sensitivit y measuremen ts, whic h are 329 small an yw a y . The same conclusions are dra wn from the computed sensitivit y 330 measuremen ts for the p oten tial deca y ( ~SD , see Supplementary Information, 331 Se ction E ). 332 0 60 120 180 240 300 360 420 480 540 600 660 720555657585960616263T1a T1b T2a T2b T3a T3b T4a T4b T5a T5b T5c T5d Figure 5: Sensitivit y measuremen ts for the p oten tial rise as a function of time. V ertical lines indicate a c hange of installed sensors (see Fig. 1). Grey bands indicate a c hange of reactor medium (see Section 2.3). A blac k line sho ws the theoretically ex- p ected sensitivit y computed with (2). V ariations in the sensitivit y are small and follo w the theoretical sensitivit y closely . 3.5. Drift mo dels 333 F or practical in ten ts and purp oses, the sensitivit y  when corrected for 334 temp erature v ariations  can b e considered constan t for the considered pro- 335 cess and sensors. W e therefore fo cus on further analysis of the oset mea- 336 suremen ts. 337 20 The left panel of Fig. 6 sho ws the oset measuremen ts for the T2a and 338 T2b sensor together with the mo del predictions and their con dence b ounds. 339 The righ t panel of Fig. 6 sho ws the prediction residuals. With Mo del 1, 340 the time of the drift onset ( tf ) is alw a ys iden ti ed as a time b efore the rst 341 measuremen t w as obtained (2.1 and 2.3 da ys), suggesting that drift o ccurs 342 throughout the exp erimen t. The same kind of result is obtained with ev ery 343 other commercially a v ailable sensor t yp e (T1-T4), except for the T1a sensor 344 (see the Supplementary Information (Se ction F) ). More imp ortan tly ho w ev er 345 is that Mo del 1 oers a rather p o or description of the data. The con dence 346 in terv als are wide and the residuals are clearly auto-correlated. In con trast, 347 Mo dels 2 and 3 pro vide narro w er con dence in terv als and residuals that do 348 not suggest presence of auto correlation. There are no clear dierences in 349 p erformance b et w een these t w o mo dels so that Mo del 3, whic h has few er free 350 parameters, is preferred. The mo deling results for the T1 and T3 sensors lead 351 to the same conclusions. F or these results and all parameter estimates, w e 352 refer to the Supplementary Information (Se ction F) . F or the T4 sensors, all 353 mo del t yp es deliv ered the same, adequate p erformance. This ma y indicate 354 that (a) the T4 sensors exhibit a drift whic h is in uenced less b y unmeasured 355 disturbances and therefore o ccurs with a close to constan t rate or (b) that 356 the shortened exp osure  6 mon ths in this case  w as to o short to capture 357 the long-term eects of unmeasured disturbances. 358 21 0 90 180 270 360 450 540 630 720 810-140-120-100-80-60-40-20020 0 90 180 270 360 450 540 630 720 810-20-15-10-505101520Figure 6: Mo deling results for the T2 sensors. L eft p anel: Oset con dence b ounds (2 ) obtained with mo dels 1 ( 1;1 ), 2 (2;2 ), and 3 (3;3 ). R ight p anel: Residuals b et w een exp ected v alues (  ) and measured p oten tials ( ~E0). Mo del 1 do es not describ e the data w ell, leading to larger con dence b ounds and auto-correlated residuals. Mo dels 2 and 3 t the data w ell and their predictions are hard to distinguish from eac h other. 4. Discussion 359 This study presen t the rst p eer-review ed results with whic h the eect 360 of long-term w ear-and-tear on w ater qualit y sensors deplo y ed in w astew ater 361 treatmen t plan ts is assessed and ev aluated in a systematic manner and at 362 this scale (12 sensors). The exp erimen tal results rev eal that commonly held 363 assumptions regarding the o ccurrence of sensors faults and fault symptoms 364 are false. First, it is demonstrated that drift in pH sensors o ccurs sim ul- 365 taneously in all commercially a v ailable sensors. Second, it is demonstrated 366 that drift o ccurs as so on as a sensor is deplo y ed in the measured medium. 367 In some cases, the immediate onset of drift is paired b y a signi can t shift in 368 22 the oset. Imp ortan tly , the data needed to compute the osets and sensi- 369 tivities as a function of time are also a v ailable in mo dern pH instrumen ts in 370 the form of a calibration logb o ok that can b e accessed through standardized 371 comm unication proto cols (e.g., Mo dbus). 372 These observ ations ha v e imp ortan t consequences for the dev elopmen t of 373 metho ds for fault detection and iden ti cation (FDI). Indeed, (i) one cannot 374 assume that faults app ear indep enden tly in distinct sensors and (ii) one 375 cannot assume to ha v e access to a fault-free historical data set. Naturally , this 376 also holds in the con text of sim ulation-based b enc hmarking of FDI metho ds. 377 Consequen tly , it is our opinion that the dev elopmen t of FDI metho ds and 378 mo del-based b enc hmarking should b e fo cused on metho ds that do not rely 379 on suc h assumptions. 380 F ortunately , our results also rev eal a n um b er of opp ortunities for the use 381 and main tenance of ion-selectiv e measuremen ts. First, the protot yp e sensors 382 tested in this study exhibit a remark ably stable oset. While these sensors 383 app ear prone to failure, as one migh t exp ect from a protot yp e, this suggests 384 that practically drift-free y et economical pH sensors will en ter the mark et 385 so on. Second, the recorded sensitivit y measuremen ts in all sensors ho v er 386 around the ideal v alues and are remark ably stable throughout the exp erimen- 387 tal p erio d. Suc h a stable sensitivit y lends supp ort for adv anced monitoring 388 and con trol strategies whic h are inheren tly robust to c hanges in the oset 389 but still assume a rather stable sensitivit y (Villez and Hab ermac her, 2016; 390 Th ürlimann et al., 2018a,b). Third, it w as sho wn that the oset dierence 391 23 b et w een t w o pH sensors in the same medium can b e predictiv e of the oset 392 of the individual pH sensors, ho w ev er only if t w o su cien tly distinct sensor 393 t yp es are selected. Com bined with a stable sensitivit y , this means that the 394 deviation b et w een t w o online pH sensor signals could b e used as a pro xy for 395 the deviation in eac h individual sensor. Suc h a pro xy measuremen t could b e 396 v ery useful for remote sensor qualit y assessmen t and predictiv e sensor main- 397 tenance, esp ecially since one can compute suc h deviations b et w een on-line 398 sensor signals while the sensors remain in their normal measuremen t lo cation 399 in the monitored reactor. 400 The obtained oset measuremen ts w ere studied in more detail b y com- 401 paring the t of 3 mo dels. F rom this, it is concluded that the excessiv e drift 402 mo del included in the BSM family (Rosén et al., 2008; Gernaey et al., 2014) 403 cannot adequately describ e the naturally o ccurring drift in ion-selectiv e elec- 404 tro des. Instead, the prop osed sto c hastic mo del, sp eci cally an in tegrated 405 Bro wnian pro cess, deliv ers a go o d description of the obtained data sets. In 406 the authors' opinion, suc h a mo del should b e included in the BSM family for 407 realistic sim ulation of measuremen ts obtained through ion-selectiv e measure- 408 men t principles. The obtained mo del also enables prediction of the exp ected 409 oset measuremen t and asso ciated con dence in terv als b ey ond the last mea- 410 suremen t. This means that suc h a mo del can b e used for predictiv e sensor 411 main tenance, e.g., b y planning a new sensor v alidation and/or calibration 412 b efore the predicted con dence in terv al exceeds a predetermined tolerance, 413 eac h time also up dating the parameters of the sto c hastic mo del. F or this, 414 24 con dence in terv als for the reference p oten tial ( E0) rather than for the mea- 415 suremen ts ( ~E0) are exp ected to b e most useful. Exploring the utilit y of this 416 idea is considered for future researc h. 417 5. Conclusions 418 Despite the abundance of literature of fault detection and iden ti cation 419 (FDI) metho ds, little is actually kno wn ab out the cause-and-eect relation- 420 ships b et w een the exp osure of w ater qualit y sensors to harsh conditions, suc h 421 as w astew ater media, and the o ccurrence of sensor faults and failures. This 422 rst long-term study of the ageing of 12 individual pH sensors giv es v aluable 423 insigh t in to this c hallenge. First, it is concluded that commonly held assump- 424 tions in FDI metho d dev elopmen t and ev aluation, suc h as the a v ailabilit y of 425 fault-free historical data and indep enden t onsets of sensor faults, are in v alid 426 for pH sensors based on the ion-selectiv e measuremen t principle. In addition, 427 the eects of oset drift in redundan t sensors is unlik ely to b e iden ti ed early 428 if these sensors are of the exact same t yp e and exp osed to the same medium. 429 A sto c hastic mo del is sho wn to oer a go o d description of the observ ed drifts 430 of the sensor osets and p erform b etter than a previously established drift 431 mo del. Finally , our results suggest that newly dev elop ed pH sensors whic h 432 exhibit stable osets will en ter the commercial mark et so on. 433 25 6. A c kno wledgemen ts 434 This researc h w as made p ossible b y the Swiss National F oundation (Pro ject: 435 157097). Mr. Ohm ura's con tributions are nancially supp orted b y T oshiba, 436 T oky o, Japan. Dr. Carba jal's con tributions are nancially supp orted b y 437 Ea w ag Discretionary F unds (gran t no.: 5221.00492.011.05, pro ject: DF2017/EMUmore). 438 All sensors w ere pro vided at no cost b y Endress+Hauser. W e thank Daniel 439 Iten and Stefan V ogel from Endress+Hauser for their v aluable advice and 440 input to this study . The sto c hastic mo dels w ere calibrated with the GPML 441 to olb o x v4.2 (Rasm ussen and Nic kisc h, 2005). 442