Spatial Interpolation With Kernels | by Arun Jagota | Towards Data Science

California has been on fire recently. Well, actually fires. Fire breeds smoke. That there’s been plenty of.
A heat map revealing the air quality in various regions is clearly useful. What are the modeling issues in building such a heat map? Seriously, entrepreneurs are pondering this question [1].
Well, we do have air quality sensors. But not everywhere. We want to estimate air quality in sparsely covered areas from measurements on nearby sensors. We’ll call this spatial interpolation.
There are also measurement errors. Sensors have varying quality. The expensive sensors are more accurate, but also fewer. So their coverage is sparse. Inexpensive sensors are much more abundant. However, less accurate.
Given these trade-offs, we should combine all data (within sufficient proximity) rather than choosing some over others. Taking into account sensor locations and their varying error rates.
There can also be anomalous readings. Sensors sometimes return whacky results. Or get sick, even die. They can also produce biased results, i.e., those that are consistently off.
Anomalies need detection so they can be discarded. Biases need estimation so they can be corrected. Frequent anomalies from the same sensor are symptoms of malfunction.
All this is interesting, right? Useful as well. Spatial interpolation also applies to temperature, wind speed, amount of rainfall, humidity, to name a few.
Let’s dig in.
Problem Statement
We have sensors located at n points (x1,y1), ..., (xn, yn). We will refer to anyone such a point as ij. The sensor at the location ij emits a reading Sij. We view this reading as a corrupted version of the unknown true value Tij at that location. We seek to estimate Txy at every point (x, y) in the region from these readings.
Example: Say we have two sensors in the San Francisco bay area. One in San Francisco, the other in San Jose. We are interested in estimating air quality in between. (Of course, we only have two sensors, so our estimates may not be great.)
Think of this as one-dimensional spatial interpolation. Estimate air qualities on the line connecting San Francisco to San Jose.
Kernels: Let’s now discuss kernels. To each location ij fitted with a sensor, we attach a kernel Kij. Kij weighs the influence of the reading Sij to locations near ij. The influence decays with distance. Formally, we express this weight as K(d(xy, ij)). Here d(xy, ij) denotes the distance between xy and ij. K(d) decreases with d, with K(0) being 1. A natural choice for K(d) is e^-ad, where a is a parameter that controls the rate of decay of the influence with distance.
Example: In our SF-SJ example, say we set the Kernel K(d) to be 1/(a*d+1) where d is in units of miles. Consider a = 1. The influence of the reading in SF would be 1⁄2 =50% one mile away from SF, and 1⁄4 =25% two miles away from SF.
Why this kernel function, and why a=1 over some other choice? As we’ll see soon, the interpolation is not particularly sensitive to the decisions we make here.
First Interpolator
Consider any point xy. Consider the AQIs { Sij } measured at sensors located near xy. The weighted average of these measurements is a good estimate of the AQI at xy. Sensors closer to xy influence the estimate more.
More formally
S^xy = sum_ij pij(xy)*Sij
where
pij(xy) = K(d(ij, xy))/ sum_i’j’ K(d(i’j’,xy))
Here S^xy denotes the intensity estimate obtained at xy this way.
Example: Consider our SF-SJ example, with the Kernel function we chose earlier. Say SF and SJ are 50 miles apart. Consider a point midway between the two, which we will call San Mateo (SM). We want to estimate the AQI at SM from SF and SJ readings, which we will take to be 160 and 180 AQI, respectively. (AQI is short for “air quality index.”)
K(SM,SF) = K(SM,SJ) = 1/(25+1)=1/26
From this, we get
pSF(SM) = pSJ(SM) = 1⁄2
So our estimated AQI at SM is 170, the average of 160 and 180.
This example also reveals that the interpolation is not as sensitive to the Kernel function's choice and its parameters as we might have imagined. What’s important are the probabilities pij derived from the Kernel function’s values, not the values themselves. We are good so long as the Kernel function helps us estimate the relative influences the readings from the various nearby sensors should have towards the interpolation.
Incorporating Domain Knowledge
Say we have some knowledge about the various sensors. We might know a sensor’s price or brand. From this information, we might be able to estimate its error rate. Even better, perhaps the error rate has already been estimated from tests. Companies selling premium sensors are inclined to do so, to highlight their value propositions.
Below is a simple heuristic extension of pij to leverage this knowledge.
pij = K(d(ij, xy)*wij
Here wij is a positive weight that captures the quality of the sensor at location ij. Higher value equates to higher quality.
With this addition, a low-quality sensor influences S^xy less than a high-quality one. That said, the cumulative influence of multiple low-quality sensors in nearby locations giving similar readings is as if there were a single higher-quality sensor with a similar reading. This is as desired.
Example: Say we set wSF = 2 and wSJ = 1. We trust the sensor at San Francisco twice as much as the one at San Jose. This modifies the probabilities to pSF(SM) = 2⁄3, pSJ(SM) = 1⁄3. Consequently, our AQI estimate at SM is about 166, closer to the reading in SF.
Coping With Anomalous Readings
Sensors may sometimes give anomalous readings. By definition, such readings can be “off the chart.” Left unchecked, they can significantly skew interpolations in their proximity. Clearly, we should detect anomalous readings and discount them.
To do this, we’ll add a new term to pij:
pij = K(d(ij, xy)*wij*aij
Here aij is an anomaly discount weight. Its value is close to 1 if the sensor’s reading is deemed normal and close to 0 if its deemed anomalous.
How can we estimate aij? The basic idea is simple. If most neighboring sensors have similar readings and differ very much from ij’s, ij’s is probably anomalous.
In more detail, obtain S^ij at that sensor based on the readings at neighboring sensors using the formula described earlier. Be sure to exclude ij itself from contributing to this formula. Compare S^ij with Sij. If the two are far apart, the reading at ij is likely anomalous.
This can be enhanced further to take into account the variability in the readings at the neighboring sensors. The more similar the neighboring sensors' readings are, the higher the confidence that the sensor (whose reading differs significantly) is anomalous.
Interestingly, the modeling here is similar to the modeling described in [3], specifically denoising images. The two-sheet architecture described there is interesting to look into in this connection.
Example: Let’s add a new sensor to our SF-SJ example, at SM, midway between the two. Also, let’s revert to trusting the SF and SJ sensors equally.
Say the SM's sensor reading is 5 AQI, whereas the SF and SJ readings were 180 and 160 AQI. The AQI estimate at SM from the SF and SJ readings is 170. 170 is very different from 5. This calls into question the SM reading.
Given this, when estimating the AQI somewhere between SF and SJ (say in Palo Alto) from the three sensors, we might want to discard the SM sensor’s reading.
Detecting Temporal Changes
A sensor may have been reporting similar readings over a time period, then suddenly reports a very different reading. Something happened. If nearby sensors did not report an abrupt change simultaneously, whatever happened is likely local to this sensor.
Using such temporal information and the spatial information, we can often increase the confidence in our inference that the reading is anomalous.
Example: Consider our trio of sensors at SF, SM, and SJ. Say we measured their readings every hour over the past 5 hours and got the following:
Hour         1   2   3   4   5SF reading  159 163 161 166 157SM reading  170 168 174 169   5SJ reading  181 178 172 178 182
Clearly, by looking at the three sensors' readings over the entire duration (not just at hour=5), we can say with higher confidence that the reading in bold is anomalous.
Using such temporal information can also help assess whether a sensor seems biased. Consider a variant of the above example.
Hour         1   2   3   4   5SF reading  159 163 161 166 157SM reading  129 121 125 122 124SJ reading  181 178 172 178 182
Either the SM sensor is biased, or there really is something else going on in SM to account for the consistently lower reading in SM than those in SF and SJ. Either way, this is good to know.
Further Reading
How accurate is AirVisual’s air quality data? | AirVisual Support CenterSpatial interpolationMarkov Random Fields And Image Processing | by Arun Jagota | Aug 2020
How accurate is AirVisual’s air quality data? | AirVisual Support Center
Spatial interpolation
Markov Random Fields And Image Processing | by Arun Jagota | Aug 2020