## 5 Evaluation
### Localization Error Estimator  
To evaluate the effectiveness of representation, we firstly examine its performance in estimating localization error. We build three types of location error estimators, as described in Section 4.3.1: DNN-R, DNN-C, RNN. They share the same loss function as the mean squared error between the regression result and true GPS error. DNN-R consists of four layers with 32, 16, 8, and 1 neuron(s), respectively, and takes ReLU as its activation function. DNN-C has a similar structure to DNN-R, with four layers, but the layer with eight neurons has an additional activation layer of soft-max, aiming to output the error range classification for eight classes. The errors are divided into eight bins, and each has a size (error range) of eight meters. The single neuron in the last layer takes the expectation for all error bins. RNN shares a similar structure as DNN-R except that the first three layers are recurrent neural layers with GRU cells. We train the three models using representations generated from our autoencoder with the training traces, and test them against those generated with the testing traces. To compare with CRBM embedding, as discussed in Section 5.2.2, we conduct the same procedure using the CRBM embedding model. The mean and median errors for all testing data are shown in Figure 6. DNN-R performs the best out of the three models, achieving a median difference of 3.65 m and a mean difference with the true errors of 5.18 m. Our model generally performs better than CRBM, except for RNN, but both models perform the worst in this case, as they tend to over-fit the sequence pattern. To showcase the change of errors along a trace, two sample traces are also shown in Figure 7, along with the comparison between the true error sequence and the estimated one, based on our model using DNN-R. Both sequences generally capture the same error trends, though the magnitudes thereof may not perfectly match each other. Figure 5: Autoencoder parameters (in quantity) vs. reconstruction error (unitless). To further evaluate the performance of our representation in estimating GPS error, we compare it with the error estimates provided by the Android system. Instead of an absolute estimated error range, both the Android system [37] and Google [38] (yes, they are different APIs) provide an indicator called _horizontal accuracy_, defined as the radius of 68% confidence. In other words, there is a 68% probability that the true location is inside the circle centered at the reported location, with a radius indicating accuracy. To make our results comparable with it, we transform our absolute estimated error into a similar \"accuracy indicator\" by obtaining the 68% percentile over the error distribution within the preceding 10 s.  
Based on the derived \"accuracy indicator\" for our model and the \"horizontal accuracy\" from the Android system, we compute the probability that the true GPS error is smaller than each accuracy range within a sliding window. A probability close to 68% implies an adequate accuracy indicator for localization performance, and one larger than 68% implies an over-conservative estimation, while one smaller than 68% implies an over-confident estimation. We also measure the probability with different sliding window sizes. Figure 8 reports the mean and median probabilities over all testing trips for different sliding window sizes, and it makes a comparison among accuracy indicators derived from our model, Android [37], and Google [38]. All models give a relatively stable median probability and mean probability over different window sizes, indicating that the estimators perform in a consistent manner. However, the indicator delivered by our model remains, almost always, around 68%, while the other two indicators always give over-confident estimations. This suggests that our model yields a more reliable location accuracy indicator, as compared with the other two systems. Figure 6: Deviations from true localization error. Figure 7: Trace samples (in second) with error estimation (in meter)