dataesr/scientific-paragraphs-categorization huggingface dataset:
When λ = 1, (9) boils down to its counterpart of (5) in batch setting. In the next two sections, we describe the two proposed algorithms for solving (9) under CP and Tucker decompositions. We make the following four assumptions for the convenience of deploying our algorithms as well as analyzing their performance. (A1). Observed tensor slices {X t } t≥1 are independent and identically distributed from a data-generating distribution, which is the underlying distribution of the dataset, having a compact set V. This assumption is very common for convergence analysis in online settings in general and adaptive tensor decomposition in particular, e. g. [8, 9, 46, 47]. Naturally, it holds in several scenarios, for instance, real-life data are often bounded such as image, video and audio. 4 (A2). Tensor slices {X t } t≥1 follow the data model  (7)  where the true underlying loading factors {U (n) t } t≥1 are bounded, i.
Evaluations In this section, experiments are conducted to evaluate the performance of the two proposed algorithms (ACP and ATD) on both synthetic and real data. We also compare them with several state-of-the-art algorithms to provide practical evidences of their effectiveness and efficiency. All experiments are implemented on MATLAB a windows computer with an Intel Core i5-8300H and 16GB of RAM. We refer the readers to the supplementary document for further experimental results. 8.
Performance of ACP We assess the performance of ACP w. r. t. the following aspects: (i) impact of algorithm parameters on its tracking ability; (ii) performance of ACP in nonstationary and time-varying environments; (iii) effectiveness and efficiency of ACP as compared with other adaptive CP algorithms. Experiment Setup: According to the setup of OLSTEC  [9] , a timevarying model for streaming tensors is constructed as follows. At t = 0, the loading factor U (n) t is generated at random whose entries are i. i. d. drawn from the Gaussian distribution N (0, 1). At time t > 0, U (n) t ∈ R In×r is varied under the model U (n) t = U (n) t-1 Q t , where Q t ∈ R r×r is a rotation matrix to control 8 Our MATLAB codes: https://github. com/thanhtbt/Tensor Tracking. the variation of U (n) between t and t -1, which is defined by Q t =     I pt-1 0 0 0 0 cos(α t ) -sin(α t ) 0 0 sin(α t ) cos(α t ) 0 0 0 0 I r-pt-1     , (40) where p t = mod(t + r -2, r -1) + 1 and α t is the rotation angle.
Video Completion In this task, four real video surveillance sequences are used (including Highway, Hall, Lobby and Park 10  ) and we refer the readers to the supplementary document for more experimental results on other real-world datasets. Specifically, Highway contains 1700 frames of size 320 × 240 pixels. Hall has 3584  We compare the proposed algorithms with OLSTEC  [9] , TeCPSGD  [8]  and CP-PETRELS  [10]. We set the value of λ at 0. 7, 0. 001 and 0. 999, respectively, for OLSTEC, TeCPSGD and CP-PETRELS. Besides, OLSTEC and TeCPSGD are also depended on the regularization parameter µ which value is fixed at 0. 1. Performance of these algorithms is shown statistically in Tab. 2 and graphically in Fig. 10. We can see that ATD outperforms adaptive CP algorithms in almost all tests. ACP also provides reasonable estimation accuracy on these data as compared to others.
Multichannel EEG Analysis We follow the experimental framework in  [64, 65]  to illustrate the use of ACP for analyzing multichannel EEG signals. In this task, we use a public EEG dataset collected on 14 subjects during the stimulation of hands 11. The EEG signals are recorded using a system of 64 channels (electrodes) and we have 28 measurements per subject in total. We construct three-order EEG tensor of measurement × channel × time-frequency by taking continuous wavelet transform to each EEG channel. Note that, the resulting time-frequency representations are reshaped into vectors of length 4392 and hence being streamed. In a nutshell, we have the EEG tensor whose size is 28 × 64 × 4392 and its rank is set at 3 as provided in  [64, 65]. At each time, data of 20 (and 40) channels are supposed to be discarded randomly for our missing observation purpose.
Conclusions In this paper, we have proposed two new low-complexity algorithms, namely ACP and ATD, for adaptive decomposition of higher-order incomplete and streaming tensors. Developed based on CP decomposition, ACP estimates a multilinear LRA of streaming tensors from noisy and high-dimensional data with high accuracy, even when the decomposition model may change slowly with time. In parallel, developed based on Tucker decomposition, ATD is a fast randomized tracker, able to recover missing entries from highly incomplete observations. Leveraging the stochastic approximation and the uniform sampling technique, ATD has been shown to be one of the fastest Tucker algorithms, much faster than the batch algorithms while providing good estimation accuracy. Experimental results have indicated that ATD outperforms stateof-the-art adaptive CP algorithms as well as the proposed ACP one in the application of video completion.
[11]  designed a system with four magnetometers to evaluate the MF strength in each direction and used the nearest neighbor method to classify MF measurements with 12 components. The MF measurements of two different floors could be distinguished. The effects of smartphones, watches, laptops, elevators, and workbenches on the intensity of the MF were analyzed. Their system used chest-hung magnetometers instead of smartphones, which, in reality, do not have four magnetometers to measure the magnetic fingerprints in all directions. Filter-based methods (e. g. , Hidden Markov Model (HMM), Kalman filter, particle filter) have many indoor magnetic field positioning applications. Filter-based methods can fuse data from multiple sensors to provide higher positioning accuracy. For example, the work of Basmag  [63]  used a Backward Sequence Matching Algorithm (BSMA) to optimize the HMM and improve the low discriminability of the magnetic signal with the help of PDR.
Thirdly, although several research works have given solutions for heterogeneous smartphones, it is not clear whether the calibration parameters will change over time and whether they need to be calibrated at every point in space. Fourthly, uncalibrated magnetic field measurement varies with the orientation of the device. Magnetic field measurements in a specific direction can only be matched to the magnetic field fingerprint in that specific direction. Finally, the above studies used homogeneous smartphones and did not use fingerprint databases from heterogeneous smartphones when testing the positioning algorithms. This study aims to provide an exhaustive analysis of magnetic field-based indoor positioning and experimentally explore the feasibility and challenges of magnetic field-based indoor positioning.
The heterogeneity of smartphones is caused by the different characteristics of magnetometers, making it very difficult to design a positioning system that can seamlessly connect various smartphones  [26, 32]. This paper investigates the MF measurements of the Android and iOS devices. All smartphones used MATLAB Mobile software to collect MF measurements to prevent the effects of software differences. In Section 4. 1, we study the statistical characteristics and Gaussianity of heterogeneous smartphones. In Section 4. 2, we compare the MF trajectories of heterogeneous smartphones, and in Section 4. 3, we test the MF characteristics of smartphones when they are rotated. In Section 4. 4, we design a magnetic field acquisition device using the Arduino Pro Mini and the LSM9DS1 and analyze the original magnetometer's statistical characteristics.
Rotation Test Magnetometers are often fused with inertial sensors for pose estimation. Accurate MF measurements are essential for determining the user's heading and orientation. The construction of an MF map would be a tedious task if the MF measurement was dependent on orientation. MF measurements in various orientations at a specific point have been investigated. A smartphone was placed on a rotatable platform, as shown in Figure  5 ; the smartphone frame was aligned with the platform frame, and the platform was rotated around the z axis of the smartphone. The MF database was collected at a sampling frequency of 100 Hz for 25 s. MF positioning is based on the stability and uniqueness of the MF signature. In the positioning phase, smartphone measurement should match the MF database. The location with the shortest distance has the highest probability.
Static Tests with Magnetometer The previous analysis used a commercial smartphone with already pre-processed MF measurements by the manufacturer. An MF acquisition device, shown in Figure  8 , was designed with the Arduino Pro Mini and 9-DoF LSM9DS1  [90]  to study the unprocessed MF measurements; the software was developed using the open-source code available on the Arduino website  [91]. Six designed devices (referred to as D1, D2, D3, D4, D5, and D6) were placed in a warehouse of width 45. 69 m and height 49. 5 m in the city of Tour, France. They continuously collected magnetic measurements in stationary mode for 9 days until the battery ran out. The robust locally weighted scatterplot smoothing (RLOWESS) method has been applied for outlier rejection  [92]. Indeed, robust estimation methods generally involve the detection and mitigation of outliers, e.
We calculate the soft iron A and hard iron b of the smartphone at a given point and a given date. We tested these transformations (i. e. , A and b) over a period of time to see if periodic re-calibration is necessary. Moreover, we tested the same calibration parameters at different points to see if they depend on the ambient location or only on the considered smartphone. To accomplish these goals, three smartphones (iPhone Xs Max, Huawei P9, Bluebird) have been used to continuously collect MF data at 10 points from February to June 2020 in 2 buildings; point 1 to point 7 are in building 1; point 8 to point 10 are in building 2. Smartphones were rotated around X, Y, Z, respectively, for 120 s for the aim of calibration. Figure  10  shows the calibration results at P1 on 7 February 2022, where the MF measurements of these smartphones are found to be spherical.
This means that their distortion matrix approximates the identity matrix and the bias is negligible. Meanwhile, there is a significant bias for Bluebird (shown in Figure  10c ), which needs to be eliminated. We calculated calibration parameters using MF measurements of P1 from three smartphones on 7 February 2020. The obtained calibration parameters were applied to Equation (  14 ) to obtain true MF measurements at different times. m n = R nb A -1 m b -b -ε. ( 14 ) The rotation matrix and bias obtained at point P1 on the first day were used to calibrate all other data to demonstrate that the rotation matrix and bias of the MF are independent of time and space and only relate to the internal parameters of the smartphone. Figure  11a, c, e  show the original MF magnitude of the iPhone Xs Max, Huawei P9, and Bluebird.
Dimensionality reduction techniques such as principal component analysis (PCA)  [96]  and singular value decomposition (SVD)  [97]  can transform high-dimensional features into low-dimensional features, significantly reducing the storage space and computational complexity of magnetic fingerprint-based positioning. The architecture of a magnetic-based positioning system is shown in Figure  13. The system consists of two phases: online and offline  [24]. First, raw magnetic data are collected at a reference location using a smartphone. Then, during the data pre-processing step, lowpass filters, smoothing filters, and calibration algorithms are applied to remove noise and bias from the smartphone's MF measurements. Thirdly, the pre-processed data construct a fingerprinting database corresponding to each location.
The fingerprinting database is trained using machine learning methods (e. g. , kNN, support vector machines, decision trees, naive Bayes, discriminant analysis) to obtain predictive models. Finally, a prediction model is used to predict the test data's position. We will focus on fuzzy kNN, support vector machines, decision trees, naive Bayes, and discriminant analysis in the following. This experiment used three different smartphone models (iPhone Xs Max, Huawei P9, Bluebird) to test the location discernibility of local magnetic fingerprinting. Among them, Bluebird is an industrial custom smartphone for location-based services. The test area is a corridor in the 3rd floor of Polytech Galilée, University of Orléans (shown in Figure  14 ); zones 1, 2, and 3 each contain 10 sampling points. Firstly, the user stands for 5 s at each point, holding a smartphone to capture a magnetic fingerprint database of 30 points with 100 Hz sampling frequency.
Next, the user stands for 1 s at each point to capture test data and calculates the average of these 100 samples; this process was repeated five times to obtain 5 test datasets. Finally, we obtained n = 15,000 samples of training data X training = {x 1 , x 2. x n | x i } ∈ R n×3 , and k = 150 samples of test data  In the experiments, the test and training sets were collected in the same direction (along the corridor direction). The sampling interval between two successive points is set to 1. 5 m. The height of a smartphone is set at 1. 05 m distance from the ground. X test = x 1 , x 2. x k | x j ∈ R k×3 ,.
Classification Result The test zones are shown as three rectangles in Figure  14 , with 10 test points in zones 1, 2, and 3, respectively. The green area is the elevator, which is a source of magnetic interference. Figure  15  shows the "calibrated" MF intensity at 10 points in zone two as an illustrative example to visually analyze the MF magnitude fingerprints in this zone. The Huawei device has a more significant variance in the MF intensity than the other two smartphones, with more overlap between the signals. Bluebird's database shows that there is an overlap between P12, P15 and P20, P18, and P19. The iPhone has a slight overlap between P12 and P13. First, we test the training and test sets from the same smartphone separately, and the results are shown in Table  5. The positioning accuracies of the iPhone Xs Max and Bluebird are relatively similar (close to 80 or above), basically meeting the needs of essential indoor positioning.
12 µT, and 0. 21 µT, respectively. As the intensity of the magnetic field varies smoothly between 23 µTesla and 62 µTesla  [87] , the larger the variance, the more likely it is to overlap with other signals within a limited variation interval, which degrades the positioning performance. Due to the significant variance of the MF measurement, Huawei's positioning accuracy is lower than the other two smartphones. Next, we use the iPhone Xs Max for the training set to obtain the prediction model and test the other two smartphones separately. Table  6  illustrates the positioning accuracy. We can see that the positioning accuracy decreases when the magnetic fingerprinting database and the test data are inconsistent compared to Table  5. We already saw earlier that the Bluebird's MF measurement is significantly different from the other two smartphones.
Incremental learning enables artificial agents to learn from sequential data. While important progress was made by exploiting deep neural networks, incremental learning remains very challenging. This is particularly the case when no memory of past data is allowed and catastrophic forgetting has a strong negative effect. We tackle classincremental learning without memory by adapting prediction bias correction, a method which makes predictions of past and new classes more comparable. It was proposed when a memory is allowed and cannot be directly used without memory, since samples of past classes are required. We introduce a two-step learning process which allows the transfer of bias correction parameters between reference and target datasets. Bias correction is first optimized offline on reference datasets which have an associated validation memory.
Authors of BiC  [33]  use a validation set which stores samples of past classes to optimize parameters. Instead, we learn correction parameters offline on a set of reference datasets and then transfer them to target datasets. The method is thus abbreviated TransIL. The intuition is that, while datasets are different, optimal bias correction parameters are stable enough to be transferable between them. We illustrate the approach in Figure  1 , with the upper showing the IL process with a reference dataset. A memory for the validation samples needed to optimize the bias correction layer is allowed since the training is done offline. The lower part of the figure presents the incremental training of a target dataset. The main difference with the standard memoryless IL training comes from the use of a bias correction layer optimized on the reference dataset.
Its introduction leads to an improved comparability of prediction scores for past and new classes. Note that the proposed method is applicable to any class IL method, since it only requires the availability of raw predic-Figure  1 : Illustration of TransIL, our proposed method, depicting states from 1 to s for a reference and a target dataset. The model M is updated in each state with data from new classes. States from 1 to s -1 are faded to convey the fact that knowledge learned in them is affected by catastrophic forgetting. The class IL process is first launched offline on the reference dataset where adBiC, our proposed bias correction layer, is trained using a validation memory which stores samples for past and new classes. Class IL is then applied to the target dataset, but without class samples shared across states since a memory is not allowed in this scenario.
The set of optimal parameters of adBiC obtained for the reference dataset is transferred to the target dataset. This is the only information shared between the two processes and it has a negligible memory footprint. The transfer of parameters enables the use of bias correction for the target dataset. The final predictions obtained in state s are improved compared to the direct use of M s predictions, since the bias in favor of new classes is reduced. tions provided by deep models M s. The second contribution is to refine the definition of the bias correction layer introduced in  [33]. The original formulation considers all past classes equally in the correction process. With  [23] , we hypothesize that the degree of forgetting associated to past classes depends on the initial state in which they were learned.
Consequently, we propose Adaptive BiC (adBiC), an optimization procedure which learns a pair of parameters per IL state instead of a single pair of parameters as proposed in  [33]. We provide a comprehensive evaluation of TransIL by applying it to four backbone class IL methods. Four target datasets with variable domain shift with respect to reference datasets and different numbers of IL states are used. An improvement of accuracy is obtained for almost all tested configurations. The additional memory needs are negligible since only a compact set of correction parameters is stored. Code and data needed for reproducibility are provided 1.
Related work Incremental learning is a longstanding machine learning task  [10, 22, 30]  which witnessed a strong growth in interest after the introduction of deep neural networks. It is named differently as continual, incremental or lifelong learning depending on the research communities which tackle it and the setting of the problem. However, the objective is common: enable artificial agents to learn from data which is fed sequentially to them. Detailed reviews of existing approaches are proposed, among others, in  [3, 20, 23, 25]. Here, we analyze works most related to our proposal, which tackle class 1 https://github. com/HabibSlim/DKT-for-CIL IL and keeps memory and computational requirements constant, or nearly so, during the IL process. We focus particularly on methods which address bias in favor of new classes  [23]  and were designed for class IL with memory.
The maintenance of both discrimination and fairness is addressed in  [34]. The ratio between the mean norm of past and new class weights is applied to the weights of new classes, to make their associated predictions more balanced. Bias Correction (BiC)  [33]  exploits a supplementary linear layer to rebalance predictions of a deep incremental model. A validation set is used to optimize the parameters of this linear layer, which modifies the predictions of the deep model learned in a given incremental state. We tackle two important limitations of existing bias correction methods. First, they are inapplicable without memory because they require the presence of past class samples. We propose to transfer bias correction layer parameters between datasets to address this problem. Second, the degree of forgetting associated to past classes is considered equivalent, irrespective of the initial state in which they were learned.
The utility of distillation in a class IL scenario was recently questioned. It is shown  [23, 27]  that competitive results are obtained if a fixed-size memory is allowed for large-scale datasets. The distillation component is removed in  [23]  and IL models are updated using fine-tuning. A simpler approach is tested in  [27] , where the authors learn models independently for each incremental state after balancing class samples. The usefulness of distillation was also challenged in absence of a memory  [2]  where standardization of initial weights (SIW), learned when a class was first encountered, was proposed in  [2]. The freezing of initial weights was tested in  [23]  and also provides significant improvements. It is thus interesting to also apply the proposed approach to methods which do not exploit distillation.
Our method is globally inspired by existing works which transfer knowledge between datasets. We mentioned knowledge distillation  [13]  which is widely used in IL. Dataset distillation  [32]  encodes large datasets into a small set of synthetic data points to make the training process more efficient. Hindsight anchor learning  [6]  learns an anchor per class to characterize points which would maximize forgetting in later IL states. While the global objective is similar, our focus is different since only a very small number of parameters are transferred from reference to target datasets to limit catastrophic forgetting on the latter.
Class-incremental learning formalization We adapt the class IL definition from  [5, 14, 28]  to a setting without memory which includes a sequence of S states. The first one is called initial state and the S -1 remaining states are incremental. A set of P s new classes is learned in the s th state. IL states are disjoint and P i ∩ P j = ∅ ∀i, j ∈ 1, S , i ̸ = j. A model M 1 is initially trained on a dataset D 1 = {(X j 1 , Y j 1 ) : j ∈ P 1 } , where X j 1 and Y j 1 are the sets of training images and their labels. We note N s the set of all classes seen until the s th state included. Thus, N 1 = P 1 initially, and N s = N s-1 ∪ P s = P 1 ∪ P 2 ∪. ∪ P s-1 ∪ P s for subsequent states. M s is updated with an IL algorithm A using D s = {(X j s , Y j s ) : j ∈ P s }. D s includes only new classes samples, but M s is evaluated on all classes seen so far (j ∈ N s ).
Adaptive bias correction layer The unavailability of past class exemplars when updating the incremental models leads to a classification bias toward new classes  [33, 34]. We illustrate this in Figure  2  (left) by plotting mean prediction scores per state for the CIFAR-100 (a) LUCIR  [14]  (b) LwF  [21]  Figure  2 : Mean prediction scores and standard deviations for CIFAR-100 classes grouped by state at the end of an IL process with S = 10 states, for LwF and LUCIR, before (left) and after (right) calibration with T ransIL. dataset with S = 10 splits using LUCIR and LwF, the two distillation-based approaches tested here. Figure  2  confirms that recently learned classes are favored, despite the use of knowledge distillation to counter the effects of catastrophic forgetting. New classes, learned in the last state, are particularly favored.