dataesr/scientific-paragraphs-categorization huggingface dataset:
IV. EXPERIMENTS In this section, we present parameters settings for the proposed framework and the corresponding results. We equally give some quantitative and qualitative analysis of the obtained results to show the advantages and eventual limits of the proposed approach. All the algorithms were tested on 6 cores Intel(R) Core(TM) i7-6850K CPU 3. 60GHz with 32 Gb of RAM computer with a NVIDIA Titan X GPU with 12 GB of RAM and developed in Python 3 programming language using PyTorch 1. 3 library on Ubuntu 16. 4. tslearn library  [35]  was used to calculate DTW distance matrices for concurrent approaches. Source files related to the project are available in the GitHub repository  3.
A. Experimental settings The chosen datasets have some differences in temporal resolution and images quality. For this reason, the proposed framework have some dissimilarities based on the particularities of each dataset. Due to the high temporal gap between the images of Montpellier dataset, we tend to extract the maximum of the information from it, so all extracted change sequences are kept. On the other hand, for the Rostov dataset, we keep only the change sequences that are at least 3 timestamps long. As the average temporal resolution between two images is less than two months, we consider that 2 timestamps change sequences most probably correspond to some false change alarms that could not be corrected when introducing the multi-temporal context. For the Rostov dataset we manually apply a water mask and a mask for a part of the city with dense constructions.
While the first mask is applied to eliminate FP changes caused by boats, the second one is indispensable to lower the number of FP changes caused by the shadows from tall buildings and changing luminosity that can not be corrected. The water mask was obtained with Sen2Cor 4  classification module. Firstly, the water surfaces were detected for every image of the dataset individually. Secondly, the pixels that were marked as water for more than 50% of the images of the dataset were added to the water mask. Dense urban area mask was created manually and has almost a rectangular shape that cover a specific city area. The last difference is related to the fact that the images of the second dataset are not perfectly aligned and some of them have one pixel size shift from the origin in different directions.
This shift could not be eliminated with automatic techniques and, for this reason, we have decided to use bigger patches comparatively to Montpellier dataset for bi-temporal change detection. 1) Bi-temporal change detection: The parameters for bitemporal change detection were set accordingly to  [24]. For the Montpellier dataset, we have chosen the patch size p = 5 and the convolution AE model for feature translation as they were proved to give the best results for 10 m resolution images  [24]. Due to the shift presented in Rostov dataset, we have increased the patch size to p = 7 in order to minimize the shift influence on patch translation. We have equally chosen fully-convolutional AE model for feature translation as it gives better results than convolutional AE for bigger patch sizes  [24].
The AE models for both datasets are presented in Table  I. For both datasets we use only 3 bands for the change detectionred, green and NIR -as they contain the essential information about the images. Before applying Otsu thresholding we equally exclude high v % of the highest reconstruction error values considering they correspond to some noise and extreme outliers. The variation of the high v only balances precision/recall values, without influencing kappa value. The higher high v is -the higher is the recall and the smaller is the precision, and vise versa. To balance these values, we set high v to 0. 5% for both datasets. The analysis of the influence of high v value on different metrics is detailed in the following subsection. The following quality criteria were used to evaluate the performances of the different approaches: precision  (10) , recall (11) and Cohen's kappa score  [36].
P recision = T rueP ositives T rueP ositives + F alseP ositives Recall = T rueP ositives T rueP ositives + F alseN egatives (10) To evaluate the performance of the proposed methods on two datasets, we have used ground truth maps extracted for different parts of each dataset. For the Montpellier dataset, the overall ground truth area was 600000 pixels, while for the Rostov dataset this area was 780000 pixels. The ground truth maps were manually created for the chosen areas by comparing the objects presented in the couples of images by one of the authors who is the field expert. 2) Image segmentation: To segment the extracted change areas, we have used a tree-merging bottom-up segmentation algorithm  [28]  described earlier. The segmentation parameters were chosen empirically and allow us to produce large segments for both SITS without mixing different classes in one segment.
For each dataset, we have manually chosen reference adjacent objects that have to be divided in separated segments. These objects need to have similar spectral properties, but still be easily distinguished by human eye. We set the same parameters for single image segmentation and for the segmentation of concatenated images, so the change objects extracted from concatenated images are relevant to single-image objects. The segmentation parameters for both datasets are the same and equal σ = 0. 1, k = 7, min size = 10. 3) Multi-temporal change detection: When the preliminary change maps with bi-temporal contextual anomalies are obtained, we move on to their analysis in multi-temporal context. t int should have the same value as τ 1 , because both parameters define the level of spatial relation between segments at different timestamps.
In other words, t int defines if changes detected between different timestamps belong to the same change process and τ 1 defines if a candidate object belongs to a reference BB. If we set t int smaller than τ 1 and a candidate change polygon is kept after multi-temporal context analysis, it may not be attached to any BB during evolution graph construction. On the contrary, if t int is set much higher than τ 1 , it will produce mostly spatially compact change processes, so small τ 1 value is unnecessary. For both datasets, we have set t int =0. 4, as τ 1 =0. 4 with corresponding α gave the best results comparatively to other parameters combinations for both datasets (explained in the next subsection). Logically, the performance of the proposed approach fully depends on segmentation quality.
The approach was qualitatively validated using manually selected reference objects. 4) Evolution graph construction: As it was mentioned before, in this work, we tend to create evolution graphs that contain only the objects that belong to the same change process. In the previous works, α value varies between 0. 3 and 0. 5. Smaller α values provide more overlapping graphs and, on the contrary, if α is high, we may obtain low graph coverage ratio, hence, a lot of change processes will be missed. To obtain the optimal BB coverage and graph overlapping, we choose α values 0. 4 and 0. 5 for the Montpellier and Rostov datasets respectively. As every object can belong to only one evolution graph that ensures the maximum possible spatial overlay with corresponding BB, we propose to omit τ 2 value and to set τ 1 value excessively small.
The smaller τ 1 is, the bigger graph footprint we might obtain. However, most evolution graphs are compact independently of τ 1 value. For both datasets, τ 1 =0. 4 provided the best graph coverage. τ 3 value has to be chosen after visual analysis of detected changes and parasite objects that correspond to false positive changes. Clearly, its value should not be too high as the footprint of change processes may have important variations at each timestamp. The parameters for evolution graph construction that gave us the best results are presented in Table  VI.
5) Feature extraction autoencoder: To cluster the evolution graphs, it is necessary to extract robust descriptors of change objects. In our work, we deploy a deep convolutional AE model for non-supervised feature extraction. We use the same model for both datasets (see Table  II ) with patch size p=9 as it provides sufficient coverage of neighborhood pixels without high influence of objects border pixels. Contrary to the change detection algorithm, we keep all 4 spectral bands for the Montpelier dataset (Green, Red, NIR and SWIR) as SWIR band may ameliorate the clustering of some similar classes. We keep B=3 for the Rostov dataset as in change detection. We encode the patches of the Montpellier and Rostov datasets in feature vectors of size 5 and 4 respectively. Small size of feature vectors was chosen to obtain only the most representative features.
Moreover, feature vectors size is equal to the number of spectral bands plus 1 (f = B + 1) to avoid potential linear correlation between a spectral band and a corresponding element of the feature vector. 6) Evolution graph clustering: After the texture extraction, we create synopses for all evolution graphs that are then passed to the GRU AE model combined with a chosen clustering method. As we do not know beforehand the exact number of change classes presented in a SITS, we choose hierarchical agglomerative clustering algorithm. Most of the existing clustering algorithms either demand a number of clusters as a parameter, or a set of parameters that will further define the clusters number for a given dataset. In most cases, this set of parameters is difficult to guess and varies from one dataset to another.
Often the initialization of the algorithm depends on the chosen number of clusters, hence, clustering model may change drastically. At the same time, data clustering models built by the hierarchical agglomerative algorithm do not depend on a number of clusters as at every step of the algorithm execution, it iteratively merges the two clusters with the highest likelihood value. Given these arguments, hierarchical agglomerative clustering is a good compromise between clustering quality and model complexity and makes the best choice for our generic change clustering approach. The model parameters were set as follows for both datasets: hidden size=150, f hidden=20. Both parameters were achieved empirically, though some general recommendations were followed to set the second parameter. For example, the size of AE bottleneck should correspond to the number of researched clusters  [37] , although, this number is not known beforehand, we can distinguish around 10-15 the most frequented types of change processes for both datasets.
To improve the clustering results, the extracted sequences are normalized by their mean and standard deviation. After the model is stabilized, we apply the hierarchical agglomerative clustering to the encoded results. As parameters for clustering algorithm, we use Ward's linkage  [23]  and Euclidean distance between the encoded points. We test the results for different number of clusters in the range from 5 to 50. Unfortunately, the right choice of the number of clusters is still an open question. No robust methods for the automatic selection of the number of clusters were find in literature and for some datasets it seems impossible to guess it. However, we can visually estimate the number of prevailing change classes and set an optimal range of the number of clusters for the hierarchical clustering.
Considering the specificity of the algorithm and that its model does not depend on the selected number of clusters, we do not need to know their exact number. Furthermore, in the case of this paper, we consulted experts in cartography specialized in these areas that gave us an idea of what number of clusters and which cluster-class mapping should be expected. Our method can easily be reused in other geographic areas, with the right number of clusters found by trials and errors, or better with the help of an expert in the specific type of landscape studied. For both datasets, we have annotated the most frequented of detected change processes to evaluate the clustering results. Note that we do not dispose of the labels for different crops, so the vegetation is annotated accordingly to its variation in the detected change sequence (for example, vegetation-weak vegetation-empty field-vegetation) and that we do not know the exact number of change process classes.
For the Montpellier dataset, the annotated change processes that were regrouped in 10 classes are: 1) 5 different vegetation variations (3644, 4429, 4307, 2542 and 2206 pixels per annotated class), 2) changes in deep water (8164 pixels), 3) construction in a dense area (4122 pixels), 4) construction of a new area (13648 pixels), 5) beginning of the construction process (bare soil/construction field) (2196 pixels), 6) changes in buildings luminosity (false positive changes) (501 pixels). For the Rostov dataset we got 11 classes: 1) 5 different vegetation variations (2864, 16930, 14805, 7521 and 6183 pixels), 2) construction of a new building (969 pixels), 3) construction of a new area (dense construction) (13669 pixels), 4) construction of a new residential area (sparse small private buildings) (241 pixels), 5) changes in bare soil (2201 pixels), 6) seasonal changes in trees/bushes growing areas (8970 pixels), 7) shadow from tall buildings (FP changes) (3708 pixels).
More details about the algorithms performance and its comparison to the state-of-art methods can be consulted in  [24]. The advantage of our method is that comparatively to the state-ofthe-art algorithms, our method is less sensitive to the changing luminosity (especially in the urban area), hence, produces less false positive change alarms. It equally gives us change objects of more "natural" shape, as we perform change detection for every pixel of the image separately by associating it to its neighborhood patch. We observe that both datasets have high recall values that indicate that most of the changes are correctly detected. At the same time, precision value is low for the Rostov dataset and depends mostly on the quality of the images itself (a lot of saturated objects and shadows are detected as changes).
As it was mentioned in the previous subsection, we set high v =0. 5% for our method to balance the precision and recall values. The influence of high v value on the precision, recall and kappa values is presented on Figure  8. It was decided that the optimal threshold high v should provide the results with recall slightly better than precision, as some FP changes will be deleted during the next steps. We observe that for the Montpellier dataset high v = 0. 5% gives us the best result. At the other hand, we notice different behavior of metrics values for the Rostov dataset ground truth and we can state less smooth changes in these values. We equally state that the best results are obtained for high v ≥ 0. 2%. Therefore, it was decided to set constant high v = 0. 5% for our method to ensure the most optimal results for both datasets.
Once the obtained change maps are analyzed in multitemporal context, we move on to the construction of the evolution graphs. Table VIII presents the evolution graph statistics for the chosen parameters that ensure the best coverage. It provides total graph coverage, graph overlapping percentage, minimum graph compactness represented by the ratio of BB footprint to the total graph coverage footprint, dataset average graph compactness, percentage of graphs with compactness inferior to 50% and 75% (to justify the choice of small τ 1 value). The Figure  9  presents an example of an evolution graph that corresponds to the construction of a rugby stadium. The constructed graph correctly reflects the overall change process, although, we can observe that some extracted changes and corresponding segments are not "pure".
For example, in timestamp 18/02/2006, some small vegetation objects are attached to larger construction segments. We can also notice that the presented change process is composed of several subprocesses. One of these subprocesses starts at 03/06/2006 and at this timestamp is presented by segment 7 -29 that corresponds to some vegetation fields that are later transformed into constructions. With the given evolution graph parameters, we obtain 4388 graphs for the Montpellier dataset and 1850 graphs for the Rostov dataset (after excluding 2 timestamps length sequences) that are used in totality for the AE model training. To evaluate the proposed clustering framework, we compare it with hierarchical agglomerative clustering with DTW  [19]  distance measure used in  [17]  that is equally able to process time series data with varying length.
DTW is a time-series distance measure algorithm that finds optimal match between two time series by scaling one to another. Ward's linkage was used as clustering algorithm parameter. DTW was applied to the extracted graph synopsis calculated for encoded features (called "DTW features") and to the raw image values (called "DTW w/o features"). We highlight the advantage of our proposed framework by comparing it with the same framework without feature extraction (i. e. we compute the synopsis for raw image values, called "Framework w/o features"). The obtained results are presented in Table  IX  for different number of clusters in order to choose the optimal one. For both datasets we equally calculate NMI for primary change classes such as changes in vegetation, construction processes and changes in water (only for the Montpellier dataset) to verify if the extracted clusters contain only one primary type of changes (Table  X ).
The execution time of GRU AE combined with hierarchical agglomerative clustering and hierarchical agglomerative clustering with DTW distance matrix measure are presented in Table  XI. Computation of DTW distance matrix is a time consuming process. hence, with growing number of change graphs, the computation time may drastically increase. At the same time, GRU AE training takes the same time for both datasets, regardless the number of graphs. We observe that for small number of classes (10-15) our proposed framework gives the best results for the reference ground truth (10 and 11 classes for Montpellier and Rostov datatsets respectively). However, for the Montpellier dataset, hierarchical agglomerative clustering with DTW measure outperformed our framework for the number of clusters superior to 15.
At the same time, primary classes of the Montpellier dataset are much better separated by our framework, even for small number of clusters. In general, for both datasets, NMI increases with the number of clusters, but at the same time, it may be complicated to interpret the obtained results when its number is elevated. In our case, as the NMI is calculated at pixel and not object level, we obtain high NMI values for 11 reference classes of the Rostov dataset due to the bigger surface of vegetation changes. As DTW measure does not depend on the classes balance, it has achieved better separation of primary classes for the unbalanced dataset. Nevertheless, for large datasets, DTW computation is time-consuming, so GRU AE is the most adapted for balanced datasets of any size, and DTW approach performs better for unbalanced datasets of a small size.
For this reason, we may find different classes of objects at the same timestamps in one evolution graph (as in Figure  9 ). This class mixture may potentially influence the graph synopsis and lower the performance of our clustering algorithms. If we lower the threshold k or increase σ values for the image segmentation, we obtain smaller segments that give us more compact evolution graphs. At the same time, the computation time for evolution graph construction will increase with the number of obtained graphs. In this case, we may obtain numerous graphs that correspond to some subprocesses that will decrease the clustering performances due to the lack of generalization of the obtained change graphs. For example, if we change only the segmentation parameters for the Montpellier dataset to σ = 0.
V. CONCLUSION In this paper, we have presented an end-to-end approach for change detection and analysis in SITS. The approach consists in the extraction of bi-temporal change maps for the whole SITS that are then analyzed in multi-temporal context in order to construct different change processes that are further regrouped in different clusters. Our framework combines graph based techniques with unsupervised feature learning with neural networks and does not depend on the temporal resolution of SITS and on its length. The proposed approach is fully unsupervised and gives us promising results on two real-life datasets. However, for unbalanced datasets, we may observe that smaller classes are not well separated from the majority ones. Later, we are planning to work on more robust graph construction and interpretation.
Moreover, we have deleted the first three images of the dataset because of the irregular temporal distance between them (distance between 1st and 2nd image is around one month, 2nd and 3rd -4 months, 3rd and 4th -5 months). The average gap between two consecutive images does not exceed 1. 5 months, minimum -20 days, maximum -5 months (corresponds to winter with deleted snow-covered images). Distribution of dataset images is presented in TableV.
Contents This is a citation summary for Chemokine receptors in the Guide to Pharmacology database (GtoPdb). It exists purely as an adjunct to the database to facilitate the recognition of citations to and from the database by citation analyzers. Readers will almost certainly want to visit the relevant sections of the database which are given here under database links. GtoPdb is an expert-driven guide to pharmacological targets and the substances that act on them. GtoPdb is a reference work which is most usefully represented as an on-line database. As in any publication this work should be appropriately cited, and the papers it cites should also be recognized. This document provides a citation for the relevant parts of the database, and also provides a reference list for the research cited by those parts.
Dincer et al. [9]  first use PCA then, a variational autoencoder for the dimensionality reduction and a LASSO to predict drug response to leukemia. Hanczar et al. [10]  use a multi-layer perceptron to predict cancers and pretrain each layer with a denoising autoencoder and a large unlabelled dataset. One of the main concerns of deep learning in medical applications is its lack of interpretability. Neural networks can be viewed as black boxes, where the gene expression profile of a patient is put to its input layer and a prediction is obtained from its output layer without providing any explanation on the decision process. The need for making deep neuronal networks more interpretable is therefore increasing, especially in the medical field for mainly two reasons. Firstly, it is important to ensure that a neural network bases its predictions on reliable representations and is not focusing on an artifact of the data.
The aim of all these studies is to identify potentially interesting genes related to the disease of interest. However, they do not explain what the network does, or what represents a neuron, or what representation of the patient is constructed in the hidden layers. Very few works tried to interpret the hidden neurons and almost all of them are based on the analysis of the values or the distribution of connection weights of the learned neural network  [16]. Way et al. [8]  analyzed the decoder's connections of their variational autoencoder and associate each neuron to the set of genes with the highest absolute values of weight. Based on these gene sets, they applied an enrichment analysis to identify overrepresented pathways and GO biological process terms to each neuron. In  [16] , authors built denoising autoencoders and stacked denoising autoencoders to extract important genes from cancer gene expression dataset.