dataesr/scientific-paragraphs-categorization huggingface dataset:
)Besides considering non-binary units, another natural extension of traditional RBMs is to consider additional hidden layers, as in Deep Boltzmann Machines (DBMs). It is possible to define and train deep TAP machines, as well. Probabilistic DBMs are substantially harder to train than RBMs as the data-dependent (or clamped) terms of the gradient updates (40-42) become intractable with depth. Interestingly, state-of-the-art training algorithms retain a Monte Carlo evaluation of other intractable terms, while introducing a naıve mean-field approximation of these data-dependent terms. For deep TAP machines, we consistently utilize the TAP equations. The explicit definition and training algorithm are fully described in Appendix E. end for t ← t + 1 until t > T VII. EXPERIMENTS VII. 1. Datasets.
consists of both face and non-face 8-bit grayscale 19×19 pixel images. For our experiments, we utilize only the face images. The database contains 2,429 training and 472 testing samples of face images. For our experiments, we normalize the samples to the dynamic range of [0, 1]. binary-MNIST real-MNIST CBCL Nv 784 784 361 N h {25, 50, 100, 500} {100, 500} 256 M 100 100 20 K 100 100 20 Prior Vis. B. Tr. Gauss. -B. Tr. Gauss. Prior Hid. B. B. B. R(•) 2 2 2 γ 0. 005 10 -2 , 10 -5 0. 005 0. 001 0. 001 0. 01 η 0. 5 0. 5 0. 5 TABLE I. Parameter settings for GRBM training.
The author pursued this method  [19]  more thoroughly to include temperature and precipitation, obtaining, on an annual basis, 64 possible typologies (four classes for cloud cover, four classes for wind, two classes for temperature and two for precipitation) of weather types for the period 1991-2010. LWT definitions were based on thresholds that were defined before classifying them and the characteristics of each class were established regardless of the input dataset. They applied this method to 60 meteorological reference stations near urban sites in France. One of the advantages of this approach is that it allows comparison of typologies obtained on distant territories. One of the drawbacks is that the number of classes and the thresholds are fixed a priori and therefore do not allow for optimizing the climatic characterization on a case-by-case basis.
This kind of approach was also applied by Alcoforado in 2004  [20] , which highlights the convenience of the weather type approach in understanding visitor numbers in a tourist area of the Lisbon coast. In their study, on-site meteorological data were measured during 120 field surveys carried out in summertime in 1994, 1995 and 1996. LWT classification was carried out for each survey based on cloudiness (3 classes) and Physiological Equivalent Temperature indicator (3 classes). For the same time periods, two types of indicators of tourist numbers were selected: number of cars parked by the beach, and subjective classification of business by two restaurants/coffee shops proving significant correlation between weather types and turnout factors. The weather type approach was used by  [21]  to determine weather situations relevant for the nocturnal UHI in Hamburg, Germany.
The climatological period of 1998-2008 from the Blagnac weather station located 8 km NW of the city center of Toulouse presented in  Hidalgo et al. (2014)   [22]  is chosen here to illustrate the aims of this paper. The variables involved in the classification were the daily thermal amplitude (Tmax-Tmin, ˚C) and the daily means of specific humidity (g kg -1 ), precipitation (mm h -1 ), wind speed (m s -1 ) and wind direction in four quadrants (1-90; 91-180; 181-270 and 271-360). The objective was to obtain from this dataset a set of clusters representing the plurality of weather situations representative of the on-site local climate. The classification was applied on an annual basis, all seasons taken together, and the seasonal characteristics in terms of diurnal cycle for each variable were interpreted and computed afterwards.
At each iteration, points in the dataset change classes. Classification stops either when there is no exchange between classes, or when the assignment of the points in the dataset to each class stops changing. In these methods (k-means and PAM) the distance measurement determines how the similarity of two elements is calculated when forming clusters. In  Hidalgo et al. (2014)   [22] , the Euclidean distance was used; and to avoid differences in scale among the variables, the standard deviation was used to normalize their values. The PAM method allows a dissimilarity matrix to be used, facilitating the treatment of wind direction as a qualitative variable based on Gower distance as a metric. Both methods produce very close results in terms of number of clusters, general features and frequency distribution, but as centroids are calculated differently, the climatology of cluster number can differ slightly.
Local weather types calculation script The clustering method including the iteration exercise to obtain the optimal cluster number has been coded in an R script and is available in the supporting information materials S1 Appendix. This code is under GPLv3 license. The script requires at least the 3. 3. 3 R version. It was developed with a linux-Ubuntu (32 bit) platform. Some specific libraries are used and are needed at least for this version. In particular, this is the case of the cluster library, which contains k-means and PAM functions. Moreover, for the post-classification analysis, the plotrix library allows wind roses to be plotted. A pre-treatment routine allows application of the main script to any type of data, observed or simulated, and for any time periods.
Applying the local weather type approach to a classic urban boundary layer analysis Through the LWT approach, UBL understanding can be better detailed and results significantly modified with respect to a monthly, seasonal or annual approach. Illustration is done here through a general UBL analysis based on the Surface Energy Balance (SEB) and Urban Heat Island (UHI) intensity evaluation based on urban atmospheric measurements. For that purpose, a dataset from the CAPITOUL field campaign held in Toulouse for a year in 2004 and 2005 was used. Finally, the 3D consistency of the LWT classification is tested through radio-soundings from the same observed database and the diurnal change in the atmospheric boundary layer properties is discussed. The CAPITOUL campaign. The Canopy and Aerosol Particles Interaction in TOulouse Urban Layer (CAPITOUL) experiment was a multi-objective and multi-scale urban climate field campaign developed on the scale of the city of Toulouse as a whole between February 2004 and March 2005.
An extensive description of the observational network can be found in  [33]. For this study, ground station and radio sounding data are used: • Ground station network description The ground station network consisted of: • 21 temporal light stations situated in the city centre and in the suburban areas of Toulouse. They recorded temperature and relative humidity with 12-minute sampling. • 5 synoptic ground stations of Meteo-France that measured the horizontal wind velocities at a height of 10 m. The temperature and relative humidity levels were recorded at a height of 2 m in a 10-km radius from the city centre. Two of these towers, one 15 km south-west (Le Fauga, situated in a grassland area) and the second one 40 km north-west of Toulouse (Saint-Sardos, situated in an irrigated maize field), measured the solar radiation components and turbulent fluxes with a 30-minute sampling.
• A pneumatic tower situated in the old heart of the city was considered the reference station for describing the properties of the city centre of Toulouse. This central 30-m tower was situated on the roof of a building situated 20m above the ground. Temperature, relative humidity, wind speed, pressure, upward and downward global short-and long-wave radiation, and the turbulent flux measurements were recorded at three different heights (47. 5m, 38. 5m, 26m above the ground). Depending on expected wind dusts, the mast height was modified, and four positions are thus defined in the dataset (positions 1 to 4 from low to heavy winds respectively). Here we will focus only on data from position 1, corresponding to 163 days of measurements. • Description of radio soundings Radio soundings were launched during the twenty Intensive Observational Periods (IOPs) corresponding to around thirty days: ten IOPs during winter, three in spring, five during summertime and two during autumn.
For example, in a recent research study presented by Dupont et al. [12]  where how people perceive different landscapes was examined, landscape photographs with discriminated rural-urban gradient (rural, semi-rural, mixed, semi-urban, & urban) were selected. Therefore, surveying the process of visual exploration in UAVs videos requires the utilization of indicative databases, including different environments, different types of UAVs capturing angles, different UAVs altitudes, etc. The majority of the available databases containing aerial video datasets (e. g. ,  [13] [14] [15] [16] [17]  etc. ) have been designed for computer vision purposes, and especially for objects and events detection processes. The main moving objects of such databases are mainly corresponded to humans and cars while in some cases bikes or boats have this role.
Their main environment is semi-urban and the available videos may differ in viewing angles (i. e. , altitude of video capturing), as well as in the available image sequences resolution. Additionally, the available videos in the existing aerial datasets are taken either from fixed point(s) with specific FOV (fixed camera(s) position(s)) or by (onboard) UAV cameras. A recent research study presented by Gunzov et al. [18] , examining the visual search behavior in complex and simulated UAV task environments for training purposes, considers four different types of visual search procedures; target-specific training, cue training, visual scanning training, and control training stimuli. Although the main goals of this study were connected to training procedures, its results highlight the importance of visual search examination towards the optimal effectiveness of design process (in this study related to target training detection).
Moreover, nowadays several existing tools are distributed as open source projects (see for example a list presented by Krassanakis et al. [32] ) supporting the full analysis of eye tracking data in simple steps (e. g. , LandRate toolbox  [33] ). At the same time, several recent studies based on eye movement analysis extend the typical examination of two-dimensional (2D) visual stimuli by examining visual stimuli with dynamic content (i. e. , video) under specific tasks or not (e. g. ,  [34] [35] [36] [37] [38] ). The aim of the present study is to examine how people perceive UAVs videos characterized by a set of different and representative parameters. These parameters are connected with the UAV's flight altitude, the main surrounding environment, the presence of sky in the background, as well as the main perceived angle between UAV flight plane and ground.
Visual Stimuli The process of the experimental stimuli design was based on the use of multiple videos adapted from the UAV123 Database  [15]. UAV123 is an up-to-date dataset containing UAV videos captured in several environments (cities, parks, sea, virtual environments, etc. ). Additionally, different types of objects (people, cars, bikes, boats, etc. ) are depicted in the available videos of this database. Considering also that this dataset is (mainly) consisted of high resolution videos, it can be served as a quite suitable source of representative UAV videos. Hence, taking also into account that the total duration of the experimental process must be kept relatively short in order to eliminate possible errors due to observers' fatigue, a part of the videos (totally 19 different videos) was selected from the aforementioned database.
At the same time, this selection was made on the basis that different and representative specifications characterize the selected videos. More specifically, videos specifications were identified qualitatively (by watching all available videos of the database) based on the main UAV altitude, the main presented (surrounding) environment, the size of the main presented object (for the cases where it was obvious there was a dominant object during the video), the presence of sky (at least once during the video), as well as the main (perceived) angle between UAV flight and ground plane. Hence, this information constitutes a product of human annotation (made by authors) and it is not a result of a computation. The specifications of the selected videos are presented in Table  1. Additionally, some basic statistics for the selected videos are presented in Table  2  in comparison with the complete UAV123 dataset.
For the computation of the statistics that are related to video durations (Tables  1  and 2 ), the specific frame rate of 30 frames per second (FPS) is considered (this value is also reported by UAV123 dataset description  [15] ). In Figure  1 , an indicative frame of each selected video is illustrated in order to highlight the variability (in terms of existing differences in the presented environment) among the experimental videos. Additionally, some basic statistics for the selected videos are presented in Table  2  in comparison with the complete UAV123 dataset. For the computation of the statistics that are related to video durations (Tables  1  and 2 ), the specific frame rate of 30 frames per second (FPS) is considered (this value is also reported by UAV123 dataset description  [15] ).
In Figure  1 , an indicative frame of each selected video is illustrated in order to highlight the variability (in terms of existing differences in the presented environment) among the experimental videos. The experimental process was performed in five successive sets (the duration of each set corresponds to approximately 3 min) in order to ensure the accuracy of the collected data (calibration and validation process were implemented for each set, see also Section 2. 1. 4 for further description). All videos were presented randomly in their native resolution (1280 × 720 px (720p)); for each observer, a unique dataset was produced by concatenating the corresponding sequences. Since the resolution of the used monitor was higher (see Section 2. 1. 2) than the resolution of videos, a grey frame (R:198, G:198, B:198, see also Section 2.
Equipment and Software The EyeLink ® 1000 Plus (SR Research Ltd. , Ottawa, ON, Canada) eye tracker system was used during the experimental process. Binocular gaze data were collected with a recording frequency of 1000 Hz while the eye tracker system was used in remote mode (using a 25 mm camera lens and without using any head stabilization mechanism or chinrest). Hence, normal (not abrupt) observers' head movements were allowed during the observation of the experimental visual stimuli. The spatial accuracy of the eye tracker system, as reported by the manufacturer for the selected mode (remote), lies in the range 0. 25 • -0. 50 • of visual angle. Additionally, the selected eye tracker equipment is fully compatible with corrective eyeglasses and contact lenses. For visual stimuli presentation, a typical 23.
75 times of monitor's width for the proper function of the eye tracker) as well as the recommendation provided by ITU bt. 710-4  [40] ; observation distance has to be equal to 3xH, where H corresponds the screen height when considering the value of acuity threshold of human vision that corresponds to approximately one minute of the visual angle. Moreover, the distance between stimuli display monitor and eye tracker camera was equal to 43 cm. The whole experimental process was programmed in MATLAB software (MathWorks ® ) using Eyelink toolbox  [41]  (included in Psychophysics Toolbox Version 3 (http://psychtoolbox. org/)) towards the communication between eye tracker's installed PC and display PC. For videos presentation, the open source MPC-HC (https://mpc-hc. org/) media player was selected, since this software constitutes a quite lightweight and customizable player (the communication between experimental script and video player manipulation was achieved using relative in-house (LS2N) MATLAB functions).
Moreover, light conditions were fully controlled using another in-house (LS2N) MATLAB script, installed in a separated PC. Furthermore, for the purposes of data synchronization based on the corresponding time stamps (between gaze data and video player) and synchronized data export from the recorded files, appropriate scripts were developed in Python, while the whole statistical analysis as well as the production of heatmap visualizations (see Section 2. 2) based on the collected eye tracking data were performed in MATLAB. Finally, the generation of visual stimuli videos based on the existing sequences was implemented using the multimedia framework FFmpeg (https://www. ffmpeg. org/). The overall orientation/geometry of the equipment used for the performance of the experimental study is depicted in Figure  2.
Quantitative Analysis Eye tracking metrics were calculated for all observers and all presented videos. In Table  3 , the average values, as well as their standard deviations, are presented for each video separately. Moreover, the overall averages and their standard deviations of all eye tracking metrics taking into account the values produced for all UAV videos and all observers were calculated. The corresponding results are depicted in Table  4. Moreover, box plots (Figures  3 4 5 ) were generated for each eye tracking metric in order to highlight the existing variation among the examined UAV videos. The provided box plots depict the minimum (down edge of the dashed line of the box), the maximum (up edge of the dashed line of the box) of the corresponding interquartile ranges and the median (red line in the box) values, the 25th (down edge of the blue box) and the 75th (up edge of the blue box) percentiles, as well as data outliers (red "+").
Dataset Distribution The collected raw gaze data, as well as the analyzed fixation and saccade events were organized in a new dataset, called EyeTrackUAV, which is freely distributed to the scientific community via anonymous FTP at ftp://ftp. ivc. polytech. univ-nantes. fr/EyeTrackUAV. Additionally, EyeTrackUAV contains all the heatmap visualizations that were produced in the framework of the presented study.
Discussion and Conclusions The outcomes produced by the computation of the average eye tracking metrics values (Table  4 ) considering gaze data of all observers during the observation of all examined UAV video may constitute characteristic evidences about how people observe this special type of stimuli and can be served as an objective ground truth for possible comparison with other types of dynamic stimuli. The robustness of the produced values is also validated by considering that the standard deviations that are connected with the majority of the reported values are, in general, small, despite the huge variability of the observed UAV videos. Additionally, among the reported metrics' values, the metrics derived by fixation events pose some interesting results. In particular, the results show a high number of fixation events per second (1.
This issue is rational, since the used dataset (UAV123) has been mainly developed for computer vision purposes connected with object detection algorithms. At the same time, the observation of the moving point objects is also confirmed while considering the qualitative analysis of the generated heatmaps. Moreover, the video datasets used for the performance of the experimental study can be ranked based on the calculated eye tracking metrics values presented in Table  3. The outcome of this ranking process is summarized and presented in Table  7. Table  7  is directly connected with the metrics produced for each video and compared qualitatively through the visualization of box plots depicted in Figures  3 4 5. Table  7  can only serve as a first benchmark for possible differences (based on the specific eye tracking metrics) among the used video datasets.
The importance of these systems is obvious when considering their direct influence in human safety, and cultural or ecological protection procedures. Finally, although that recent experimental studies validate that low cost eye tracking devices can be used for scientific purposes (see e. g. ,  [66] [67] [68] ), the raw gaze data, which are distributed through the EyeTrackUAV dataset, have been collected with one of the most precise and accurate eye trackers, serving as a robust ground truth for future studies.
Future Outlook The present study constitutes, to the best of our knowledge, a first attempt to monitor human visual behavior process during the observation of UAVs videos visual stimuli towards understanding the leading factors that may influence this procedure. Although that the examination of human visual behavior during the observation of UAVs videos stimuli can be based on the exploitation of different solutions, an existing database (UAV123) was used for the purposes of the presented study. Such solutions may include the capturing of new UAV videos or the production of virtual demos using simulation and/or geographic information tools. Despite that in both cases stimuli design process could be more effective, the use of an existing and up-to-date database, such as UAV123, allows the future comparison of the produced results with existing data (e.
This is largely due to their respective high auto-correlations at the annual time-scale. Hence, as mentioned by the reviewer, using this kind of proxy record indeed brings a lot of reconstruction errors due to overestimated weights, dating uncertainties, but also, because they induce erroneous validation scores as the link between these proxy records and the NAO index is overestimated. Concerning the dating uncertainty, it is also present in annually-resolved proxy records and this aspect is not accounted for in the present version of the code. (3) Following this comment we have updated our code, manuscript and data with the use of the 2017 version of the Pages 2k database as suggested by Reviewer 2. Then, using this new proxy database, and in order to address this comment, we decided to remove the proxy records that are not annually resolved.
(1) p. 18 l. 1-2 Or the "significant" correlation with the NAO could be spurious. Also note that non-stationarity violates one of the fundamental assumptions of these (and nearly all) reconstruction approaches. (2) Indeed, we also ask ourselves if the significant correlations we found could be spurious but it is relatively difficult to determine whether they are or not. An indirect way to "verify" this significance of correlation is the location of the proxy records that have high correlations with the NAO. A way to rule out spurious correlation is the use of pseudo-proxies like in Ortega et al. (  2015 ), but handling pseudo-proxies from different datasets was an arduous task for this multimethod paper. Nevertheless, the fact that most proxy records selected for the highest levels of correlation significance (i.
How do the reconstructions compare given the uncertainties? (2) (3) As mentioned above, in the revised version we use the coefficient of efficiency to validate our reconstructions and we include and discuss regression uncertainties in our main text and dedicated figures. -Response to Anonymous Referee # 2: 1 Scientific Comments (1) I'll start with what I like about the paper: it applies several methods to the same dataset, and the results are fairly consistent among methods and with another recent reconstruction, in which one of the authors was involved  (Ortegal et al, 2015). That's about it. (2) (3) We thank the reviewer for this positive comment. Nevertheless, as a general response to the main reviewer's criticisms below, we would like to highlight that our study is proposing novel regression methods that have, to our knowledge, not yet been applied to climate signal reconstructions.