dataesr/scientific-paragraphs-categorization huggingface dataset:
Surface observations are usually too few and far between to properly assess multidecadal variations at the local scale and characterize historical local extreme events at the same time. A data assimilation scheme has been recently presented to assimilate daily observations of temperature and precipitation into downscaled reconstructions from a global extended reanalysis through an Ensemble Kalman fitting approach and to derive high-resolution fields. Recent studies also showed that assimilating observations at high temporal resolution does not guarantee correct multidecadal variations. The current paper thus proposes (1) to apply the data assimilation scheme over France and over the 1871-2012 period based on the SCOPE Climate reconstructions background dataset and all available daily historical surface observations of temperature and precipitation, (2) to develop an assimilation scheme at the yearly timescale and to apply it over the same period and lastly, (3) to derive the FYRE Climate reanalysis, a 25-member ensemble hybrid dataset resulting from the daily and yearly assimilation schemes, spanning the whole 1871-2012 period at a daily and 8 km resolution over France.
However, due to the low number of daily meteorological observations before the 1950s  (Caillouet et al. , 2019) , these reanalyses are usually limited to the second half of the 20th century  (Minvielle et al. , 2015). This lack of sufficient daily historical observations in many countries in Europe led to the creation of several long-term high-resolution reconstructions. These datasets are mainly built using statistical downscaling of global atmospheric reanalyses  (Dayon et al. , 2015; Minvielle et al. , 2015; Caillouet et al. , 2019; Horton and Brönnimann, 2018) , but in data rich areas, some are also built as an interpolation of surface observations  (Keller et al. , 2015). Published by Copernicus Publications on behalf of the European Geosciences Union. A. Devers et al. :  FYRE Climate: a high-resolution reanalysis of daily precipitation and temperature in France In the past few years, some studies have also developed or used different processes to take advantage of both historical observations and downscaled reconstructions.
In paleoclimate studies in which the propagation step may be typically highly computationally demanding, DA methods have been applied "offline"  (Goosse et al. , 2006; Annan and Hargreaves, 2012; Bhend et al. , 2012; Hakim et al. , 2016; Valler et al. , 2019) : the background is computed by the dynamical model once for the entire period, and the DA comes down to the analysis step  (Matsikaris et al. , 2015). Some recent studies have also attempted to follow the offline DA methodology at higher resolution to assimilate daily observations into various reconstructions. Pfister et al. (2020)  have assimilated daily temperature over Switzerland into a statistical reconstruction, leading to an improvement using only a limited number of stations -25 over all of Switzerland. Devers et al. (2020a)  also developed a DA scheme of daily precipitation and temperature over France into the SCOPE Climate (Spatially COherent Probabilistic Extension Method) downscaled reconstruction dataset  (Caillouet et al.
In short, based on information on large-scale atmospheric circulation from, e. g. , a global reanalysis, SCOPE generates an ensemble of highresolution daily meteorological fields through a resampling of an archive of such fields. Note that the resulting fields from each ensemble member are coherent in space as well as across variables, thanks to the use of the Schaake Shuffle  (Clark et al. , 2004). The application of the SCOPE method using the ensemble mean values of the Twentieth Century Reanalysis  (Compo et al. , 2011)  as a source of large-scale information -predictors -and the Safran reanalysis  (Vidal et al. , 2010b)  as an archive for analogs -predictands -has led to the creation of the SCOPE Climate dataset  (Caillouet et al. , 2019). This daily 25-member ensemble reconstruction is available on a 8 km grid (see Fig.
1 ) over the 1871-2012 period for precipitation  (Caillouet et al. , 2018a) , temperature  (Caillouet et al. , 2018b) , and Penman-Monteith reference evapotranspiration  (Caillouet et al. , 2018c). Note that as SCOPE Climate resamples Safran data, the daily temperature is actually computed as the daily average of hourly temperature. The comparison of SCOPE Climate with the independent Météo-France long-term homogenized series  (Moisselin et al. , 2002, see Sect. 2. 4. 2 for details about this dataset) has put forward a low and steady error -at the monthly timescale -over the whole 20th century  (Caillouet et al. , 2019). Two background ensembles were extracted from SCOPE Climate for this study: for daily DA, the 25-member ensemble of daily values of temperature and precipitation from SCOPE Climate; for yearly DA, the 25-member ensemble of yearlyaverage temperature and the yearly-accumulated precipitation from SCOPE Climate.
Assimilated observations Surface observations originate from the Météo-France database composed of the daily sum of precipitation, and daily minimum and maximum temperature. The observation network has evolved from less than 10 stations of temperature and precipitation in the 1870s to more than 2500 stations for temperature and 4300 for precipitation at the end of the 20th century (Fig. 1a ). The number of stations with a full year of available data evolved in parallel. This large number of observations is partially based on a strong voluntary observation network in France  (Galliot, 2003; Capel, 2009). A. Devers et al. : FYRE Climate: a high-resolution reanalysis of daily precipitation and temperature in France Variables used as observations over the 1871-2012 period are for daily DA: the daily sum of precipitation and the daily mean temperature; for yearly DA: the yearly sum of precipitation and yearly mean temperature for stations with a full year of available data.
The standard deviation for precipitation is selected as the maximum between the percentage value and the minimum value indicated in brackets (in mm). (b) Representativeness of the site  (Leroy and Lèches, 2014). Temperature ( • C) 0 0 0. 5 1 2. 5 Precipitation (%) 0 2. 5 7. 5 12. 5 50 1958 onwards and is updated annually  (Vidal et al. , 2010b). In this study, daily precipitation and daily temperature -computed as the average of hourly values -are extracted from the Safran database over the 1 January 1958-29 December 2012 period. The Safran reanalysis is used here to assess features of the background and the different reanalyses over the last 50 years or so.
Monthly homogenized series (SMR) The monthly homogenized series -called SMR for "Séries Mensuelles de Référence" -are produced by Météo-France. The homogenization is intended to detect and correct potential homogeneity breaks related to changes in location or instrumentation  (Moisselin et al. , 2002; Gibelin et al. , 2014). SMR comprise two different datasets: -1583 time series for precipitation and 308 for temperature allowing for the capture of the spatial patterns over France but only covering the period 1959-2009  (Gibelin et al. , 2014) ; -332 time series for precipitation and 88 for temperature covering the period 1900-2000  (Moisselin et al. , 2002). Although stations are not distributed in a homogeneous way over France, these series constitute a reliable reference for analyzing multidecadal variations as well as long-term trends.
European pattern climatology The monthly gridded reconstructions of precipitation and temperature developed by  Casty et al. (2005 Casty et al. ( , 2007) )  -here called European pattern climatology (EPC) -were created by regressing a network of station data against a modern climate dataset (CRU TS2,  Mitchell and Jones, 2005). Transfer functions via principal component regressions are computed over a recent period where both products are available. Finally, the transfer functions are fed by a limited number of precipitation and temperature stations having a long instrumental record. The reconstruction covers the 1766-2000 period over the North Atlantic and European sector with a spatial resolution of 0. 5 •. It is important to note that this methodology assumes a stationary behavior during the entire period.
Furthermore, nonhomogeneity may appear because the dataset is composed of the CRU TS2 between 1901 and 2000 and of a climate field reconstruction based on principal component regression before 1900. For this study, values are extracted over the 254 cells covering the France area between January 1871 and December 2000 for precipitation and temperature. The EPC reconstruction will be used to evaluate the coherence of the multidecadal variations of the background and the reanalyses over a long period.
Application of the Ensemble Kalman fitting The EnKf described in Sect. 3. 1 is applied here for the two timescales. The FYRE Daily reanalysis is created using the scheme proposed by  Devers et al. (2020a). The assimilation is done independently each day from 1 January 1871 to 29 December 2012 using the 25 SCOPE Climate members of temperature and precipitation as the background. The assimilated observations are daily in situ measurements of temperature and precipitation originating from the Météo-France database. FYRE Daily is thus a daily gridded reanalysis composed of 25 time series of precipitation and temperature fields. The FYRE Yearly reanalysis is produced using yearlyaveraged temperature values and yearly-accumulated precipitation. Once again the background is given by SCOPE Climate, and observations from the Météo-France database are assimilated (see Sect.
, 2015). Note that such a transformation leads to largest changes in higher precipitation values, and that dry days from FYRE Daily will remain unchanged in FYRE Climate. Each member of FYRE Climate is generated as follows. First, the ratio of precipitation β and temperature α where y and d are the year and day considered, D the number of days during year y, c the cell, P and T the value of precipitation and temperature, respectively, with the index defining the dataset considered: daily for FYRE Daily and yearly for FYRE Yearly. Then, the time series of FYRE Climate are computed using the previously defined ratio and the daily time series of FYRE Daily: P climate [d, c] = P daily [d, c] × β[y, c] T climate [d, c] = T daily [d, c] + α[y, c], (13) with notations as above. The climate index refers to the final FYRE Climate values.
Over the 1900-2000 period Most of the comments made above with the most recent SMR dataset are also valid here for the post-1950s period, and a focus is thus made on centennial evolutions. The average bias of temperature between SMR and SCOPE Climate roughly varies between -0. 5 and +0. 5 • C (Fig. 7a ). Before the 1950s, the two reanalyses do not share the same bias characteristics: FYRE Daily shows a slightly positive bias as well as a strong reduction of the ensemble spread after the 1900s, while FYRE Climate shows a strong dependency on the background and an ensemble spread that gradually shrinks over the 20th century. The correlation of the two reanalyses with SMR is clearly linked to the den-sity of assimilated stations (see Fig. 1 ), with slightly reduced values before 1950, and drops during the two world wars.
Multidecadal variability The long-term consistency between different datasets allows for further evaluation of the two reanalyses. To that end, anomalies are computed over the 1871-2012 period for several long-term datasets described in Sect. 2. 4, using the 1900-2000 period as a reference. Anomalies are computed for each cell over France (see Sect. 2. 4 for the number of cells in each dataset). Their median value is retained, smoothed with a 20-year rolling mean, and plotted in Fig. 8. Smoothed anomalies are computed for each member when available. EPC and SMR show a similar evolution of both temperature and precipitation over the 20th century. Negative temperature anomalies are found before the 1940s -from around -0. 35 • in 1910 and down to -0. 6 • in 1880 for EPC -and around the 1970s, and positive ones for other periods, with a steep recent warming from the 1980s onward reaching 0.
Time series analysis Time series over the Cévennes case study cell (see Fig. 1 ) derived from observations, Safran, SCOPE Climate, FYRE Daily, and FYRE Climate are presented in Figs. 9 and 10 to exemplify the behavior of the different datasets at different timescales and for selected periods differing in the amount of data assimilated: 1871-2012 at the annual timescale, years 1900, 1936, and 2000 at the monthly timescale, and June 1900, June 1936, and June 2000 at the daily timescale. For temperature, all long-term datasets are well correlated at the annual timescale, but FYRE Daily values are systematically hotter before 1950. The ensemble spread is rather constant for SCOPE Climate while it is shrinking in the two reanalyses when more observations are assimilated. The underestimation of the recent warming by SCOPE Climate is once again visible here.
Assimilating yearly-averaged observations Assimilating yearly observations allowed for the recovering of multidecadal variations consistent with other products such as SMR and EPC, as opposed to daily-only DA. For precipitation, this gain could be linked to the non-Gaussian properties of daily precipitation -even after anamorphosis -in contrast to log-transformed yearly precipitation. However, it is not so clear why this is the case with daily temperature. Nonetheless,  Steiger and Hakim (2016)  showed that assimilating low-frequency data improves the low-frequency components of reconstructions compared to using only highfrequency data. In addition, most of the observations are assimilated twice in FYRE Climate: through the daily DA, and through the yearly DA. This can be problematic, as it is strongly advised not to assimilate the same observations twice in a DA scheme to avoid overemphasizing observations with respect to the background.
Hence, the choice was made to conduct the validation through comparison with a diversity of products. Even if these products are mainly based on similar observations, the comparison still allow for grasping the overall quality of the product assessed. Furthermore, the study of extreme events (Sect. 4. 5) highlights the crucial value of the literary source when assessing a long-term product both in terms of quantitative or qualitative description. Further comparisons to other shorter products could be conducted, for example, with the E-OBS ensemble dataset available across Europe  (Cornes et al. , 2018). In order to better apprehend the characteristics of the reanalyses, an indirect validation using independent streamflow observations could also be performed through hydrological modeling as done in several studies  (Raimonet et al.
Section 4. 2 also showed better performance (bias, correlation, RMSE) than SCOPE Climate but also than Safran when compared to monthly homogenized time series. Section 4. 5 lastly showed that both reanalyses perform very well in reproducing extreme temperature and precipitation events, which was a weak point in SCOPE Climate  (Caillouet et al. , 2019). All these elements clearly show the benefit of data assimilation for century-long reconstructions. Section 4. 3 highlighted the most important difference between FYRE Daily and FYRE Climate. While FYRE Daily clearly improves on the reconstruction of multidecadal variability as inferred from the SMR and EPC long-term datasets at the scale of France, FYRE Climate displays variations much more consistent with these datasets than FYRE Daily, for both precipitation and temperature, and for all subperiods.
FYRE Climate thus provides the best features, by performing as well as FYRE Daily on small timescales, and much better at longer timescales. FYRE Climate is therefore the final product of this work: a daily surface reanalysis of precipitation and temperature at the daily timescale and at a 8 km resolution over France between 1 January 1871 and 29 December 2012. Moreover, FYRE Climate is an ensemble reanalysis composed of 25 members whose spread reflects the uncertainty in both the reconstruction used as background for DA, and the assimilated observations. As such, it is the first century-long surface reanalysis at a country scale, paving the way for assessing the long-term evolution of climate at the local scale and studying past extreme meteorological events. To this aim, FYRE Climate is made available freely to the research community through two joined datasets: precipitation  (Devers et al.
, 2020b)  and temperature  (Devers et al. , 2020c). Data availability. FYRE Climate is available as netcdf files on the zenodo. org platform. For practical reasons, the dataset is split into one for precipitation (https://doi. org/10. 5281/zenodo. 4005573, Devers et al. , 2020b)  and another for temperature (https://doi. org/10. 5281/zenodo. 4006472, Devers et al. , 2020c). Each dataset comprises 25 netcdf files, one for each ensemble member. Please note that ensemble member #1 for temperature should be associated with member #1 for precipitation, and so on. Values are available over the Safran grid (see  Vidal et al. , 2010a) , but only for grid cells located within French borders, as for SCOPE Climate  (Caillouet et al. , 2019). SCOPE Climate, described in  Caillouet et al. (2019) , is available as netcdf files on the Zenodo platform for precipitation (https://doi.
org/10. 5281/zenodo. 1299760, Caillouet et al. , 2018a) , temperature (https://doi. org/10. 5281/zenodo. 1299712, Caillouet et al. , 2018b) , and reference evapotranspiration (https://doi. org/10. 5281/zenodo. 1251843, Caillouet et al. , 2018c). Author contributions. AD designed the study and the analysis with support of JPV, CL, and OV. AD performed the analysis and drafted the manuscript. All authors revised the manuscript. Competing interests. The authors declare that they have no conflict of interest. Disclaimer. Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Acknowledgements. The authors would like to thank Météo-France for providing access to the Safran surface reanalysis, the monthly homogenized series, as well as to surface observations and associated metadata.
Datasets Our datasets consist of 1,956 individual Device Tree Blob (DTB) files from Linux Kernel version 5. 11. 4  1  and 618 manufacturer-provided System View Description (SVD) files 2  conforming to the Cortex Microcontroller Software Interface Standard. DTB files are standardized descriptions of hardware (CPU and on/off-chip peripherals) for Type-1 embedded systems, agnostic of OS or architecture, and are parsed by an OS kernel during boot to drive hardware initialization. Systems in our Linux dataset range from development boards (e. g. , ARM Versatile Platform Baseboard) to commercial products (e. g. , Nintendo Gamecube). Linux DTB support dates back to 2005  [51]  with usage in PPC kernel ≥ 2. 6 and ARM kernel ≥ 3. 7. The SVD files describe ARM Cortex hardware for Type-2 and Type-3 embedded systems, such as medical devices and mesh network transmitters respectively.
Though OS-agnostic, they describe exclusively ARM architecture systems. SVD files are used during development and debugging to understand the memory-mapped interface between a CPU and an SoC's peripherals  [2]. Prior firmware measurement studies  [10, 19]  used web crawlers to collect publicly available firmware images. These approaches are difficult to reproduce as copyright restrictions prevent corpora distributed and link rot prevents crawlers from running successfully after release. In contrast, kernel source and SVD datasets are largely available and mirrored, making our approach fully reproducible. Appendix B describes our methodology in detail. Our analysis code is publicly available and containerized 3.
Approach We use two corpora to analyze 2,574 real-world hardware configurations. We only consider architectures (DTB corpus) or silicon vendors (SVD corpus) with 10 or more distinct samples. Each sample maps to exactly one real-world embedded system. We quantify: (1) VXE availability by contrasting corpora CPU models against those supported in a mature, open-source HES; (2) VE diversity by measuring the number of unique peripherals in corpora SoCs; (3) VE complexity by measuring complexity of corpora SoCs as a function of peripheral driver code size; and (4) Tractability of HES implementation by simulating VE creation to measure transferable work. All of these analyses use data from our DTB corpus while diversity (2) and tractability (4) are supplemented with information from our SVD corpus.
Complexity: Driver SLOC as a Proxy We can approximate peripheral and SoC complexity for embedded Linux systems by measuring Source Lines Of Code (SLOC) for open-source drivers corresponding to peripherals referenced in our DTB corpus. Driver SLOC is a proxy for complexity: a device driver implements only half of the OS/hardware "conversation. " Each driver interfaces with a hardware peripheral whose internal states and logic may not always be proportionally complex to that of the device driver. Thus, we use SLOC data to provide a rough estimate of the software engineering effort required to build a model of a given peripheral. SLOC computation methodology and an analysis of the correlation between QEMU peripheral vs. device driver SLOC is described in Appendix B. 3. Table  3  shows the mean and standard deviation of SLOC for device drivers and SoCs.
Tractability: Simulation of Emulating Hardware Systems One potential strategy to build an HES that supports a large number of SoCs is to incrementally add support for new hardware components the first time each component is present in an SoC of interest. If such a strategy were to be pursued from scratch, building the HES for the first SoC would require modeling the VXE and all its peripherals. Extending the HES to support subsequent SoCs would require less effort if the VXE and peripherals were used by a prior SoC and thus already supported by the HES. This approach would be viable for generating VEs if, after some initial effort building models for common VXEs and peripherals, the problem were to become significantly easier. By running two Monte Carlo simulations of building an HES to support SoCs from our datasets, we discover that this is not the case and conclude that building or extending HESs is an impractical approach to building VEs for SoCs of interest.
This value is shown in Fig. 2 , across all rounds for each of our simulations. Note that for the final, 195th, Linux system, we still have to implement 11. 4±14. 4 peripherals, on average. These simulations suggest that even if an analyst chose to manually implement thousands of peripherals into an HES, missing peripherals would still be a major roadblock to supporting subsequent SoCs. Note that the simulation's random selection naturally accounts for "most common" peripherals. The simulation results show that peripherals are so diverse that prioritization is not helpful. On average, developing an HES capable of supporting a randomly selected 10% of our SoCs would require supporting 3, 947 ± 159 peripherals for Linux systems or 1, 730 ± 125 peripherals for ARM Cortex systems. By contrast, QEMU 5.
g. , memory corruption) but not hardware vulnerabilities. In practice, emulation-based rehosting techniques often modify the firmware to simplify the rehosting process and, in the process, excise some of the firmware from the VE. Type 1 firmware are commonly rehosted by using a generic kernel combined with the firmware's file-system in order to run user space binaries  [10, 20, 88]. With Type-2 and Type-3 firmware, one rehosting technique is to allow an analyst to manually define models of peripheral behavior at higher abstraction layers  [15-17, 50, 54]. For example, HALucinator  [15]  hooks calls into vendor-specific Hardware Abstraction Layer (HAL) functions and replaces them with Python approximations of the requested hardware functionality. An alternative approach is building peripheral models at lower abstraction layers  [29, 37, 57].