dataesr/scientific-paragraphs-categorization huggingface dataset:
A range of vegetation phenology datasets are available, varying in spatial and temporal extent and resolution  16. On-the-ground observations of individual organisms have been made by citizen science observers for decades, and contributed to databases such as the U. S. National Phenology Network  17. At much broader spatial scales, satellite-based remote sensing platforms (e. g. , Moderate Resolution Imaging Spectroradiometer (MODIS), Landsat, and Sentinel 2) provide time-series of land-surface greenness (e. g. , normalized difference vegetation index), allowing for characterization of vegetation phenology across the earth's entire surface  18, 19. However, landscape heterogeneity is unresolved under relatively course pixel resolutions provided by satellites sensors (e. g. , ≈500 meters in MODIS), potentially confounding phenological signals  20, 21.
Phenocams are also being deployed as part of the NEON (National Ecological Observatory Network) and LTAR (Long Term Agricultural Research) networks in the USA, ICOS (Integrated Carbon Observation System) in Europe, TERN (Terrestrial Ecosystem Research Network) and the Australian Phenocam Network in Australia, and the e-Phenology project in Brazil  16, [31] [32] [33] [34]. The previously-released PhenoCam Dataset v1. 0 28 is a curated and publicly available (CC0 Public Domain Dedication) data set that includes both digital camera imagery (https://doi. org/10. 3334/ORNLDAAC/1560)  35 and data derived from that imagery (https://doi. org/10. 3334/ORNLDAAC/1511)  36. This initial public release of data from PhenoCam included imagery through the end of 2015 from 130 cameras, comprising almost 750 site-years of data  28.
Here, we describe a major update-v2. 0-to the PhenoCam Dataset, with two significant improvements. First, coverage has been substantially increased. The new dataset includes imagery through the end of 2018 from 393 cameras, comprising 1783 site-years of data (see Fig. 1 ). Second, data and imagery have been screened to exclude cameras programmed for "grey world" automatic white-balancing (AWB), as AWB can negatively impact the quality of the derived time series. In addition, we also present here an analysis of the representativeness of the current distribution of PhenoCam network cameras, to identify ecoregions and climate zones that are under-represented.
Data Records The PhenoCam Dataset v2. 0 consists of five different Data Records. Data Record 1 contains general metadata for each camera site, whereas Data Records 2 through 5 have been calculated for specific ROIs from each camera. For example, Data Record 2 contains the files used for image processing steps, i. e. ROI mask files, and information about the range of images over which these should be applied. Data Record 3 contains time-series of color-based statistics (e. g. , chromatic coordinates), calculated for each image in the archive. Data Record 4 contains "summary product" 1-day and 3-day time-series for a variety of phenologically-relevant color-based metrics. Data Record 5 contains phenological transitional dates (i. e. , phenophases) obtained from the 1-day and 3-day summary time-series.
csv • <sitename>_ <veg_type> _ <ROI_ID_number> _3day_transition_dates. csv Here, <sitename> is the name of each camera site, < veg_type > is a two-letter code defining the type of vegetation for which data have been processed (see Table  1 ), and < ROI_ID_number > is a unique identifier to distinguish between multiple ROIs of the same vegetation type for a given site. Together, these five data records are contained within Seyednasrollah  39  , and are derived from the imagery in Milliman  40. The structure of the data records in the PhenoCam Dataset v2. 0 is identical to that of the PhenoCam Dataset v1. 0, and users of the Dataset v2. 0 are directed to the user guide (https://daac. ornl. gov/VEGETATION/guides/ PhenoCam_V2. html) and our previous data descriptor  28  for an explanation of the headers and columns in each data record.
technical Validation Efforts to maintain quality control. The PhenoCam image archive and derived data products are regularly reviewed by the PhenoCam project team (the authors of this Data Descriptor) to maintain the quality of the dataset. This consists of visual inspection of the imagery for each site, and of the G CC time series data derived from the imagery and displayed in near-real time on the project web page. Imagery from each PhenoCam is also checked for field of view shifts, interruptions to data continuity, and camera misconfigurations. Should any issues be identified, site contacts are notified by email, and asked to investigate and implement corrective measures if possible. In the case of field of view shifts, ROI masks are adjusted or redrawn as needed, and the imagery is reprocessed to ensure that the effects of field-of-view shifts are minimized  28.
As a result, the chromatic coordinates obtained from AWB images may not be suitable for quantifying vegetation phenology: the data may be noisy  42  , or even wrong. Figure  2  shows an example of how AWB can affect both the digital images themselves, and the extracted G CC time-series, using imagery from the Snipe Lake PhenoCam site. In 2011, the Snipe Lake camera was set to fixed white balance, whereas in 2017 the camera was set to AWB. The purple cast of the sky, visible in the 2017 image (Fig. 2a ), is the direct result of AWB compensating for the greenness of the foreground vegetation. The white balance setting also influences the derived G CC time series, which is not only noisier for the 2017 (AWB) imagery than the 2011 (non-AWB) imagery, but also leads to mischaracterization of vegetation seasonality: the rise in G CC in March 2017 is due to snowmelt, rather than vegetation greening up.
This configuration is implemented by the PhenoCam Installation Tool (PIT; http://khufkens. github. io/phenocam-installation-tool/). Because of the negative impact of AWB on PhenoCam imagery and data products, we have implemented several procedures to identify whether a given camera may be recording imagery using the "grey world" AWB model. For standard PhenoCam cameras (Type I), configured using the PIT, the metadata text file associated with each Table  1. Vegetation type abbreviations for ROIs (region of interests), and the corresponding number of siteyears of data in the PhenoCam dataset described here (v2. 0). For comparative purposes, the number of siteyears of data in the previous dataset release (v1. 0) is also presented. The absence of MX ROIs in the v2. 0 data release is due to the fact that we have delineated separate ROIs for the plant functional types comprising the mixed stand (i.
e. , separation of EN and DN ROIs). Note that non-vegetated ROIs are not included in the v2. 0 data release. image reports whether the camera is on manual (i. e. fixed) color balance ("balance = 0") or AWB ("balance = 1"), and these files are scanned regularly to identify cameras which have been erroneously set to AWB. For non-standard cameras (Type II and Type III), we have developed an ad hoc method to identify potential AWB cameras. Briefly, we define the deviation from grey, Δ, as in Eq. 2: Here R DN , G DN , and B DN are the average red, green and blue (respectively) digital numbers, calculated across the entire image. As shown for the Snipe Lake imagery in Fig. 2c , Δ tends to be very close to zero when cameras are on AWB. We identify images with Δ < 0. 02 for more than 30 consecutive days as "AWB suspects" and conduct further investigation.
In some cases, imagery from cameras not on AWB has low Δ because the image is dominated by neutral tones-when the ground is snow-covered, for example. But, if further investigation leads to the conclusion that the camera was likely on AWB, that imagery has been excluded from this dataset. Data from roughly a dozen camera sites has been excluded because of concerns about poor-quality data resulting from AWB. We note that this approach is imperfect; on some cameras, for example, only the brightest pixels are used to determine the white balance, and in this case our method would not necessarily work. We are working on the development of more general methods to detect AWB imagery. Δ =           + + -       +      + + -       +      + + -            (2) R R G B G R G B B R G B.
Comparison of Transition Dates between the PhenoCam Dataset v1. 0 and v2. 0. As discussed above, the processing steps and data quality of the PhenoCam dataset have been improved from v1. 0 to v2. 0 but the new release of the dataset does not invalidate the previous version. We showed this by comparing 535 rising Fig. 2  Qualitative and quantitative effects of Auto-White Balance (AWB) on phenological data. The left and right panels show when AWB was off and on at the Snipe Lake PhenoCam site (60. 6°N, 154. 3°W, tundra), respectively: (a) sample images taken on the same day (summer solstice) of the year in 2011 and 2017, (b) fullyear green chromatic coordinates extracted from an ROI representative of vegetation greenness, and (c) the red, green and blue chromatic coordinates (R CC , G CC and B CC ) and the deviation from grey (Δ) extracted from the whole image.
and 577 falling seasonal cycles that were common between the two versions (Fig. 3 ). We compared the rising and the falling 10%, 25% and 50% transition dates between the two datasets, based on the 3-day 90 th percentile GCC timeseries. The intercomparison showed a strong agreement (R 2 > 0. 99 for all the 10%, 25 and 50% transition dates) between v1. 0 and v2. 0. Median absolute error was only 1 day for all the 10%, 25 and 50% transition dates. Root mean square deviation (RMSD) was 3. 9, 3. 6 and 4. 3 days for the 10%, 25 and 50% transition dates, respectively. We identified less than 1% of the transition dates were the difference was more than 20 days. The small discrepancies between the two datasets may be caused by several factors, including updated masks, and newly available data since the last release.
For example, the revised masks for howland1 EN 2000 resulted in a shift of 22 days in transition 50% of year 2008, because the old mask had contaminated with a deciduous signal. In another instance, the 2015 end-of-season transition dates for turkeypointenf39 EN 1000 shifted more than two weeks, caused by the properly constrained winter baseline with the additional data for 2016 onward. Representativeness of the PhenoCam Network. Cameras in the PhenoCam network are widely distributed across North America, and the over-arching objective of the network is to sample the diversity of ecosystems and climate zones across the continent. Cameras located at sites with a range of different vegetation types (Table  1 ), including agriculture (226 site years), deciduous broadleaf (643 site years), deciduous needleleaf (45 site years), evergreen broadleaf (28 site years), evergreen needleleaf (265 site years), grassland (279 site years), shrub (142 site years), tundra (62 site years), understory (18 site years), and wetland (64 site years).
Usage Notes The curated PhenoCam Dataset v2. 0 is permanently and publicly available through the Oak Ridge National Lab (ORNL) DAAC (Distributed Active Archive Center for Biogeochemical Dynamics) data repository (https:// doi. org/10. 3334/ORNLDAAC/1674). We have also developed an interface (http://explore. phenocam. us/) to facilitate data exploration and visualization, from which the user can also download data on a site-by-site basis. All imagery and data (updated in near-real time, and including data from sites where the data record is shorter than six months, or the data are not considered to be high enough quality, for inclusion in a curated data release) are also available through the project web page (http://phenocam. sr. unh. edu).
xROI. xROI is an open-source R package to extract time-series data from large sets of digital images. With a graphical user interface, xROI provides functionality to delineate ROIs, to detect data discontinuities (FOV shifts, clouds, etc. ), and to derive high-quality color-based statistics (digital numbers, chromatic coordinates) from stacks of images. The xROI software can be used to extract data from PhenoCam imagery for custom ROIs, or from imagery not included in the PhenoCam archive. While the package is available from the R Comprehensive Archive Network (CRAN), the source code (https://github. com/bnasr/xROI) is open under the GNU Affero General Public License (AGPLv3) and can be downloaded from refs  46, 47. For more details see Seyednasrollah et al. 48.
Hazer. hazer is an open-source R package for detecting foggy or hazy images. Haze statistics are calculated from the frequency distribution of RGB digital numbers across the image. The getHazeFactor function returns the "haze degree" value, varying from 0 to 100%. High haze degree values indicate high probabilities of haze or fogginess (Fig. 5a ). We consider images with haze degree > 40% to be foggy or hazy; the distribution of the haze degree value across all PhenoCam sites is shown in Fig. 5b. While the haze degree values are used for quality check, the hazy images are not excluded from the data release v2. 0. The package also presents additional functionalities to obtain brightness, darkness and contrast matrices for an image. Hazer is open source under the GNU Affero General Public License (AGPL-v3)  49.
Phenocamr. The phenocamr R package facilitates the retrieval and post-processing of PhenoCam time series  50. The post-processing of PhenoCam data includes outlier removal and the generation of data products, in particular the phenological transition dates as included in this dataset. The package is available from the R Comprehensive Archive Network (CRAN) while the source code is open under the GNU Affero General Public License v3. 0 and available from https://github. com/khufkens/phenocamr.
Instruments such as the 3MI/MetOp-SG, MAIA, SpexOne and HARP2 on PACE, POSP, SMAC, PCF, DPC-Lidar, ScanPol and MSIP/Aerosol-UA, MAP/Copernicus CO2 Monitoring, etc. are planned to be launched by different space agencies in the coming decade. The concepts of these future instruments, their technical designs, and the accompanying algorithm development have been tested intensively and analyzed using diverse airborne prototypes. Certain polarimetric capabilities have also been implemented in such satellite sensors as GOME-2/MetOp and SGLI/GCOM-C. A number of aerosol retrieval products have been developed based on the available measurements and successfully used for different scientific applications. However, the completeness and accuracy of aerosol data operationally derived from polarimetry do not yet appear to have reached the accuracy levels implied by theoretical sensitivity studies that analyzed the potential information content of satellite polarimetry.
As a result, the dataset provided by MODIS is still most frequently used by the scientific community, yet this sensor has neither polarimetric nor multi-angular capabilities. Admittedly polarimetric multi-angular observations are highly complex and have extra sensitivities to aerosol particle morphology, vertical variability of aerosol properties, polarization of surface reflectance, etc. As such, they necessitate state-of-the-art forward modeling based on first-principles physics which remains rare, and conventional retrieval approaches based on look-up tables turn out to be unsuitable to fully exploit the information implicit in the measurements. Several new-generation retrieval approaches have recently been proposed to address these challenges. These methods use improved forward modeling of atmospheric (polarized) radiances and implement a search in the continuous space of solutions using rigorous statistically optimized inversions.
A number of future satellite polarimetric instruments and missions are planned and scheduled for launch in the coming decade. Many airborne versions of orbital polarimeters have been developed and deployed during field campaigns to test and improve the concept of polarimetric remote sensing. Polarimetric observations of aerosol properties have also been implemented by groundbased radiometer networks. In addition, several in situ and laboratory polar-nephelometer systems have been designed for accurate measurements of spectral, angular, and polarimetric characteristics of light singly scattered by aerosol particles. Nevertheless, the overall volume of polarimetric observations of the atmosphere remains small compared to that of photometric observations. Furthermore, the currently available polarimetric observations are mostly considered as useful datasets for understanding the potential of polarimetry and for designing future missions rather than as an indispensable source of aerosol information for specific climatological and environmental applications  [13].
This situation is undoubtedly the result of the general complexity of polarimetric observations and theory. Firstly, obtaining consistent, highly accurate, simultaneous multi-angular polarimetric observations in a sufficiently wide spectral range is a technically difficult task requiring substantial efforts for designing, building, and implementing adequate detection systems. Secondly, and probably more fundamentally, the interpretation of multi-angular multi-spectral polarimetric data is quite challenging. Polarimetry is highly sensitive to a large number of atmospheric parameters, and accounting adequately for all these sensitivities in the retrieval algorithm is very demanding, especially in satellite applications where large volumes of data need to be processed in near-real time or with a minimal delay.
Not surprisingly, applications of conventional algorithm types that performed well with intensity-only satellite data (e. g. , Refs. [24, 25]  ) to polarimetric observations failed to realize the significant advantages of aerosol polarimetry  [26]. Therefore, the need to develop more robust algorithms for deriv-ing aerosol properties from polarimetry has been clearly identified by the satellite community. As a result, several such highly optimized algorithms have been developed and demonstrated to provide enhanced aerosol retrievals from satellite polarimetry  [27] [28] [29] [30] [31]. Yet it remains clear that additional effort s are needed for the understanding and utilization of the full potential of aerosol retrievals from polarimetric observations. As it currently stands, satellite polarimetry remains an underexploited area of aerosol remote sensing which requires more attention and investment from the remote sensing community since advancements in this area are likely to drive progress in aerosol (and overall atmospheric) monitoring.
The main objective of this paper is to support ongoing effort s aimed at the advancement of aerosol polarimetry by gathering detailed information about the available and planned polarimetric observations and providing references to other supplementary information on the existing data products and their distribution. We also summarize recent progress in the areas of forward modeling and retrieval algorithm development, outline the most challenging aspects of polarimetric retrievals, and discuss potentially promising ideas for further advancement of polarimetric retrieval methodologies. Owing to its nature, this paper contains a large number of references (Refs. ). As such, it can also be considered a representative database of publications relevant to polarimetric remote sensing of tropospheric aerosols.
Polarimetric observations This paper is primarily focused on passive polarimetric observations from satellites. However, the most common airborne, groundbased, and laboratory measurements are also discussed for the sake of completeness. Both currently available and expected future observations are considered. The description of relevant instruments is summarized in four tables using a maximally standardized format. Below we provide a brief description of the most important polarimetric datasets currently available. Pertinent information about instruments under development is also included. The description is separated into sections discussing different types of observation, including orbital, airborne, ground-based, and in situ measurements. Yet the main emphasis is on satellite missions, since every such mission involves a thorough design and development stage and is expected to provide a long data record once launched.
APS on GLORY The Aerosol Polarimetry Sensor ( Fig. 1  ,  [195]  ) was launched on the Glory satellite on 4 March 2011. This NASA mission would yield comprehensive and highly accurate data on the chemical, microphysical, and optical properties of aerosols and their spatial and temporal distributions. Unfortunately, due to a malfunction on the launch vehicle, the APS did not reach its intended orbit and hence provided no data. Nonetheless, the development of the APS concept, relevant scientific analyses, and analyses of data from its airborne prototype (e. g. , Ref. [44]  ) have strongly stimulated the progress of and interest in polarimetry as a remote-sensing tool. As a consequence, the conceptual design of several recent polarimeters relies on the APS heritage. The APS design yields a high polarimetric accuracy (0.
Furthermore, the POLDER instruments also measure the synthetic I , Q , and U owing to the difference in acquisition times along the satellite track. Again, in the SGLI case, the angular difference between the three different measurements with a polarizer at -60 °, 0 °, and + 60 °is small. Each viewing angle of each polarizer is included in the SGLI level 1B polarization dataset. Also, such small polarization is weak from the signal-to-noise-ratio point of view. Note that the measurements of I , Q , and U at two wavelengths are only available from one viewing angle, compared to a maximum of 14 directions from POLDER-1 and -2 or 16 directions from POLDER/PARASOL. However, the non-polarized VNIR optics (i. e. , most of observations from 380 nm to 12 µm) always takes nadir-looking measurements.
The NASA PACE mission  [266]  , slated for launch in late 2022, will carry three instruments. The primary instrument is the wideswath (2-day global coverage at 1-km resolution) OCI, which will have sensitivity from 320 nm to 885 nm at 5-nm spectral resolution, and seven discrete channels in the SWIR. It will carry two contributed multi-angle polarimeters on a do-no-harm basis, one of which is HARP2. It will also carry the SpexOne multi-angle polarimeter described in the following section. When launched into its planned 676. 5-km, local 13:00 equator crossing orbit, PACE is designed to last three years, with 10 years of fuel. The contrasting, yet complimentary nature of the OCI, HARP2 and SpexOne instruments will provide for a dataset with great potential for remote sensing of aerosols.
g. , Ref. [10]  ). The main technical specifications of SpexOne are listed in Table  3. On PACE, the SpexOne hyperspectral polarimetric measurements at five viewing angles have perfect synergy with the hyperangular measurements from HARP and the radiometric measurements from the OCI ranging from the UV to the SWIR. The Spex-One, HARP, and OCI together will provide a unique capability in aerosol and cloud remote sensing, including a combination of both (aerosols above and in between clouds). The SpexOne instrument can be transformed into a wide-swath instrument by using separate modules, each with their own detector, for the different viewing angles. This wider-swath realization of SPEX is referred to as the SpexLite instrument  [227].