dataesr/scientific-paragraphs-categorization huggingface dataset:
Applied to synthetic catalogs 8  , when comparisons with both the true and biased data are possible, the algorithm results in statistically corrected datasets, namely, the distance moduli (thus distances and peculiar velocities) are in agreement at better than an average 1%. This step also permits us to set the parameters of the algorithm. Subsequently, as applied to observational catalogs of the Cosmicflows project, the algorithm gives datasets that are input into the Wiener-filter technique to reconstruct the Local Universe. The Wiener-filter technique, a classical restoration method, is specifically chosen for its inefficiency in taking into account the biases. Resulting reconstructed density fields are a great match to the 2MASS Galaxy Redshift Catalog (2MRS) and the velocity field does not present any significant outflow or infall, signs of biases, out of or onto the Local Volume.
Within this context, the new proposed method improves the quality of the peculiar velocity catalogs as a whole. The newly derived version of the peculiar velocity datasets will be used for future constrained simulations of the Local Universe. Nevertheless, reducing biases in cosmological data and those appearing while exploiting the data is still an open and challenging problem. It leads to new questions, which we will tackle in future works. For instance, following  Sorce (2018) , the extra-smoothing of the velocity field by the Wiener-filter technique, due to the fading of the number of data points with the distance from the observer, will be taken care of. In the meantime, subsequent studies will also focus on using different H 0 values to probe on the possibility to extract an estimate of the H 0 value directly from the data.
Acknowledgements. The authors would like to thank the referee for their careful reading of the manuscript and their sensible comments that helped improved its quality. The authors acknowledge the  Gauss Centre for Supercomputing  e. V. (www. gauss-centre. eu) and GENCI (https://www. genci. fr/) for funding this project by providing computing time on the GCS Supercomputer SuperMUC-NG at  Leibniz Supercomputing Centre  (www. lrz. de) and Joliot-Curie at TGCC (http://www-hpc. cea. fr). J. S. acknowledges support from the  ANR LOCALIZATION  project, grant  ANR-21-CE31-0019  of the  French Agence Nationale de la Recherche. E. T. acknowledges support by  ETAg  grant  PRG1006  and by the  EU  through the  ERDF   CoE  grant  TK133. This work was supported by the  Programme National Cosmologie et Galaxies (PNCG) of CNRS/INSU  with INP and  IN2P3 , co-funded by  CEA  and  CNES.
Metrics used Two tools were used to compute sentiment metrics: the VADER package  (Hutto and Gilbert, 2014)  of the Natural Language Toolkit (NLTK)  (Bird et al. , 2009) , and the PATTERN python module  (De Smedt and Daelemans, 2012). Both use a lexical approach, i. e. , based on a large dictionary of terms associated with scores. NLTK has been trained on social media text, so it is ideal for our dataset. PATTERN is trained on official texts, but is also used for its unique 'subjectivity' and 'modality' metrics. The 3. 6 versions of Python, VADER and PATTERN were used. The sentiment metrics (computed with NLTK or PATTERN) is a real number taking their values in the interval  [-1, 1]. It measures to what extent the post expresses a positive or negative sentiment, from fully negative (-1) to fully positive (1).
The subjectivity metrics (computed with PATTERN) is a real number in [0, 1]. It measures to what extent the post uses subjective language, from fully objective (0) to fully subjective (1). The modality metrics (computed with PATTERN) is a real number in  [-1, 1]. It measures to what extent the author uses a confident or credulous tone, from fully credulous (-1) to fully confident (1). The three metrics are independent from each other (see  Wankhade et al. , 2022  for more details).
In the NLTK regression results, infant cereals are associated with positive sentiment. It is noticeable that "pesticides" are prevalent among discussion of infant cereals, as well as "preservatives, sweeteners and additives". Both hazard categories are themselves associated with a positive sentiment (although not 5% significant), which can be justified by strict regulation preventing infant food from these substances. This may explain infant cereals being associated with positive sentiment. The Campylobacter and Cronobacter cases illustrate well the role of subjectivity and modality analyses in the understanding of concerns expressed by parents. In the case of Campylobacter, the language used is objective, which means the way the topic is addressed is not emotional but rather factual information and scientific evidence oriented.
Acknowledgments This work benefited from remarks by many people, including  Alin Bostan ,  Richard Brent ,  Frédéric Chyzak ,  Thibault Hilaire ,  Philippe Langlois , and  Nicolas Lou- The initial impulse came from discussions with  Fredrik Johansson ,  Guillaume Melquiond , and  Paul Zimmermann. Frédéric Chyzak  suggested to work with el- ] in Section 11. I am especially grateful to  Guillaume Melquiond  and  Anne Vaugon  for many insightful comments at various stages, and to  Paul Zimmermann  for his thorough reading of a preliminary version.
Namely, we find that realignment works better on tasks for which alignment is correlated with cross-lingual transfer when generalizing to a distant language and with smaller models, as well as when using a bilingual dictionary rather than FastAlign to extract realignment pairs. For example, for POS-tagging, between English and Arabic, realignment can bring a +15. 8 accuracy improvement on distilm-BERT, even outperforming XLM-R Large by 1. 7. We thus advocate for further research on realignment methods for smaller multilingual models as an alternative to scaling.
Realignment methods typically require a translation dataset and an alignment tool, like FastAlign  (Dyer et al. , 2013) , to extract contextualized pairs of translated words that will be realigned. Despite some encouraging results on specific tasks, current realignment methods might not consistently improve cross-lingual zero-shot abilities of mBERT and XLM-R  (Wu and Dredze, 2020b). When tested with several seeds on various finetuning tasks, improvements brought by realignment are not always significant and do not compare with the gain brought by scaling the model, e. g. , from XLM-R Base to XLM-R Large. However, these realignment methods were not tried on smaller models like distilmBERT as we do here. The mitigated results of realignment methods raise the question of whether cross-lingual transfer is at all linked with multilingual alignment.
Evaluating cross-lingual transfer A model has high CTL abilities when, after finetuning for one language, it can obtain a high evaluation score on other languages. To evaluate it for a given task, we compute the relative difference between the evaluation metric m en on the English development set and the evaluation metric m tgt on the target language: cross-lingual transfer = m tgt -m en m en (1) The monolingual metric is a score between 0 and 1, like accuracy or f1-score, where higher is better. Then our metric gives scores between -1 and +∞. A negative score is obtained if and only if m tgt < m en , which should always be the case in practice. Values closer to 0 then indicate better CTL for a specific task and language. It must be noted that for datasets where the target language test set is a translation of the English one, the normalization in Equation 1 allows the metric to boild down roughly to minus the proportion of correct answers in English that were misclassified when translated, assuming there isn't not to many misclassified English examples that were correctly classified in the target language, which should be the case since there are not that much misclassified English examples in general.
Evaluating alignment To evaluate multilingual alignment, we use the same method for extracting pairs of translated words with their context as  Gaschi et al. (2022). Provided a source of related pairs of words from both languages, a fixed number of pairs of words are randomly selected and a nearest-neighbor search with cosine similarity is performed. The top-1 accuracy of the nearest-neighbor search is the score of the alignment evaluation. To extract contextualized pairs of translated words from a translation dataset, FastAlign is the most widely used word aligner in realignment methods  (Wu and Dredze, 2020b; Cao et al. , 2020; Zhao et al. , 2021; Wang et al. , 2019) , but it is prone to errors and thus generates noisy training realignment data  (Pan et al. , 2021; Gaschi et al. , 2022).
Experimental details We evaluate cross-lingual transfer with three multilingual tasks, the sizes of which are reported in Table  1 : • Part-of-speech tagging (POS-tagging) with the Universal Dependencies dataset  (Zeman et al. , 2020). Similarly to Wu and Dredze (2020b), we use the following treebanks: Arabic-PADT, English-EWT, Spanish-GSD, French-GSD, Russian-GSD, and Chinese-GSD. • Named Entity Recognition (NER) with the WikiANN dataset  (Pan et al. , 2017). • Natural Language Inference (NLI) with the XNLI dataset  (Conneau et al. , 2018). It must be noted that XNLI is the only dataset with translated test sets, and thus the only one for which the cross-lingual transfer metric is strictly comparable across languages. In our experiments, high correlation will nonetheless be observed between CTL and alignment for the two other tasks, suggesting that the CTL metrics is not so much affected by difference in size and domain between the test sets.
Following a similar setting as  (Wu and Dredze, 2020b) , realignment data from the five pairs of languages (English-target) is interleaved to form a single multilingual realignment dataset. Models are fine-tuned on POS-tagging or NER for five epochs and 2 epochs for NLI because its training data is larger. We use the opus100 translation dataset  (Zhang et al. , 2020)  from which we extract pairs of words using bilingual dictionaries. We also tested with the multiUN translation data  (Ziemski et al. , 2016) , which conditioned our choice of languages, and with other ways to extract alignment pairs: FastAlign  (Dyer et al. , 2013)   and AWESOME-align  (Dou and Neubig, 2021). Changing the translation dataset does not fundamentally change the results, and using probabilistic alignment tools made realignment methods less effective.
Limitations We worked with only five language pairs, all involving English and another language: Arabic, Spanish, French, Russian and Chinese. This is due to using the multiUN dataset  (Ziemski et al. , 2016)  for evaluating alignment and performing realignment. We also used the opus100 dataset  (Zhang et al. , 2020) , which contains more pairs and is the dataset that eventually figured in our paper, but we stuck to the same language pairs for a fair comparison with multiUN in Appendix F. This narrow choice of language limits our ability to understand why realignment methods work well for some languages and not others. And we believe that making a similar analysis with many language pairs, not necessarily involving English, would be a good lead for further research investigating the link between the success of the realignment method and how two languages relate to each other.
Acknowledgements We would like to thank the anonymous reviewers for their comments, as well as Shijie Wu, who kindly explained some details in the implementation of his paper  Wu and Dredze (2020b). We are also grateful for discussion and proof-reading brought by our colleagues at Posos: François Plesse, Xavier Fontaine and Baptiste Charnier. Experiments presented in this paper were carried out using the Grid'5000 testbed, supported by a scientific interest group hosted by Inria and including CNRS, RENATER and several Universities as well as other organizations 3.
B Experimental details B. 1 Scientific artifacts used We relied on the following scientific Python packages for our experiments: the Hugging-Face's libraries transformers  (Wolf et al. , 2020) , datasets  (Lhoest et al. , 2021)  and evaluate 4 , PyTorch  (Paszke et al. , 2019) , NLTK  (Bird et al. , 2009)  and its implementation of the Stanford Chinese Segmenter  (Tseng et al. , 2005) , seqeval  (Nakayama, 2018)  for evaluating NER, NumPy (Harris et al. , 2020), and AWESOME-align  (Dou and Neubig, 2021) , FastAlign  (Dyer et al. , 2013) , and MUSE dictionaries  (Lample et al. , 2018)  for extracting alignment pairs. We used the following datasets: two translation datasets for building evaluating alignment and realigning, multiUN  (Ziemski et al. , 2016)   (Zhang et al. , 2020) , XNLI  (Conneau et al.
, 2018) , the Universal Dependencies dataset  (Zeman et al. , 2020)  for POS-tagging, and the WikiANN dataset for NER  (Pan et al. , 2017). Finally, we worked with four different models: distilmBERT, which was released with distilBERT  (Sanh et al. , 2019) , mBERT, which was released with BERT  (Devlin et al. , 2019)  and XLM-R Base and Large  (Conneau et al. , 2020a).
B. 2 Multilingual alignment data From a translation dataset, pairs were extracted either using a bilingual dictionary, following  (Gaschi et al. , 2022) , with FastAlign  (Dyer et al. , 2013)  or AWESOME-align  (Dou and Neubig, 2021). For FastAlign, we produced alignments in both direction and symmetrize with the grow-diag-final-and heuristic provided with FastAlign, following the setting of  Wu and Dredze (2020b). For all methods of extraction, we kept only one-to-one alignment and discard trivial cases where both words are identical, again following Wu and Dredze (2020b).
B. 5 Computational budget The first experiment was performed on Nvidia A40 GPUs for an equivalent of 3 days for a single GPU (including all models, tasks and seeds). For the second experiment, training (fine-tuning and/or realignment) was performed on various smaller GPUs (RTX 2080 Ti, GTX 1080 Ti, Tesla T4) for distilmBERT, mBERT and XLM-R Base, and on a Nvidia A40 for XLM-R Large. The experiment took more than 10 GPU-days on the smaller GPUs, combining all models, realignment methods (including baseline), random seeds, translation datasets and pairs extraction methods. For XLM-R Large, for which we only trained the baseline, it still required 30 GPU-hours on Nvidia A40.
F Detailed results of the controlled experiment This section provides detailed results of realignment methods for POS-tagging and NER, for all tested models, languages, translation datasets, and methods of extraction for realignment data. It also contains results for XNLI, for which only one translation dataset (opus100) and one extraction method (bilingual dictionaries) were tested. Results are shown on Tab. 13 (POS, opus100), 14 (POS, multi-UN), 15 (NER, opus100), 16 (NER, multi-UN), 17 (NLI, opus100). A light gray cell indicates that the realignment method obtained an average score that is closer to the baseline with the same model than the standard deviation of the said baseline. A dark gray cell indicates that the realignment method provokes a decrease w. r. t. the baseline that is bigger than the standard deviation.
This leads us to specifically consider the potential contribution of GeoAI, which nowadays offers new methods for processing and reasoning on complex and time-related geospatial data, either structured or unstructured, and that of GeoVA, which provides cutting-edge visualization capabilities for the exploration of complex data. Together, GeoAI and GeoVA are well suited for providing a data processing framework associated with cyberspace in close connection with geographical space and humans-in-the-loop. The goal is to derive a series of conceptual data interaction principles that when considering together will bring new opportunities and synergies for urban, environmental, and earth sciences. Given exciting recent developments in artificial intelligence (AI) that can provide novel computational solutions to mimic human intelligence to a certain degree, this paper proposes to recast GIScience in the era of cyberspace and envision its future characteristics.
Based on a conceptual framework that identifies the respective intertwined roles of geographical space and cyberspace  (Liu et al. , 2022) , we review current challenges and future directions offered by close interactions between the two spaces, as illustrated in Fig. 1. First, the data generation dimension covers a much larger information space, from geolocated smart sensors to human-generated geospatial data. Second, data modeling approaches are evolving, from well-structured data to the integration of very large heterogeneous datasets not always explicitly generated for further use. Third, the geographical space and geospatial data favor the exploration of novel AI data processing and visual explorations, where humans are directing different processing and interactive steps. Finally, the application scope is expanding at an unprecedented rate and reaching novel application areas.
It is acknowledged that cyberspace cannot exist or function without geographical space, even though cyberspace has specific properties distinct from it  (Batty, 1997; Mohebbi et al. , 2020). Cyberspace is not considered a simple abstract virtual space, instead, it includes human activities and interactions that might be related to geographical space. First, human activities in cyberspace are mainly dependent on sensing and communications and network infrastructures (e. g. , data servers and Fig. 1. An integrated view of GIScience and cyberspace, with perspectives offered by GeoAI and GeoVA clients) deployed in the physical world. Second, geographical space and cyberspace have the same essential components (i. e. , humans who exist and carry out physical activities) while humans themselves must exist and (so far) rely on the real geographical world.
In particular, there is a need to identify the most appropriate filtering and aggregation rules to conserve the minimum set of data that will be not only useful at a given time, but also possibly for the next generation, and with the difficulty of identifying these future needs that are not easily identifiable at present. This motivates the development of an integrated reference meta-model and data infrastructures for geographical space and cyberspace information. Cyberspace can be considered an interdependent network of data and global infrastructure. Data in cyberspace is captured from different sources, platforms, and users' actions, but a marginal amount of the data generated are generally stored and kept, thus generating the need for advanced real-time or live GeoAI capabilities. As cyberspace is physical distance free, and where people from different origins can interact, definitions of standards, management, and governance rules are difficult to implement.
AI also provides a novel form of exploration of human behaviors in the real world  (Torrens, 2018). The ability and potentiality of AI for GIScience have been progressively strengthened with the availability of large databases and the search for novel data manipulation and exploration approaches. The emergence of data mining approaches  (Miller and Han, 2009 ) and computational machine learning programs applied specifically to geographical data have been a rising trend in the past few years. Machine learning applied to geographical space encompasses a wide range of techniques from the discovery of spatial associations, clusters, and patterns to anomaly detection  (Li et al. , 2016). Neural network algorithms have been widely applied to the detection and extraction of objects in large images  (Tang et al.
GeoAI and collaborative cyberspace for current data science The development of ICT introduces the concept of collaborative cyberspace, which enables different subjects (e. g. , governments, organizations, communities, enterprises, and citizens) from different regions of the world to participate in cyberspace activities through cooperation. Because a considerable amount of data generated in cyberspace and geographical space is the consequence of people's passive or active participation in user-generated content  (See et al. , 2016) , there are two major challenges. First, cyberspace is filled with data of unknown validity as previously stated. Second, maintaining active participation of people in cyberspace, which is easier in projects where people contribute passively, such as social networks, but more difficult when people are actively, knowingly, and voluntarily generating data, such as the OpenStreetMap project.
GeoAI and collaborative cyberspace suggest approaches to addressing these challenges. GeoAI can be employed to validate the quality of user-generated content in cyberspace. Computer vision, and particularly image recognition algorithms, have been widely used to evaluate user-generated information, such as in species identification applications  (Wäldchen and Mäder, 2018) , but there has been less emphasis on using AI to analyze geospatial user-generated content. GeoAI can help validate usergenerated big data in cyberspace by training algorithms to model geographical space using other data sources such as sensors and satellite images and then using these trained models to validate data contributed by people. For example, using satellite imagery and deep learning algorithms to assess the positional accuracy or completeness of Open-StreetMap data  (Xie et al.
, 2019). Another possibility is to utilize GeoAI to evaluate social media posts. For example, there has been research on how to evaluate fake news and misleading information on social networks such as Twitter during natural disasters  (Rajdev and Lee, 2015; Torabi Asr and Taboada, 2019) , but this work has primarily focused on textual analysis. One potential application of GeoAI may be to train models using sensor observations that forecast the likelihood of a natural disaster occurring to assess the accuracy of geo-tagged posts on social media about such events. Furthermore, combining semantic information from big data in cyberspace with geographical data may improve the performance of AI algorithms, allowing for more accurate verification of future contributed data to cyberspace. For example, geo-tagged images of obstacles in streets, sidewalks, and parks, as well as textual information about the degrees of walkability and accessibility in a city, when combined with satellite and aerial imagery, can produce a more accurate classification map of a city's walkable/accessible areas than results using only one type of data.
, 2015; Stock, 2018). For example, there is a significant number of non-geotagged postings on social networks, but by utilizing existing geotagged posts, it may be possible to train algorithms that can predict the geolocation of non-geotagged posts, such as geolocating Tweets  (Dutt and Das, 2021). Besides data quality, GeoAI can propose potential solutions for the second challenge in collaborative cyberspace: sustaining people's contributions to cyberspace. As mentioned, the contributed data is either passive/involuntary (e. g. , social media data, delivery applications, etc. ) or active (e. g. , crowdsourcing, VGI, citizen science, etc. ). In active contributions, initiating and sustaining people's participation are known challenges  (Lee et al. , 2020; Lotfian et al. , 2020). More specifically the main problems are to first reach out to the subjects that are potentially interested to contribute and second to provide motivational factors for them to continue contributing.