dataesr/scientific-paragraphs-categorization huggingface dataset:
Toplines and baselines. The Baseline models. We build baselines in three steps: acoustic modelling, clustering, and language modelling. The acoustic model uses Contrastive Predictive Coding (CPC,  [1] ), where the representation of the audio is learned by predicting the future through an autoregressive model. The CPC model uses a convolutional encoder genc to map an input signal x as a sequence of embeddings z = (z1,. , zT ) at a given frame rate. At each time step t, a predictor network gpred takes as input the available embeddings z1,. , zt and tries to predict the K next future embeddings {z t+k } 1≤k≤K by minimizing a constrastive loss using Nt negative embedding samples. We used the PyTorch implementation of CPC 1   [6] , which is a modified version of the CPC model with the following architecture: the encoder genc is a 5-layer 1D-convolutional network with kernel sizes of 10,8,4,4,4 and stride sizes of 5,4,2,2,2 respectively, resulting in a downsampling factor of 160, meaning that, for a 16KHz input, the embeddings have a rate of 100Hz.
The predictor gpred is a multi-layer LSTM network, with the same hidden dimension as the encoder, followed by a 1-layer transformer. We report results from a 4-layer LSTM, trained on the clean-6k version of Libri-light (see  [4]  for additional results from a smaller model). Note that we are still able to train lowand higher-budget baselines on the output of this model, as we take the critical and potentially costly part of the pipeline to be language model training, and therefore calculate the GPU budget based on the language model training time only. The clustering module uses k-means on the outputs of a given hidden layer of the autoregressive model. The clustering is done on the collection of all the output features at every time step of all the audio files in a given training set. After training the k-means clustering, each feature is assigned to a cluster, and each audio file can then be discretized to a sequence of discrete units corresponding to the index of the assigned cluster.
We trained the BERT model using a 12-layer transformer encoder and use this as a reference high-budget system (90M parameters). The implementation was done via fairseq (  [23] ). For further details of the models, see  [4]. The Topline models. We trained a BERT model on forcealigned phoneme labels (one per frame) using the gold transcription of the LibriSpeech dataset. We also employed the span masking similarly to the baseline model. In addition to the forced-alignment BERT, we also included a BERT model trained on the gold phonetic transcription of the LibriSpeech dataset (no framewise repetitions), with the difference that we only mask one token instead of a span of tokens, since each token is the width of a phoneme rather than a frame. For an absolute topline comparison, we used the pretrained RoBERTa large model (  [24] ), which was trained on 50K subword units on a huge dataset of total 160GB, 3000 times bigger than the transcription of the LibriSpeech 960h dataset.
Submitted systems and results Eight systems were submitted. All were low-budget. The systems are in principle less comparable than in previous years, because the choice of training sets was freer, but all systems made use of either the baseline representations or the same choices of corpus for re-training, with the exception of HL, which trained its units on LibriSpeech 100 hrs (clean), a smaller data set. System BN  [25]  begins with the baseline CPC representations and applies speaker normalization before re-running kmeans. An LSTM language model architecture is used. The two systems of  [26]  both use continuous representations for the acoustic evaluation. JC1 uses the baseline CPC representations for the acoustic tests, while JC2 improves on the baseline representations using speaker embeddings, and by pulling the vectors in the direction of a cluster mean.
For the other evaluation measures, these submissions use different approaches for each measure. While this does not directly assess language modelling, it helps understand the limits of the acoustic representations for finding and encoding different types of linguistic information. Beginning from a (k-means) discretized version of the respective acoustic CPC representations, for sWUGGY, they treat the training corpus as a dictionary and extract a distance to the best match; for sBLIMP, they train an LSTM; and, for sSIMI, they perform segmentation to discover word types. They then train word embeddings using word2vec (JC1) or fast-text (JC2) and use this embedding space to calculate similarity. System HL  [27]  trains representations using Mockingjay, an approach based on bidirectional self-attention  [28].
Syntactic. The sBLIMP scores show smaller improvements over the LSTM baseline than sWUGGY. Unlike for sWUGGY, the toplines show that access to the gold transcription alone is not sufficient: while it helps, good performance is only achieved by RoBERTa, which notably trains on much more data than our other toplines. We discuss the implications below. Thus, this appears to be a harder metric to do well on. Lexical semantics. sSIMI is even harder. Even the bestperforming reference model, the RoBERTa topline, shows fairly weak correlations with human judgements. And, as with syntax, the gold transcription alone is not sufficient even to reach this weak correlation. More is needed, as evidenced by the large drop in performance for the phone topline compared to RoBERTa. Here, however, it would appear that an additional factor matters: the temporal resolution of the input representations.
Results are promising. Most prominently, progress has been made in the span of one iteration of this challenge on the sWUGGY metric. This is great news for LM applications that are heavily driven by the lexicon. Thus, the approaches taken here should already be helpful for ASR decoding (note also that some recent self-supervised pretraining approaches to ASR have shown good results without explicit LMs:  [33] ). There is a big gap between discovered units and text-so, still major room for improvement-but the speed of progress is very encouraging. Above the word level, things are harder. As for sWUGGY, sBLIMP performance shows promising improvements over the baselines. Still, the gaps with text-based approaches are more pronounced for both sBLIMP and sSIMI. Speech-based language modelling needs to make more progress before being applicable to tasks like translation and dialogue, for which good language modelling depends on making syntactic and semantic predictions.
Acknowledgements The work for MS, PR and for EDupoux and TAN in their EHESS role was supported by the Agence Nationale de la Recherche (ANR-17-EURE-0017 Frontcog, ANR-10-IDEX-0001-02 PSL*, ANR-19-P3IA-0001 PRAIRIE 3IA Institute) and grants from CIFAR (Learning in Minds and Brains) and Facebook AI Research (Research Grant). The work for EDunbar was supported by a Google Faculty Research Award and by the Agence Nationale de la Recherche (ANR-17-CE28-0009 GEOMPHON, ANR-18-IDEX-0001 U de Paris, ANR-10-LABX-0083 EFL).
In addition, data transferred in the V2X network can be modified by an attacker to mislead vehicles, which can lead to traffic accidents. Appropriate security schemes can be adopted, but this could cause additional latency  [31]. This is of utmost importance in emerging intelligent connectivity networks, leveraging cloud-based capabilities to support critical security services with stringent security, trust, and privacy requirements  [16]. Yang et al. [48]  proposed two lightweight anonymous authentication schemes for the V2X network. One scheme is applicable for V2V communication, while the other is suitable for V2I communication. Both schemes considered limitations of V2X, such as OBU resource constraints and latency. He et al. [3]  introduced the preservation of confidentiality based on a security scheme for V2V and V2I communications in VANETs.
We evaluated to fake the community decision by sending faked messages using USRPs. We evaluated our systems under different conditions for the exclusivity of the data and the situations tested, in four different scenarios. In all scenarios the USRPs have performed as the attacker, whether in static or dynamic way. • Scenario 1 (Campus static test): The first scenario was performed on the campus circuit with the static attacker (USRPs cards). • Scenario 2 (Static Road Test): The second scenario took place on the driving road with the static attacker. • Scenario 3 (Dynamic Campus Test): The third scenario took place on the campus circuit with the dynamic attacker. • Scenario 4 (Dynamic Road Test): The fourth scenario was done on the driving road with the dynamic attacker.
V. DISCUSSION The range of communication is linked to detection accuracy. The Indicator 3 -I3-of our algorithm in section III-C could be based on several parameters: Velocity, distance, direction, yaw-rate, and weather conditions. It turned out from our previous work  [9]  that the most impacting parameter is velocity. Indeed, this parameter impacts the other parameters in a big way. We have evaluated one of the most widely used datasets in V2X communication simulations, VeRiMi [1]. Although the date set shows slight variation, all the fake messages are  far away from their real position. Therefore, it is far from the real-world conditions where a Sybil node could have stated a nearby antennas source. Consequently, because hackers are always at a distance from receivers, a machine-learning model could easily make the right decision.
VI. CONCLUSION AND PERSPECTIVES This contribution corresponds to the definition of an algorithm capable of building autonomous blockchain communities to evaluate their ''goodness'' and thus revoking malicious vehicles in real-time. The proposed solution allows a collaborative system between individual vehicles and the structure since we cannot rely on only one in the revocation process. Although evaluated in real experimentation, the defined approach could meet the real-time requirements. We can conclude that our algorithm is more accurate than other frameworks simulated through VeriMi dataset as we have used V2X equipment in real-world conditions. Thus, our community algorithm permits using blockchain technology in vehicular communication and adds value to cyber-physical security. Our perspective is to make a proof-of-concept by turning our algorithm into a real traffic management server and hosting a real-time blockchain to use it in our circuit.
Acknowledgements: Immense thanks are due to  S. Angiboust ,  M. Locatelli ,  B. Dubacq ,  S. Guillot ,  M. Handy ,  J. Wakabayashi ,  A. Vitale-Brovarone ,  T. Duretz ,  M. Jentzer  and many others, for fruitful and friendly discussions over the years.
Recruitment ME/CFS patients and HC were contacted using the National Centre for Neuroimmunology and Emerging Diseases (NCNED) patient database. Participants were screened in accordance to the Canadian Consensus Criteria (CCC) and International Consensus Criteria (ICC) case definitions for ME/CFS using a comprehensive online questionnaire. ME/CFS patients were included if they met the CCC or ICC case definitions for diagnosis and reported being diagnosed by a physician. Potential eligible participants were invited to volunteer in this project. Of those contacted using the NCNED database, 10 ME/CFS patients from South-East Queensland were invited to volunteer in this project. ME/CFS patients were age-and sex-matched with HC. The HC group was defined as those who have not been diagnosed with any underlying illness and are non-fatigued.
Participant data collection and sample collection All participants completed an online questionnaire to provide sociodemographic background, medical history, medications, and symptom history for ME/CFS patients. The 36-item short form health survey (SF-36) and World Health Organization (WHO) Disability Assessment Schedule (DAS) were used to determine level of disability and quality of life (QoL)  [32, 33]. Between 7:00 a. m. and 11:00 a. m. at collection locations including Griffith University, Royal Brisbane and Women's Hospital, Robina Hospital, Toowoomba Base Hospital, Sunshine Coast University Hospital and Tweed Hospital, a total of 84 ml of whole, non-fasted blood was collected from consenting participants into ethylenediaminetetraacetic acid (EDTA) tubes via venepuncture by a qualified phlebotomist.
Statistical analysis Shapiro-Wilk test was used to assess normality of distribution of investigated parameters. Additional visual observation of histogram plots was completed. Data presented as mean ± standard deviation (SD) unless otherwise stated. Differences were tested using the Mann-Whitney U non-parametric T test. Flow cytometry data were exported from Accuri C6 software. Fluorescence and time-course data were exported from NIS-Elements Advanced Research version 5. 2. Statistical analysis was done using GraphPad Prism V8 (Graphpad Software Inc. , Version 8, La Jolla, CA, USA) and OriginLabs (OriginLab Corporation, Northampton, MA, USA). Significance was set at p < 0. 05.
Trisomy 21 Research Society COVID-19 Initiative The DS population has been disproportionately affected by Coronavirus 2019 (COVID-19) in terms of experiencing severe illness and death  [Huls et al. , 2020]  dition to health effects, efforts to curb the spread of CoV-2 infection through social distancing may have significant detrimental effects on the daily lives of individuals with DS. The T21RS COVID-19 Initiative was launched as an international survey for clinicians or caregivers to systematically quantify and characterize individuals with DS that were infected with SARS-CoV-2, giving rise to CO-VID-19. This cross-sectional study collected data from different countries, and nearly 1,050 cases were enrolled between April and October 2020. Data were analyzed and compared with a UK survey (ISARIC4C0) of hospitalized COVID-19 patients with and without DS.
Risk factors for hospitalization were similar to the general population (increasing age, male, diabetes, and obesity), with the addition of congenital heart defects. Risk factors for mortality were also similar to the general population and included increasing age, being male and having dementia. Even after accounting for known risk factors, individuals with DS were still 2. 5 times more likely to die after being hospitalized due to COVID-19. The initiative also conducted a study that focused on pediatric COVID-19 cases (<18 years of age) with DS, drawing data from the same international survey and using a US comparison group  [Emes et al. , 2021]. Overall, children with DS were at increased risk for more severe presentations of COVID-19. Hospitalized pediatric cases with DS (n = 127) had a higher frequency of presenting signs and symptoms than those without DS (n = 224), although the profile of presentation was similar.
The H21/Life-DSR Cognitive Test Battery: Identification of a Trial Outcome Measure for Early Intervention in DS and AD There is no consensus on what cognitive tests should be included in a neuropsychological test battery to use as an endpoint in clinical trials of AD in DS. Andrew Aschenbrenner (USA) analyzed data from 5 longitudinal cohorts to identify specific cognitive domains that are most sensitive to rates of change in healthy adults with DS. A panel of experts was convened to agree on which particular measures to include in the final cognitive test battery  [Aschenbrenner et al. , 2021]. Criteria considered included ease of administration and scoring in a global clinical trial (i. e. , the test must not be culturally specific). The final selected battery measures a variety of cognitive domains, including memory, attention, language, and executive function, and may have utility as an outcome measure in clinical trials.
Longitudinal Data Outcomes on Adults with DS and Dementia Supported in Group Homes Matthew Janicki (USA), co-chair of the US National Task Group on Intellectual Disabilities and Dementia Practices, reported on data collected over 10 years on individuals with DS and other intellectual disabilities diagnosed with dementia and living in 3 group homes (N = 22 residents, a third of them with DS)  [Jokinen et al. , 2018; Santos et al. , 2018]. Data collected included resident function demographics, health, and other related information that provided insight into the impact of dementia longterm care in specialty group homes. The majority of early admitted adults were individuals with DS. The mean years for entry to death was 5. 4 years. Adults with DS did survive longer, however, perhaps because they had been admitted earlier in the course of the disease.
Background literature review The primary database used for the literature search was Scopus. Keywords such as "information systems" or "information technology" or "IT sector" were combined (AND operator) with keywords "environment", "sustainability", "sustainable" and "climate". The search taxonomy retrieved articles that had the combination of keywords in the article title, author keywords, or abstract. Via a process of filtering to eliminate non-relevant studies, a total of 372 articles remained. These articles were evaluated via their abstract to assess their suitability against the following two research questions.
How IM/IT/IS can be utilized to improve situation regarding climate change. I see several aspects: • IT/IS permits the use of more efficient products and designs that save raw materials and energy. For example, digital twins can help to better design improved products, e. g. , car batteries  (Merkle, Pöthig, & Schmid, 2021)  • Improving methodological innovations in greenhouse gas emissions calculations. • Improving weather models to mitigate climate change and risks related to this change. • Development of awareness campaigns dedicated to mitigating our personal and professional environmental impacts related to IT and IS. • Increasing the conception and use of new big data analytics tools to mitigate the impacts of our society on climate change. • Developing charters in companies including best practices (see below).
Another core dimension for a Smart City is that of Smart Policy, meaning that any Smart City initiative should align with the strategy of the city, the country (government laws), intergovernmental agreements and of course global policies  (Garg, Schmitt, & Stiller, 2017; Yadav, Hasan, Ojo, & Curry, 2017). Policies here include environmental policies and those related to information governance and therefore IM. The privacy of personal data and information is causing much concern and therefore acts and regulations are important in the context of a Smart City. In South Africa the Protection of Personal Information Act (POPIA) was implemented in 2013 to protect the information and privacy of people, but was only enforced on 1 July 2020 (POPIA, 2020). The General Data Protection Regulation (GDPR) is the European equivalent to the POPIA for South Africa  (GDPR, 2018).
In addition to these acts, companies are also pushed to have their own information governance and management policies and guidelines. Another dimension that is closely linked to climate change issues is that of Smart Mobility. This dimension focuses on factors that involve sustainable, innovative and safe transport systems that are accessible locally, nationally and internationally  (Calderón, López, & Marín, 2017; Yadav et al. , 2017). IoT is an integral part of Smart Mobility and is regarded as an enabling technology of big data  (Bibri, 2019). However, big data is only valuable once quality data is produced, which is required for intelligent decision making to contribute to sustainable smart cities. The Smart City definition illustrates the importance of information exchange within a Smart City and therefore the importance of the provision of open data and the creation of open data hubs.
Competencies related to sustainability and environmental issues are also included in the IS2020 guidelines as social and ethical considerations. In Germany and South Africa modules related to Green IT, and environmental information systems have been introduced in recent years. So whilst some progress has been made, much more effort and investigations need to take place to introduce more competencies regarding sustainable technologies, data management, data analytics and data science into IS/IT and CS degree programs. Much research has been done on the topics of smart cities, Green IT and ICT for sustainability. However, reported empirical studies on IS for meeting the SDGs are still considerably low, particularly with regard to sustainable management of water and sanitation (SDG 6); access to affordable, reliable, sustainable and modern energy for all (SDG 7); and conserve and sustainably use the oceans, seas and sustainable behavior (SDG 14).
Contribution 19 -climate change and role of information technologies -Professor Babita Gupta 3. 5. 1. 1. Information technologies and climate change. Information technologies require energy at every stage of its life-cycle and contribute to the carbon footprint. However, IT is also helping reduce net carbon emissions in various industry sectors. Technology-enhanced working using the video conferencing and cloud-based collaboration that has accelerated during the COVID-19 pandemic is proving to be a doubleedged sword  (Frontier technologies, 2020). While these technologies helped the environment by significantly reducing the impacts of travel and commuting, they also increased carbon emissions since the backend infrastructure, such as massive data centers, is enormously energy-intensive. Current estimates of information and communication technology (ICT) impact range from 1.
4% to 3. 9% of global CO2 emissions, highlighting the need for accurate data on ICT's impact on the environment  (Cunliff, 2020; Murugesan, 2021; Robertson, 2021). Several information technologies are poised to play a critical role in monitoring, managing, and mitigating climate change-related phenomena and increasing environmental resiliency. For example, artificial intelligence and robotics are being utilized for automated detection and monitoring of environmental hazards, remote sensing to track deforestation activities and marine life, monitoring biodiversity, traffic flow optimization, estimating real-time precipitations, and managing flood risks and emergency planning. Increasing occurrences of extreme weather events are affecting food production and distribution globally. Advancements in information technology utilizing multi-source spatial and temporal data, robotics, drones, Internet-of-Things (IoT), etc.
, are being explored in the agriculture and transport industry. For example, precision agriculture technology is an environmentally sustainable strategy for optimizing water input, weed control, disease eradication, pest reduction, and yield enhancement to avert water and food insecurity and optimize the food production chain. Data centers are the backbone for managing big data, cloud computing, AI, blockchain, and IoT. They also have massive power consumption needs contributing to climate change. Researchers and innovators are exploring novel technologies such as liquid cooling that improve heat-dissipation efficiency and reduce their impact on the environment to reduce the carbon footprint of data centers and other IT infrastructure.
Contribution 21 -climate change -IT -Data Science perspective -Ms Jeel Dharmeshkumar Shah The world's climatic conditions seem to deteriorate every day, and the evidence from the research  (Shaftel, 2021)  held at NASA further bolsters this claim. The researchers at NASA further believe that the unprecedented warming trends today are the effect of human actions. Since most of these actions involve the development or use of technology, it certainly plays a significant role in climate change. The world today seems to be in a state of quandary. Whether to restrict the use of technology or invest in further research in the same field is a concern. However, in this case, I believe that the cause is the solution-just like the snake venom that is fatal for humans but is also proven to be medicinal and used to save lives.