dataesr/scientific-paragraphs-categorization huggingface dataset:
Adult language research We still know surprisingly little about language use in the daily life of adult speakers. Prior research, sometimes relying on data from mobile phones and social media, has focused on using typical language patterns to investigate language-based measures of interactional patterns  (Lee et al. , 2013) , personality  (Park et al. , 2015) , mental health (e. g. ,  Karam et al. , 2014) , and also to design technologies such as voice command software for individuals with speech and motor disabilities  (Nicolao, Christiansen, Cunningham, Green, & Hain, 2016). However, LFSE recordings have important insights to add to these lines of applied research-there are still many unanswered basic questions about adults' language experience: What proportion of minutes per day are spent in face-to-face conversation? How many speakers are typically encountered in a day? What mediates the variation in those measures across different lifestyles and communities, and at different points in the lifespan? Could the quantity and quality of linguistic interactions be used as predictors for neurocognitive status, including measuring the effects of interventions? The quantity and diversity of language encountered over the course of the day shapes our cognitive processes for producing and comprehending language, but we do not yet know precisely what is affected, and how.
Goals for future LFSE research Future work with LFSE recordings faces several challenges, as follows. In order to gain a comprehensive sample of human language use (i. e. , good estimates of variance in adult and child language environments) we must both increase the diversity of represented human linguistic communities with LFSE data while also decreasing our reliance on manual annotation, which is too slow and sometimes altogether impossible to work at large scales (i. e. thousands or, perhaps eventually, millions of hours of data).
Increasing diversity Daily linguistic experiences are bound to be affected by the wide-ranging cultural, economic, and linguistic differences that exist between speakers living in different communities around the globe. Yet most of the participant samples used in LFSE research so far have primarily come from healthy adults and typically developing children in industrialized (often Western) settings. We need data from a more diverse sample of human language environments and clinical conditions to discover humanwide principles underlying language development and language processing. This sampling bias holds for psychology at large  (Henrich, Heine, & Norenzayan, 2010) , but also specifically for the sub-fields of adult psycholinguistics  (Norcliffe, Harris, & Jaeger, 2015) , developmental psychology  (Nielsen, Haun, Kärtner, & Legare, 2017) , and developmental language studies (e.
g. ,  Slobin, 2014). For example, while some collections of naturalistic language data have attracted linguistically diverse recordings (e. g. , CHILDES; https:// childes. talkbank. org/; MacWhinney, 2000) , populations from industrialized, often Western populations are still overrepresented relative to the global set of cultural communities existing today (see also the DOBES Archive). Among studies that use LFSE recordings of everyday language, there have been a few comparative samples of cultural groups from industrialized societies (e. g. ,  Ramírez-Esparza et al. , 2009; Weisleder & Fernald, 2013; the ongoing ACLEW project, ACLEW Project, 2018) , with a handful more from traditional indigenous and/or nonindustrial societies  (Casillas, Brown, & Levinson, under review; Scaff, Stieglitz, Casillas, & Cristia, in preparation; Abels & Vogt, 2018).
Decreasing manual annotation Many of the studies mentioned above rely on manual annotation, in part or in whole, to create meaningful input for their analyses. A continued reliance on manual annotation is impractical on the scale of LFSE recordings, which easily accumulate to thousands of hours of data. Even well-funded projects with unrestricted access to human annotators would benefit in time savings and greater comparability of output by using automated tools as part of their data processing pipeline (e. g. , finding segments of speech vs. silence). Basic automated tools for everyday speech corpora are still in their infancy. While LENA's software has been able to produce automated annotations since its inception in the late 2000's, its software is proprietary, outdated, and the output is limited to the information that the LENA organization originally decided to focus on (e.
g. , broad speaker type like "female adult" rather than individual speakers). Alternative tools for automated annotation are an active area of research. For example, Hansen and colleagues (e. g. ,  Ziaei, Sangwan, Kaushik, & Hansen, 2015)  have been trying to further develop tools for data collected with a LENA recording device on an wide range of tasks, including: speech activity detection (finding stretches of speech vs. non-speech), diarization (assigning speech stretches to speaker sources), finding specific words in the audio, word-counting, and identifying the speaker's location/activity. However, the recordings they are working on are limited to a single participant's range of daily activities, and their tools are, for the most part, not accessible to other researchers. While the first few automated processing steps (such as finding where someone is talking versus silent) may seem trivial, research on this topic shows that modern systems do not perform well on LFSE recordings.
Our aim As we will see, researchers planning to collect or analyze LFSE recordings face a number of unique challenges. The first pertains to the ethical and legal aspects associated with data that cannot be de-identified, such as voice recordings. This is an issue that affects all stages of the research, from its conception and submission to an ethics committee, to the archival of the data. A second set of challenges relates to the fast-changing, ever-improving, and yet always seemingly suboptimal technology required to collect and analyze LFSE datasets. A third set of challenges concerns the design and validation of measurements extracted from these datasets. In this paper, we aim to help potential users of LFSE recording methods to decide whether such data can feasibly provide answers to their research questions.
Accessing existing data The advantages of re-using existing data are numerous. First, one avoids the investment of time required to learn about (and put into practice) the laws and ethical procedures for collecting LFSE recordings. Second, by re-using an existing dataset, one can often build on top of previous annotations. Third, investing annotation time in a recording that has already been annotated for information different from the topic under study (e. g. , adding gesture annotations to transcriptions of verbal behavior) opens the door to additional analyses that combine both information sources. Fourth, large datasets are typically rife with small mistakes (e. g. , typos, unmarked sections where the target participant took the device off). These small issues can be found and addressed as a natural part of data re-use.
If there is already an existing dataset that has the right participant sample and recording methods for one's research question, re-using data can be the ideal approach. Naturally, there are many cases where re-use is not possible, for example, if the research question involves participants with Alzheimer's and there are no existing datasets on this population. There are an increasing number of datasets, including ones with LFSE recordings, available for re-use via specialized scientific archives, or collaboration with the individuals who have collected them. Regarding the former option, one place to find ready-to-re-use LFSE recordings is  HomeBank (homebank. talkbank. org; VanDam et al. , 2016; HomeBank, 2018). HomeBank is part of the TalkBank framework (talkbank. org; MacWhinney, 2007), which hosts an enormous collection of language recordings and annotations (e.
g. , CHILDES, AphasiaBank, and many more). On HomeBank, users can find not only raw recordings, but also any automatic or manual annotations that exist. For the time being, HomeBank contributors are mostly developmental language researchers. The participants wearing the recorder are therefore usually children. That said, the creators of HomeBank have made it clear that adult LFSE data are also welcome at their repository, and so we expect that other participant groups will also be represented in HomeBank's archives in the future. HomeBank data are shared at different access levels (see section 3. 2. 4 for more details). If readers visit the HomeBank webpage (homebank. talkbank. org), they will be immediately able to download and inspect excerpts that have been approved for public distribution over the Internet.
At the time of writing, the fully public section of HomeBank contained one 14-hour, fully transcribed LFSE recording, 159 five-minute fully transcribed segments extracted from recordings from 53 families (3 segments per family), and four daylong recordings (audio only; no annotations), all from American English children  (VanDam, 2018a; 2018b; Fausey & Mendoza, 2018). At the same time, the members-only section contained around 5,000 hours of audio gathered from approximately 300 children (primarily acquiring North American English). Some existing LFSE recording datasets are not included on HomeBank because the researchers who collected the recordings did not ask for the ethical approval to share their recordings at the start of their project. These researchers cannot therefore deposit their recordings on a shared, secure archive.
Nonetheless, another mechanism for data re-use is to directly contact these data holders as potential collaborators on specific projects. For example, readers interested in child language and/or social development can reach out to members of the DARCLE (DAylong Recordings of Children's Language Environments, darcle. org; DARCLE, 2018) network, many of whom are data holders open to collaboration on new projects. In general, professional organizations that are focused on naturalistic language recordings (such as DARCLE) may be a good place to find non-shared existing LFSE datasets. In the long run, however, the goal for language scientists collecting LFSE recordings should be to maximize future data re-use. It is therefore critical for researchers planning to use LFSE recordings to get ethical approval for (eventual) sharing, a topic covered in the next section.
For example, there is no search function allowing one to find datasets bearing on participants of a certain linguistic group or age, or annotations relevant to, e. g. , talk addressed to familiar interlocutors vs. strangers. On OSF, one or more people can be listed as contributors to a project; contributors are administrators who can give other visitors to OSF the ability to view existing files, add new files, or both. Sometimes contributors are not authors (e. g. , a lab assistant who is using the platform to manage other data annotators). In these cases, nonauthor contributors can be tagged such that, when a DOIbased citation of the project is generated, their names will not be acknowledged. Anyone with a web browser can interact with the OSF framework, which is very light and, in our experience, works well even in places with low-bandwidth Internet connections.
Newly created projects are automatically set to be private (i. e. , are not searchable and not accessible by users other than the collaborators listed). Contributors with administrator rights can generate view-only, anonymized links of parts or all of the project, which is useful when sharing stimuli, data, analyses, and supplementary materials during peer review. Researchers can also set their projects to be, partly or wholly, publicly accessible, at which point anyone can visit it on the web. The fact that some parts can be public and some parts private is useful because it allows data holders to open some parts of their dataset to the public (e. g. , anonymized or secondary annotations), but still limit access to other sections. OSF also offers a number of Application Programming Interfaces (APIs) and plugins (i.
In the latter case, data holders may ask that the embargo is lifted at a certain date, or when an event occurs (e. g. , a paper describing the dataset is published, or when the data contributor passes away). The next level of sharing is Members, meaning that recordings are available only to the HomeBank members; "members" are researchers who have been vetted in a one-on-one interview with a representative from the HomeBank team. During the interview, conditions for data re-use and ethics are discussed, and applicants submit a signed data re-use agreement acknowledging their understanding of the rules. There is also a special section of HomeBank called "Sensitive", intended for sensitive data (e. g. , involving vulnerable participant populations). Data within this section are only available through a special password that members need to apply for separately by describing their long-format speech environment (LFSE) recordings Art.
24, page 10 of 21 need for access and gaining approval by the data holder and the HomeBank administration team. The final and most inclusive level for sharing is Public, i. e. , accessible by anyone on the Internet. Notably, the basic metadata for all the collections, including the Public, Membersonly, and Sensitive databases, is visible on the repository overview (https://homebank. talkbank. org/access/) for those considering data re-use. In our view, one limitation of all three repositories pertains to version control for annotations and the structure of data curation. All three repositories assume that someone will be in charge of the data and will update it as, for instance, more annotations become available. But it is not clear how these different versions of the annotations should be stored and made available.
Collecting new data Decisions regarding what recording device to use (i. e. , the hardware) should be related to decisions about the kinds of analyses that will be performed which, in turn, relate to the population being studied. In simple terms, readers working on American English children aged 0-3 years of age should strongly consider using the LENA device paired with the proprietary LENA software because that set of tools has been developed targeting their specific population. We will introduce LENA's hardware and, in broad strokes, the measurements its software provides in section 3. 3. 1; a fuller evaluation of the LENA software performance in American English learners and other populations is provided in section 4. 2 and Appendix C. Readers interested in other languages and age groups may also consider the LENA system, but should only do so with the understanding that the measurements it provides may be inappropriate or inaccurate for their use case.
LENA Only one piece of equipment comes with pre-processing software that works out of the box: the LENA. We will provide here a very short overview, since there is a host of information on the possibilities and limitations of the LENA products elsewhere (see, e. g. ,  Gilkerson & Richards, 2008 , for an introduction and Ganek & Eriks-Brophy, 2018a, for a review of published research using LENA). In a nutshell, LENA's hardware and software are durable, stable, and simple to use by all participants involved in developmental language studies (researcher, practitioner, parent, and child). We very briefly describe what the LENA system is like in the following section, but we recommend that interested readers check out Ganek and Eriks-Brophy (2018a)'s excellent review of research that has been carried out with LENA.
It provides an overview of the variety of ways in which researchers have harnessed the strengths of the LENA hardware and software. The LENA system's automated estimates are most reliable if the wearer is recorded for 12 or more consecutive hours within a single day, though the device can accommodate recordings being split over several days  (Xu, Yapanel, & Gray, 2009). Once the recording is done, the researcher/practitioner/other user connects the recording device to a Windows computer with access to LENA-licensed software (either locally or via a web browser) that automatically extracts and analyzes the recording. At the time of writing, there are two licenses ("Pro" and "SP") allowing users to keep a copy of the recordings. The Pro and SP licenses differ crucially in that the former does all speech processing in the local machine, whereas the latter uploads recordings to a remote ("cloud") storage location where they are processed, a set of statistics is returned, after which the original files are deleted.
Although there are no plans to discontinue the Pro license, there is also no budget for updating it as Windows systems change, meaning that it will become more difficult to maintain the necessary hardware and software needed to use the Pro system with time. If improvements are made to LENA algorithms in the future, Pro users may also not have access to them. In contrast, the SP license accesses LENA's software via an Internet connection-either from a computer on which LENA Hub software is installed or through a web browser such as Safari or Chrome. The ability to securely upload, process, and inspect results from recordings from any Internet-connected computer may be an important advantage for studies spread over multiple locations. The LENA system provides several types of automated annotations.
Others LENA products are less appealing when the derived measures are not relevant to the research question (e. g. , the study is on music exposure or adult participants). Some potential users may also decide that the performance is unlikely to be good enough for their research purposes, for instance, when a previous validation study on one's language or population of interest revealed levels of performance below what is necessary to estimate the effect of interest. Or, for example, manual inspection of a recording may reveal that the audio quality of the recordings is not high enough for one's needs (e. g. , for a researcher interested in the detailed acoustic properties of sounds like 's' and 'sh' in English). There are many alternative recording devices for LFSE recordings-any wearable, lightweight recording device is a candidate.
e. , transcription of the audio), and thus users must invest time to cull just the relevant steps for their own work. That said, there are several excellent tutorials available should researchers want to take on this task (e. g. , KALDI for dummies, 2018). Although the there is no real open-source, languagegeneral, and population-general version of LENA, there is a relatively easy-to-use, open alternative being developed: DiViMe (Le  Franc et al. , 2018; ACLEW/DiViMe, 2018). long-format speech environment (LFSE) recordings Art. 24, page 14 of 21 As of July 2018, DiViMe contains two alternative voice activity detection routines, a talker diarization system and an evaluation package. Although DiViMe can only be used on the command line, its creators are motivated to make it easy to use by anyone with basic technical knowledge.
g. , "sounds like a child and speaks close to the microphone") and age-related features. As this description hopefully makes obvious, it is unlikely that any individual researcher has the time and resources to train another general purpose, multi-age system to this level of accuracy. Readers may wonder whether it is truly necessary to try and attain adaptable automated analysis tools for LFSE recordings specifically. Evidence shows that it is: we recently helped organize the "DIHARD Challenge"  (Ryant et al. , 2018 ; see also  Ryant et al. , 2019) , a competition in which speech technology teams applied cutting-edge solutions to voice activity detection and talker diarization of conversational speech. We found that all of the systems performed worse on the LFSE recording data than in the other, non-LFSE datasets included in the challenge.
The current reality is that speech technology specialists are not targeting recordings of real-life conversations. As a result, they are not developing tools that generalize to the very challenging LFSE recording context. Solving this issue will necessarily require that researchers who collect such datasets share at least some portion of them, so that they can be included in similar challenges to promote further development of tools that can perform well on recordings of this type (see also  Schuller et al. , 2017; 2019).
Manual annotations and subsequent analyses In this section, we provide advice as to when and how to gather manual annotations. We believe that, for many research questions, it is not feasible to rely exclusively on out-of-the-box automatic measures because most of the relevant speech technology tools are still in their infancy. The possible exception to this warning are studies that are designed to be based on American English learners aged 0-3 years with the LENA system's welldocumented outcome measures (e. g. , adult word counts). For every other case, including when LENA is used with non-validated populations, some amount of manual annotation is unavoidable. However, this should not deter readers: If one has a clear idea of what the goal is, it may be possible to devise a form of annotation that serves one's purposes and is feasible, despite the large scale that LFSE recording datasets tend to have.
Database management Most people will not annotate full LFSE recordings, but focus annotation instead on short clips extracted from them. That means that, for each full recording, one also typically has a set of smaller sound files with time-aligned annotations. For certain tasks, researchers may want to extract even smaller clips for different levels of analysis, for example, individual vocalizations from multi-minute clips. These desired features lead to a nested representation of the data file: vocalizations are found within annotated clips, which themselves are only part of a very long (and only partially annotated recording). This situation can quickly become much more complicated. Imagine that you have done a first pass through the recording to segment out periods with speech (e. g. , using LENA's software or DiViMe), and have produced one clip, from which you extract "vocalizations" for trained annotators to transcribe.
e. , recordings belong to target participants, clips belong to recordings, vocalizations belong to speakers within clips, etc. , and (b) its heterogeneous nature, e. g. , recordings and clips are audio files, target participants are represented by clusters of metadata, annotations only partially cover the recording duration, and annotations might be stored in different formats or produced by different annotators. To our knowledge, there is only one open source software option that fulfills all these needs and more: the Emu Speech Database Management System (Emu-SMDS;  Winkelmann, Harrington, & Jänsch, 2017). Some of the strengths of Emu-SMDS include: -Database managers define the global architecture, which allows the system to perform checks for well-or ill-formedness of certain annotations; -It does not assume that the user will perform exhaustive annotation in the file and therefore does not automatically attempt to load the whole sound file onto memory when inspecting a subpart; -It does not rely on files being available locally, but rather allows users to access them (and annotate them) via a web browser.