dataesr/scientific-paragraphs-categorization huggingface dataset:
Optional Short Vowels Arabic text contains diacritics representing most vowels that affect the phonetic representation and give different meaning to the same lexical form. 4 Nowadays, the modern version of Arabic is written without diacritics, creating a one-to-many, unvocalized-tovocalized, ambiguity (Alkharashi 2009), which gives mutually incompatible morphological analyses for the same surface form. As such, most Arabic texts that appear in the media (whether in printed documents or digitized format) are undiacritized. This is comprehensible for native Arabic speakers, but not for a computational system. The simplification made by ignoring such diacritics had led to structural and lexical types of ambiguity because different diacritics represent different meanings. These ambiguities can only be resolved by contextual information and an adequate knowledge of the language (Benajiba,  Diab, and Rosso 2009a).
Lack of Uniformity in Writing Styles Arabic has a high level of transcriptional ambiguity: An NE can be transliterated in a multitude of ways  (Shaalan and Raza 2007). This multiplicity arises from both differences among Arabic writers and ambiguous transcription schemes  (Halpern 2009). The lack of standardization is significant and leads to many variants of the same word that are spelled differently but still correspond to the same word with the same meaning, creating a many-to-one, variants-to-well-formed, ambiguity. For example, transcribing (also known as "Arabizing") an NE such as the city of Washington into Arabic NE produces variants such as , , ,. One reason for this is that Arabic has more speech sounds than Western European languages, which can ambiguously or erroneously lead to an NE having more variants.
A gazetteer entry provides internal evidence to fully or partially match a candidate NE in the input. Whenever a predefined NE that appears in the relevant gazetteer is detected in the input text, the NER system should recognize it directly as an NE of this type. Very large gazetteers are publicly available from the CJK Dictionary Institute 10  under license agreement in the form of Arabic person, organization, company, and location name databases. However, researchers who find these resources difficult to acquire build their own gazetteers from different resources such as the Web and from organizations (Benajiba and Rosso 2008;  Shaalan and Raza 2009). Some systems used a blacklist  (Shaalan and Raza 2009)  that allows for discarding of negative evidence. A filtering mechanism is used to reject incorrect matches.
Language-Specific Features These features are related to certain aspects of the Arabic language. Table 8 lists subcategories of language-specific features. They specifically describe part-of-speech (POS), morphological features, and base-phrase chunks (BPC). Arabic words generally carry rich morphological information  (Marton, Habash, and Rambow 2010) , some of which includes noun-adjective agreement and special markings indicating nominals in compounds. The MADA toolkit has been found to be very useful in generating a number of informative language-specific features for each input word (Habash, Rambow, and Roth 2009). One of these features is the POS morpho-syntactic tag, which plays a significant role in Arabic NLP. An Arabic NE usually consists of either noun (NN) or proper noun (NNP) tags.
It facilitates the development of rule-based NER systems by providing the user with the capability of implementing grammatical rules as a finite state transducer using JAPE. It also has an Arabic plug-in that contains a tokenizer, gazetteers, an OrthoMatcher component, and a grammar, all of which are used within a simple Arabic rule-based NER application built as a part of GATE. GATE can be used to extract basic entities, such as date, name, location, organization, and so on. A number of scholars have used the GATE environment in their research studies on Arabic NER, including  Maynard et al. (2002) ,  Elsebai, Meziane, and Belkredim (2009) ,  Elsebai and Meziane (2011), and Abdallah, Shaalan, and Shoaib (2012). NooJ: 13  This is a freely available linguistic development environment for many languages.
The toolkit is language-, domain-, and genre-independent. It supports the development of different language processing tasks such as POS tagging, spelling correction, NE recognition, and word sense disambiguation. The NER component is based on hidden Markov models and the learned model can be evaluated using k-fold cross validation over annotated data sets. LingPipe recognizes corpora annotated using the IOB scheme. The LingPipe NER system has been applied by ANERcorp to demonstrate how to generate a statistical NER model for Arabic; the details and results are presented on the toolkit's official Web site. AbdelRahman et al. (2010)  used ANERcorp to compare their proposed Arabic NER system with LingPipe's built-in NER.
Machine Learning Tools In the Arabic NER literature, the ML tools of choice are data-mining-based tools that support one or more ML algorithms, such as Support Vector Machines (SVM), Conditional Random Fields (CRF), Maximum Entropy (ME), hidden Markov models, and Decision Trees. These tools are YASMET, CRF++, YamCha, and WEKA. They all share the following features: a generic toolkit, language independence, absence of embedded linguistic resources, a requirement to be trained on a tagged corpus, the performance of sequence labeling classification using discriminative features, and a suitability for the pre-processing steps of NLP tasks. YASMET:  15  This free toolkit, which is written in C++, is applicable to ME models. The toolkit can estimate the parameters and computes the weights of an ME model.
The word is selected with or without short vowels. Then it is segmented and compatibility-checked for the correct combination of its segments, producing all possible analyses of the input word. BAMA transliteration of the output makes it readable; this is more useful for readers who do not have the ability to read the Arabic script but are familiar with Latin script. In addition, the transliteration 20  output can be converted directly to Unicode Arabic with a minimal amount of automatic processing. BAMA has been made available through the Linguistic Data Consortium. Some of the Arabic NER studies that rely on BAMA for performing morphological analysis include  Farber et al. (2008) ,  Elsebai, Meziane, and Belkredim (2009), and Al-Jumaily et al. (2012). (MADA+TOKAN). 21 MADA stands for Morphological Analysis and Disambiguation for Arabic.
The combined package is built on top of BAMA as a natural successor that builds on prior successes and meets the growing requirements of many Arabic NLP applications  (Habash, Rambow, and Roth 2009). The package consists of two components. Morphological analysis and disambiguation are handled in the MADA component. Morphological analysis also supports the ability to tokenize and stem deterministically. Because there are many different ways to tokenize Arabic (tokenization is a convention adopted by researchers), the TOKAN component allows the user to specify any tokenization scheme that can be generated from disambiguated analyses. The MADA+TOKAN package provides one solution to all of the basic problems in Arabic NLP, including tokenization (the segmentation of clitics from a word with attendant spelling modifications), diacritization (insertion of disambiguating short-vowel diacritics), morphological disambiguation (determining the full morphological information for each word given its context), POS tagging (determining specific morphological information for each word), stemming (reducing each word to its base form), and lemmatization (determining the citation form lemma of the set of word lexemes to which each word in the data belongs).
The research assumes that an NE appears close to lexical triggers no more than three words from the cue word and that the NE has a maximum length of seven words. Some names may be attached to different types of lexical triggers and to more than one lexical trigger in the same phrase. For example, the phrase (Dr. Khaled Shaalan the Chairman of IT Department) has the lexical triggers (Dr) and (Chairman Department). In  Abuleil's (2004)  work, Arabic NER is part of a question-answering system. The system starts by marking the phrases that could include names. Afterwards, it builds up a graph that represents the words in these phrases and the relationships between them. Finally, rules are applied to classify and generate the NEs before saving them in a database. The system has been evaluated on 500 articles from the Al-Raya newspaper, published in Qatar.
The filter improved the overall Precision from 84% to 90%; the Recall was very high at 97. 5%. Mesfar (2007)  used NooJ to develop a rule-based Arabic NER system. The system identifies the following NE types: person, location, organization, currency, and temporal expressions. The Arabic NER is a pipeline process that goes through three sequential modules: a tokenizer, a morphological analyzer, and Arabic NER. Morphological information is used by the system to extract unclassified proper nouns and thereby enhance the overall performance of the system. An evaluation corpus was built from Arabic news articles extracted from the Le Monde Diplomatique newspaper. The reported results based on individual NE types were as follows: Precision, Recall, and F-measure range from 82%, 71%, and 76% for Place names to 97%, 95%, and 96% for Time and Numerical expressions, respectively.
Alkharashi (  2009 ) described the formation of an Arabic person name from root and pattern using the traditional Arabic morphology and suggested relevant computational resources. The author introduced a set of database tables in order to assist Arabic NER: root-pattern, a frequency list of roots, and lexical trigger tables. A corpus was created from Saudi person names with specific person name tags: root of person NE, features indicating the possibility of affixation, and gender characteristics. The main objective was to recognize the constituents of the person NE, these being the simple form, the affix, and connectors. For example, the name of the Umayyad caliphate (Al-Waleed bin Abd Al-Malik) has (Malik) and (Waleed) as simple names, (Abd) and (Al) as name prefixes, and (Bin) as a name connector.
This system at present includes 19 languages and is able to analyze large volumes of news text. The EMM-NewsExplorer architecture is optimized for ruled-based systems. The adaptation resulted in a rule-based Arabic NER system (RENAR; Zaghouani 2012), which uses a handwritten set of language-independent rules  (Steinberger, Pouliquen, and Ignat 2008)  in combination with specific resources for Arabic. Rules are described using the following notations: "\w+" for an unknown word, "\b" for an obligatory word boundary (white space, possibly with punctuation), "+" for one or more elements, and "*" for zero or more elements. For example, consider the rule: Organization BEG + \b known Name\b name Infix * \b Known Name * \b Organization END This rule recognizes complex company names such as (company of Mohamed Abu Al-Majd and Brothers), which include person (known) names (Mohamed Abu Al-Majd) and the preceding and following organization internal evidence trigger (company) and (Brothers), respectively.
Advances in human language technology require an ever increasing amount of data and annotation. The number of current state-of-the-art of Arabic linguistic resources is still insufficient compared with Arabic's actual importance as a language. Many existing Arabic NER resources are annotated manually or are only available at significant expense. We have described some research that adopted semi-automatic (bootstrapping) methods in order to enrich Arabic NER resources from diverse text types such as Web sources and (multilingual) corpora developed within evaluation projects. In the Arabic NER field, NEs falling under proper names representing person, location, and organization names are commonly applied to newswire domains, reflecting the importance of these limited NEs in this domain. We have described three main approaches that have been used to develop Arabic NER systems: linguistic rule-based, ML-based, and hybrid approaches.
We reviewed many attempts to select features that investigate the sensitivity of each entity when applied to different sets of features. We showed how researchers applied different techniques that benefit differently from the enabled features and obtain different results for varying NE types. Some suggest that NER for Arabic use not only language-independent features but also Arabic-specific features. Researchers sometimes exploit language-independent features based on promising variables, such as lexical and orthographic features, to overcome the problems related to the Arabic language and orthography. Lexical features avoid complex morphology by extracting the word prefix and suffix sequence of a word from the character n-gram of leading and trailing letters. Orthographic features attempt to overcome the lack of capitalization for NEs in Arabic by relying on the corresponding English capitalization of NEs.
WordNet (Fellbaum 2005) is a large lexical database that originally implemented for English. Arabic WordNet is still a limited lexical resource. On the other hand, Wikipedia is a popular ubiquitous source of corpus data for information extraction because of its size, currency, rich semi-structured content, diverse topics, and closer resemblance to web text than newswire(Balasuriya et al. 2009; Mohit et al. 2012 ).
A possible explanation is that the dataset consists primarily of fiber users with 1GB bi-directional links, and the lack of buffering is simply due to the very high access link speeds. In this paper, we examine a different vantage point to characterize the size and impact of excessive buffering, and this is on the end-host itself. By placing ourselves on the end-host, we are better able to measure and understand the act ual performance of an application as experienced by the end-user. It is extremely challenging to close the loop and actually state whether a bad performance episode indeed has an impact on the user -such observations are very subjective and depend on application, mood, expectations, etc. In our work, we simply measure how some application characteristics are degraded by excessive buffering, which serves as a proxy for the actual experience of the end-user.
Dataset We study bufferbloat using data passively collected directly on users' machines with the Hostview  [10]  monitoring tool. HostView was developed to study user perception of network performance. It collects network and system performance as well as user feedback on the quality of network experience. In this paper, we only study the network performance of active application connections. HostView extracts network performance from packet-header traces collected with libpcap. It labels traces with information about the network environment: (i) the hash of the MAC address of the default gateway, (ii) the interface name, (iii) the type of the interface. HostView uploads traces from the user machine to a server every four hours. We merge all consecutive traces collected in a single environment collected less than one hour apart into a session.
We then process packets in each session with tcptrace  [11]  to obtain round-trip time (RTT) estimates over each TCP connection in the session. Note that because the data is collected at the end-host we can only estimate RTTs for upload traffic and at the beginning and end of every connection (since the RTT estimations relies on ACKs). This is not a problem for our study because bufferbloat has mainly been observed in upload traffic  [1]  [4]  [5]. HostView also maps connections to the application executable name that originated them using GT  [12]. This allows us to identify the application running within particular intervals. Users, mostly computer scientists, ran HostView between November 2010 and April 2012. We study data from 55 users who contributed at least four hours of cumulative trace over ethernet.
Bufferbloat detection Given RTT samples computed from packet traces collected on the end-host, we must detect periods when packets are experiencing bufferbloat. We develop a methodology that works in two steps. First, we identify instances of sustained buffering, which we call buffering episodes, using a heuristic inspired by coDel  [3]. Then, we apply this heuristic to the HostView data to identify episodes of excessive buffering and quantify their magnitude. From these, we select the largest episodes and analyze them in detail. This section first presents a brief overview of the heuristic coDel uses to detect buffering, then it presents how we extend this heuristic to work only with the data that we have available at end-hosts. Finally, we present our analysis of buffering on HostView data and the definition of bufferbloat we will consider for the rest of the paper.
Identifying bufferbloat We apply the heuristic described in the previous section to the HostView data to identify the beginning and end of buffering episodes. Not all buffering episodes are significant, in fact most buffering is just part of normal TCP/network behavior. We now want to select a subset of the buffering episodes that are significant enough to potentially impact users. Figure  2  presents the cumulative distribution of the additional delay per episode (the x-axis is in log scale). We compute the additional delay for an episode using the area under the curve defined by the additional delay corresponding to the successive RTT samples within this episode. For this computation we only take into consideration samples for which the baseline latency was computed outside of an episode.
The HostView dataset has the application executable name for 84% of TCP connections. We exclude the 16% of connections for which we don't have an application name from the rest of the analysis in this section. Table  1  presents the rank of application classes based on the percentage of the bufferbloat episodes with traffic from each application class. We see that the vast majority of episodes affected interactive and web applications. Both these applications are sensitive to additional delays of few hundreds of milliseconds. Although bulk applications are often pointed out as the cause of bufferbloat, not all bufferbloat episodes co-occurred with bulk applications. Note that we are only measuring one end-host behind the buffer, so it is possible that another end-host is doing a bulk transfer that is causing bufferbloat but we don't observe it in our data.
In the project, the perception task is based on both cameras and laser sensors. A modeling task provides the metric and the topologic maps describing the airport environment with the different checkpoints to be reached successively. Path planning has to compute a path allowing the robot to reach the aircraft inspection site starting from its initial position. Furthermore, the localization method has to provide the pose of the robot relatively to the aircraft and to a global fixed frame. The robot's motion will be determined by a supervisor, a high level controller, which will choose the most appropriate controller depending on the context. In our case, several controllers will be available: a trajectory following controller allowing to reach the inspection site; a Visual Servoing (VS) controller allowing to reach each checkpoint thanks from visual data of the aircraft taken by the embedded cameras; an OA controller ensuring collision free execution of the task.
Simulation Environment The Air-Cobot-Robot will be using ROS. Since the recommended simulator for ROS is Gazebo we chose this simulator as our test environment. However, in order to visualize our findings we figured that neither ROS, Gazebo or even RVIZ can provide a sufficient presentation of our findings in just a few images. Thus, in order to present our simulation results we used the ROS recording tool (ROS-bag) and a ground truth Gazebo plug-in to capture the robot and obstacle positions and orientation. Those recordings are then read into Matlab and plotted as trajectories in a 2Dplot. At "http://homepages. laas. fr/mfutterl/" we provide a recording (video file) of the second experiment. In Figure  8  we present a scenario where the OA controller is coupled with a VS controller. The simulation environment is set up with the projects requirements in mind (provided video file).
Especially the second obstacle is interesting as it offers the robot a convenient trap forcing it to recomputing the spiral center point two times during its avoidance (C. 2 and C. 3). For the other two obstacles, the recomputation of the SCP is not necessary. By consulting the results of this experiment we can conclude that our approach is applicable and works as expected. In Figure  9. top the linear velocity for the previous experiment is presented. As before, the data is obtained with the help of a Gazebo ground truth plug-in. We can clearly see the time intervals in which the robot was using the VS controller ( 1. 8 m s ) and the OA controller ( 0. 4 m s ). The slight "up and downs" dur- F. 2 ing the respective phases are due to the skid steering drive controller and are matter of optimization at the moment.
In addition to representing electrophysiology data in memory for the purposes of analysis and visualization, the Python implementation provides a set of input/output (IO) modules for reading/writing the data from/to a variety of commonly used file formats. Support is included for formats produced by most of the major manufacturers of electrophysiology recording equipment and also for more generic formats such as MATLAB. Data representation and data analysis are conceptually separate: it is easier to write robust analysis code if it is focused on analysis and relies on an underlying package to handle data representation. For that reason, and also to be as lightweight as possible, the Neo object model and the associated Python package are deliberately limited to representation of data, with no functions for data analysis or visualization.
, 2013)  based on the HDF5 file format. A comparison of 19 different biomedical data formats is provided by  Schlögl (2010). The second approach is to define a language that can describe the structure of data files in different formats, and then write descriptions of each of the formats of interest using this language. Conversion between data formats, or reading a data file into software, can then be fully automated. This approach was taken by  Durka and Ircha (2004) , who developed the SignalML markup language for describing biomedical time series. The third approach is to define an application programming interface (API) that takes care of the data conversion, and implement this as a library which can be used by software developers to provide uniform access to data, independent of the file format.
An advantage of this compared to the other two approaches mentioned above is that it encourages modularity in software development: the data-conversion library defines certain data structures which can then be used by developers of analysis and visualization tools, which thereby gain greater interoperability. A previous effort to define an API for electrophysiology data in neuroscience was undertaken by a consortium of recording hardware manufacturers, which in 2000 began development of the Neuroshare API (http://neuroshare. org). The Neuroshare API is available as a C library and as a MATLAB (The Mathworks Inc. ) extension. These are distributed as compiled dynamic-link libraries (DLLs), one per manufacturer; the source code is generally not available, and most are only available for the Windows operating system.