dataesr/scientific-paragraphs-categorization huggingface dataset:
Tools for solving level populations and line excitation Several software tools are available online for the calculation of line emission. While not covering all, the tools presented and discussed at this workshop are listed alphabetically in Table  2  for quick reference and comparison. Before comparing the tools, it is worth noting that they are built to achieve different levels of precision at the cost of computation time. As such, codes like LIME, MOLLIE, and RADMC-3D are very accurate in 3D, but slow compared to CLOUDY and DESPOTIC which solve for chemistry and temperature in 1D and are intended for larger parameter studies that can be analyzed fast. In terms of chemical networks, CLOUDY is the most comprehensive tool with 625 species and a large range of permitted densities that allows for application to models on scales from clouds and HII regions to galaxies [e.
g. [56] [57] [58]. DESPOTIC has been applied in post-processing to galaxy simulations mainly [e. g. 21, 22, 59]  but also to smaller regions such as the Galactic central molecular zone  [60]. As mentioned above, CLOUDY and DESPOTIC are restricted to problems in 1D whereas the remaining tools in Table  2  work in 3D. LIME, MOLLIE, and RADMC-3D do exact radiative transfer in 3D and are typically used on smaller scales where non-symmetric features such as filaments and turbulence become important [e. g 61,62]. ART 2 also does radiative transfer in 3D including the continuum from far-UV to radio wavelengths and incorporating the resonant line Lyα [e. g 29,63,64] (an updated version of the code including CO and some prominent far-infrared fine-structure lines such as CII, OI, OIII and NII will be out later this year).
Finally, MAIHEM is unique in taking into account the non-equilibrium effects of turbulence on line emission (c. f. Sec. 3. 3). (1  [13] 10 -6 -10 15 cm - 3  1D 625 species (including atomic ions); CHIANTI, Stout, LAMDA databases no (9)   DESPOTIC (2)  [31] Cool atomic and molecular ISM 1D C, O, H, and He, plus a super-species M that represents a composite of metallic elements no (9)   LIME (3)  [26] 10 2  -10 12 cm -3 3D LAMDA database yes MAIHEM (4)  [32],  [65] ,  [66]  Has been tested at 0. 4-1200 cm -3.
Correcting heavy-element energy levels Predictions on line emission modeling depends on the reliability of fundamental atomic data. Software tools such as those discussed in the previous section use theoretical and experimental atomic and molecular rates and energies to calculate emissivities and line wavelengths respectively, mostly from external databases as CHIANTI  [67, 68]  or NIST. Nowadays, theoretical energy levels of heavy many-electron ions disagree with experimental values, leading to incorrectly predicted wavelengths. This happens in all the energy and wavelength ranges. For instance, there are differences between the energies of Fe II reported by the NIST and CHIANTI databases, where several terms reported by NIST are missing in CHIANTI. Similarly, efforts to fit the spectra of X-ray satellite observations have accentuated the necessity of including transitions between levels with high main quantum number n in the theoretical calculations [e.
As different codes calculate new sets of data as well as fit spectra (e. g. FAC  [81] , GRASP  [82] , AUTOSTRUCTURE  [83] ) and new data are stored in databases such as CHIANTI  [67, 68] ) and HULLAC  [71] , there is better general agreement between theory and experiments [see for  example 84]. However, there are still lines which remain unidentified, while others are poorly described because they are blended or have wrongly assigned transitions  [85]. Theoretical and experimental energies for levels obtained from the CHIANTI (v7,  [86] ) database. (Top) Fe 16+ : Observed energy levels (up to n=5):  [87] ; observed energy levels (n=6,7) from the wavelength measurements of  [88]. (Bottom) Mg + : theoretical energies:  [76] ; observed energies from NIST:  [89]  To illustrate this, two possible cases of disagreement between theory and experiment, both taken from CHIANTI (v7,  [86] ), are plotted in Fig.
For a simulation with a few million cells or particles, the problem becomes extremely computationally expensive, and the non-local couplings induced by radiation render it practically intractable with the computational means available at present. An alternate solution is to solve for the hydrodynamics and the microphysics separately, in post-processing. In this approach, a code such as those listed in Table  2  is required to simulate the microphysics. CLOUDY  [13]  performs 1D spectral synthesis simulations of astrophysical clouds, accounting for all ions of elements up to Zn, a large number of molecular species, and all important microphysical processes. The propagation of radiation is done with the escape probability formalism, which is an expedient, and generally accurate approximation  [102].
However, a few pathological cases exist for which escape probabilities are known to not give correct results  [103]. Exact radiative transfer methods are required if a code is to be used in conjunction with hydrodynamical codes. The CLOUDY team will seek to implement exact radiative transfer, eventually with full treatment of the interactions of dust and radiation. Even with these improvements, however, due to their 1D nature these simulations will not capture the time-varying three-dimensional radiative fields that exist in nature. An exact 3D treatment of radiative transfer will be needed to connect the microphysics and 3D hydrodynamical models.
Implementing turbulence and shocks in simulations of the ISM In addition to the importance of modeling stars as described in the previous two subsections, another important source of heating and ionization is turbulence and shocks. The introduction of density gradients and additional sources of ionization through these processes can have a significant impact on the observed spectrum, and hence the emission line diagnostics. Examples of how different levels of turbulence affect the density structure of ISM gas is shown in Fig. 5. One solution to understanding the effects of turbulence is through the publicly available MAIHEM  10  code. This FLASH (version 4. 4) based code tracks ionizations, recombinations, and species by species radiative cooling for hydrogen, helium, carbon, nitrogen, oxygen, neon, sulfur, calcium, and iron  [32, 65, 66].
By including a solenoidal stirring mechanism, turbulence is introduced into the gas that then propagates through all spatial scales. At the meeting, there was an interest in applying MAIHEM to Baldwin-Phillips-Terlevich (BPT) diagrams  [1, 113]  and other UV diagnostics. Specifically, turbulence in the gas might help explain the observed dispersion in some spatially-resolved BPT diagrams of nearby AGN  [114]   In addition, the treatment of shocks is particularly complex as it requires understanding the physical conditions in the emission line gas before, during, and after the shock event. These events can cause collisional ionization and the resulting recombinations may release additional ionizing photons that interact with surrounding material, depending on the shock velocity. A new database of shock models calculated using the MAPPINGS V 11  code  [115]  was presented at the workshop, and details can be found by consulting the documentation available on the Mexican Million Models database (3MdB) website  12.
Discussion: How can we as a community move forward? This workshop was the first of its kind (to our knowledge) to attempt to create a community of astronomers simulating line emission and working on the comparison with observations. In order to continue the sharing of knowledge within the community, we discussed ways in which to make the material of the workshop available to a larger group and encourage collaborations in the future. For this workshop, it was decided to use zenodo. org where all speakers had the option to upload their slides together with a recording of their talk (where available) to a community space  15. This approach automatically provides a DOI for each upload for easy reference and we hope that future workshops of this kind will likewise attempt to make the presentations publicly available.
The CLOUDY community Small tutorials on using CLOUDY have been uploaded to youtube by individuals not part of the CLOUDY developer team and such video uploads are much appreciated by many and strongly encouraged by the CLOUDY community. It was suggested to establish a CLOUDY-branded youtube channel or youtube playlist for these videos. Also, many people are not aware that you can edit the databases used by CLOUDY to for example include new species. The CLOUDY quickstart guide has been noted to be an excellent tool for introducing new researchers and students to making predictions with the software. Maintaining it and keeping it up-to-date will be a priority. The CLOUDY team is always looking for more participation on the Yahoo Groups forum. A goal is that questions and discussions will take place there among users, rather than solely between a user and cloudy developers.
LIME LIME resides on the code hosting platform GitHub (https://github. com/lime-rt/lime) from where it can be downloaded. A small team of active developers maintain and develop the package and minor releases appear regularly, approximately once per year. Users that experience problems when using LIME or have ideas to improve on the code, are welcome to create issues on GitHub where they will be answered by one or more developers. It is also possible to fork the code and develop it for specific needs. Documentation exists in the form of a written PDF manual as well as an issue tracker. Both are available on GitHub. A bi-annually summer school on Monte Carlo techniques in radiative transfer problems, aimed at PhD-students, is held at University of St Andrews School of Physics & Astronomy and this school offers both a lecture and tutorials on how to use LIME.
This paper presents a comprehensive test of the principal tasks in document image analysis (DIA), starting with binarization, text line segmentation, and isolated character/glyph recognition, and continuing on to word recognition and transliteration for a new and challenging collection of palm leaf manuscripts from Southeast Asia. This research presents and is performed on a complete dataset collection of Southeast Asian palm leaf manuscripts. It contains three different scripts: Khmer script from Cambodia, and Balinese script and Sundanese script from Indonesia. The binarization task is evaluated on many methods up to the latest in some binarization competitions. The seam carving method is evaluated for the text line segmentation task, compared to a recently new text line segmentation method for palm leaf manuscripts.
Balinese Script and Language Although the official language of Indonesia, Bahasa Indonesia, is written in the Latin script, Indonesia has many local, traditional scripts, most of which are ultimately derived from Brahmi  [18]. In Bali, palm leaf manuscripts were written in the Balinese script in the Balinese language, in the ancient literary texts composed in the old Javanese language of Kawi and Sanskrit. Balinese language is a Malayo-Polynesian language spoken by more than 3 million people, mainly in Bali, Indonesia (www. omniglot. com/writing/balinese. htm) [19]. Balinese is the native language of the people of Bali, known locally as Basa Bali  [18]. The alphabet and numbers of Balinese script are composed of ±100 character classes including consonants, vowels, and some other special compound characters.
Corpus In Cambodia, Khmer palm leaf manuscripts (Figure  2 ) are still seen in Buddhist establishments and are traditionally used by monks as reading scriptures. Various libraries and institutions have been collecting and digitizing these manuscripts and have even shared the digital images with the public. For instance, the École Française d'Extrême-Orient (EFEO) has launched an online database (http: //khmermanuscripts. efeo. fr) [20]  of microfilm images of hundreds of Khmer palm leaf manuscript collections. Some digitized collections are also obtained from the Buddhist Institute, which is one of the biggest institutes in Cambodia responsible for research on Cambodian literature and language related to Buddhism, and also from the National Library (situated in the capital city, Phnom Penh), which is home to a large collection of palm leaf manuscripts.
Moreover, a standard digitization campaign was conducted in order to collect palm leaf manuscript images found in Buddhist temples in different locations throughout Cambodia: Phnom Penh, Kandal, and Siem Reap  [21]. In Cambodia, Khmer palm leaf manuscripts (Figure  2 ) are still seen in Buddhist establishments and are traditionally used by monks as reading scriptures. Various libraries and institutions have been collecting and digitizing these manuscripts and have even shared the digital images with the public. For instance, the École Française d'Extrême-Orient (EFEO) has launched an online database (http://khmermanuscripts. efeo. fr) [20] of microfilm images of hundreds of Khmer palm leaf manuscript collections. Some digitized collections are also obtained from the Buddhist Institute, which is one of the biggest institutes in Cambodia responsible for research on Cambodian literature and language related to Buddhism, and also from the National Library (situated in the capital city, Phnom Penh), which is home to a large collection of palm leaf manuscripts.
Ancient palm leaf manuscripts contain artifacts due to aging, foxing, yellowing, strain, local shading effects, low intensity variations or poor contrast, random noises, discolored parts, fading, and other types of degradation. The second challenge is the complexity of the script. The Southeast Asian manuscripts with different scripts and languages provide real challenges for document analysis methods, not only because of the different forms of characters in the script, but also because the writing style of each script (e. g. , how to join or separate a character in a text line) differs. It ranges widely from a binarization process  [23] [24] [25] , text line segmentation  [26, 27] , and character and text recognition tasks  [25, 28, 29] , to the word spotting methods  [30]. In the domain of DIA, handwritten character and text recognition has been the subject of intensive research during the last three decades.
Document Image Analysis Tasks and Investigated Methods Heritage document preservation is not just about converting physical documents into document images. With many physical documents being digitized and stored in large document databases, and then sent and received via digital machines, the interest and demand grew to require more functionalities than simply viewing and print the images  [33]. Further treatment is required before the collection of document images can be explored more extensively. For example, a more specific research field needed to be developed to add machine capabilities for extracting information from these images, reading text on a document page, finding sentences, and locating paragraphs, lines, words, and symbols on a diagram  [33]. In this work, the methods for each DIA task were investigated for palm leaf manuscripts.
Previous studies on isolated character recognition in palm leaf manuscripts have already been reported, but only with the Balinese script as the benchmark dataset  [28, 29]. In that first work, an experimental study on feature extraction methods for character recognition of Balinese script was performed  [28]. For the second work, a training-based method with neural network and unsupervised feature learning was used to increase the recognition rate  [29]. In this paper, we will conduct a broader evaluation of the robustness of the methods previously tested on Balinese script, using the other two palm leaf manuscripts with Khmer and Sundanese scripts. In the next subsections, we provide a brief description of the methods. For a detailed description of each method, interested readers can refer to our previous works.
Previous studies on isolated character recognition in palm leaf manuscripts have already been reported, but only with the Balinese script as the benchmark dataset  [28, 29]. In that first work, an experimental study on feature extraction methods for character recognition of Balinese script was performed  [28]. For the second work, a training-based method with neural network and unsupervised feature learning was used to increase the recognition rate  [29]. In this paper, we will conduct a broader evaluation of the robustness of the methods previously tested on Balinese script, using the other two palm leaf manuscripts with Khmer and Sundanese scripts. In the next sub-sections, we provide a brief description of the methods. For a detailed description of each method, interested readers can refer to our previous works.
Handcrafted Feature Extraction Methods Since the beginning of pattern recognition research, many feature extraction methods for character recognition have been presented in the literature. In our previous work  [28] , we investigated and evaluated the performance of 10 feature extraction methods with two classifiers, k-NN (k-Nearest Neighbor) and SVM (Support Vector Machine), in 29 different schemes for Balinese script on palm leaf manuscripts. After evaluating the performance of those individual feature extraction methods, we found that the Histogram of Gradient (HoG) features as directional gradient-based features  [9, 49]  (Figure  5 ), the Neighborhood Pixels Weights (NPW)  [50]  (Figure  6 ), the Kirsch Directional Edges  [50] , and Zoning  [12, 32, 50 ,51] (Figure  7 ) give very promising results.
We then proposed a new feature extraction method applying NPW on Kirsch edge images (Figure  8 ) and concatenated the NPW-Kirsch with two other features, HoG and Zoning method, with k-NN as the classifier. and evaluated the performance of 10 feature extraction methods with two classifiers, k-NN (k-Nearest Neighbor) and SVM (Support Vector Machine), in 29 different schemes for Balinese script on palm leaf manuscripts. After evaluating the performance of those individual feature extraction methods, we found that the Histogram of Gradient (HoG) features as directional gradient-based features  [9, 49]  (Figure  5 ), the Neighborhood Pixels Weights (NPW)  [50]  (Figure  6 ), the Kirsch Directional Edges  [50] , and Zoning  [12, 32, 50 ,51] (Figure  7 ) give very promising results. We then proposed a new feature extraction method applying NPW on Kirsch edge images (Figure  8 ) and concatenated the NPW-Kirsch with two other features, HoG and Zoning method, with k-NN as the classifier.
Word Recognition and Transliteration In order to make the palm leaf manuscripts more accessible, readable, and understandable to a wider audience, an optical character recognition (OCR) system should be developed. In many DIA systems, word or text recognition is the final task in the processing pipeline. However, normally in Southeast Asian script the speech sound of the syllable change is related to some certain phonological rules. In this case, an OCR system is not enough. Therefore, a transliteration system should also be developed to help transliterate the ancient scripts on these manuscripts. By definition, transliteration is defined as the process of obtaining the phonetic translation of names across languages  [54]. Transliteration involves rendering a language from one writing system to another.
One drawback of RNN is the vanishing gradient problem. To deal with this problem, the LSTM architecture was introduced. The LSTM network adds multiplicative gates and additive feedback. Bidirectional LSTM is an LSTM architecture with two-directional (forward and backward) context processing. LSTM architecture is widely evaluated as a generic and language-independent text recognizer  [55]. In this work, the OCRopy (https://github. com/tmbdev/ocropy) [56]  framework is used to test and evaluate the word recognition and transliteration tasks for the palm leaf manuscript collection. OCRopy provides the functional library of the OCR system by using RNN-LSTM architecture (http://graal. hypotheses. org/786)  [57, 58]. We evaluated the dataset with unidirectional LSTM and the (Bidirectional LTSM) BLSTM architecture.
Datasets The palm leaf manuscript datasets for binarization task are presented in Table  1. For Khmer manuscripts, one ground truth binarized image is provided for each image, but for Balinese and Sundanese manuscripts, each image has two different ground truth binarized images  [17, 25]. The study of ground truth variability and subjectivity was reported in the previous work  [24]. In this research, we only used the first binarized ground truth image for evaluation. The binarized ground truth images for Khmer manuscripts were generated manually with the help of photo editing software (Figure  11 ). A pressure-sensitive tip stylus is used to trace each text stroke by keeping the original size of the stroke width  [59]. For the manuscripts from Bali, the binarized ground truth images have been created with a semi-automatic scheme  [17, [23] [24] [25]  (Figure  12 ).
The binarized ground truth images for Sundanese manuscripts were manually  [22]  generated using PixLabeler  [60]  (Figure  13 ). The training set is provided only for the Balinese dataset. We used all images of the Khmer and Sundanese corpuses as a test set because the training-based binarization method (ICFHR G1 method, see Section 5. 1) was evaluated for the Khmer and Sundanese datasets by using only the pre-trained Balinese training set weighted model. images have been created with a semi-automatic scheme  [17, [23] [24] [25]  (Figure  12 ). The binarized ground truth images for Sundanese manuscripts were manually  [22]  generated using PixLabeler  [60]  (Figure  13 ). The training set is provided only for the Balinese dataset. We used all images of the Khmer and Sundanese corpuses as a test set because the training-based binarization method (ICFHR G1 method, see Section 5.
Datasets The palm leaf manuscript datasets for text line segmentation task are presented in Table  2. The text line segmentation ground truth data for Balinese and Sundanese manuscripts have been generated by hand based on the binarized ground truth images  [17]. For Khmer 1, a semi-automatic scheme is used  [26, 59]. A set of medial points for each text is generated automatically on the binarization ground truth of the page image. Then those points can be moved up or down with a tool to fit the skew and fluctuation of the real text lines. We also note touching components spreading over multiple lines and the locations where they can be separated. For Khmer 2 and 3, an ID of the line it belongs to is associated with each annotated character. The region of a text line is the union of the areas of the polygon boundaries of all annotated characters composing it  [21, 27].
Datasets The palm leaf manuscript datasets for isolated character/glyph recognition task are presented in Table  3. For the Balinese character dataset, Balinese philologists manually annotated the segment of connected components that represented a correct character in Balinese script from the word-level binarized images that were manually annotated  [11, 17, 20]  using Aletheia (http://www. primaresearch. org/tools/Aletheia)  [62, 63]  (Figure  14 ). The Sundanese character dataset was annotated manually  [22]  (Figure  15 ). For the Khmer character dataset, a tool has been developed to annotate characters/glyphs on the document page. The polygon boundary of each character is traced manually by dotting out its vertex one by one. A label is given to each annotated character after its boundary has been constructed  [21]  (Figure  16 ).