Automatic Extraction of Structured Mineral Drillhole Results from Unstructured Mining Company Reports (2022)

Abstract: Aggregate mining exploration results can help companies and governments to optimise and police mining permits and operations, a necessity for transition to a renewable energy future, however, these results are buried in unstructured text. We present a novel dataset from 23 Australian mining company reports, framing the extraction of structured drillhole information as a sequence labelling task. Our two benchmark models based on Bi-LSTM-CRF and BERT, show their effectiveness in this task with a F1 score of 77% and 87%, respectively. Our dataset and benchmarks are accessible online.

## introduction mineral exploration involves drilling for core samples to assess their mineral composition. these assays are published in annual reports and other public announcements such as press releases. often these results are presented in a semi-consistent non-tabular form. there is an industry demand for up-to-date mineral exploration results given that aggregate mineral composition information across a region or country can guide and optimise mineral exploration, however, current solutions involve manual collection of data directly from public company resources, which is expensive, time-intensive, and out-of-date @xcite . this has become more important as the transition from fossil fuels to renewable energy has accelerated the demand for minerals such as lithium, nickel and rare earth metals. an assay report contains "drillhole sentences" which are phrases containing a unique drillhole code, depth, material, type and material percentage. see the example in figure 1 from a mining company press release. the results are buried in long reports that contain images and natural language text with varying nomenclature, 1 link to the dataset format and placement in the report across companies, geologists and mineral sectors, making their automatic extraction by regular expressions very challenging. the format of the drillhole sentences figure 1 : excerpt from pilbara minerals asx announcement pilbara @xcite presents an opportunity to apply natural language processing techniques to automatically extract drillhole data. in this paper, we assess the performance of neural network models in extracting structured information about drillhole mineral exploration results. to the best of our knowledge, this work is the first to focus on extracting drillhole results from unstructured text, acknowledging prior work on extracting other geological information such as rick types @xcite . our contributions are: a) developing a novel dataset for structured drillhole result extraction from 23 public australian stock exchange (asx) listed mining companies involved in mineral exploration.; b) formulating the extraction task as sequence labelling and presenting two benchmarks: a bidirectional lstm network with a conditional random field layer (bi-lstm-crf) @xcite and bert @xcite , showing that both perform fairly well; and c) performing error analysis and identifying major error types to guide future work. ## related work in this section, we provide insight into previous work performed on extracting geological data from reports and an overview of neural network models for sequence labelling tasks. ## data the dataset consists of 50 reports from 23 asxlisted mining exploration companies. the selection criteria for reports, extraction and segmentation of text from the pdf files and the annotation process are reported in this section. additionally, to test the generalisation ability of the models, the dataset is split into training, dev and test sets based on a) random; b) material; and c) company, to find out if the benchmark models will be able to generalise across materials and companies. ## method to measure the generalisation ability of a sequence labeling task trained on our dataset we used two benchmarks, both evaluated by precision, recall, and f1 using seqeval library @xcite . bi-lstm-crf: we use the bi-lstm-crf model proposed in @xcite for the sequence labeling task of identifying drillhole result segments. this model uses both word and character embeddings which is suitable for our task which involves chemical formulas and numerical tokens that might only be captured through character information. the character and word embeddings are concatenated and fed to a bi-lstm to capture sequential and contextual information. the resulting final hidden states of the two directions are concatenated and fed into a conditional random field layer that models the conditional probability of the tags. bert: we use bert @xcite to find out the effect of pre-training on massive amounts of text on the performance of our task given the relatively small training set and the ability of the pretrained transformers to transfer knowledge across tasks in low-resource settings. due to the computational demands, we only experiment with the base (uncased) version of bert which is lighter compared to bert-large in terms of the model size. given that bert does not take into account the characters, it is interesting to find out if it can outperform bi-lstm-crf which uses character information. ## results evaluation results are shown in upon further inspection of tag-specific results (shown in appendix), the recall of bi-lstm-crf is 38% which is substantially lower than that of bert with a recall of 66%. the identification of drillhole is an essential component of extracting the structured drillhole results from reports as it can uniquely identify a drillhole across several reports. ## conclusion we present our work in creating a novel dataset for extracting structured drillhole results from unstructured mining exploration reports. we formulate this task as sequence labeling and show that while both our two benchmarks bi-lstm-crf and bert perform well with an f1 score of 77% and 87%, respectively, bert substantially outperforms bi-lstm-crf and is more robust to variation in language and format. we performed error analysis on the bi-lstm-crf predictions and identified context variation and unseen tokens in training data to be the most frequent error types. our error analysis indicates improvement pathways for the bi-lstm-crf model which is more efficient for use in most common computing settings. angela riganti, terence r farrell, margaret j ellis, felicia irimies, colin d strickland, sarah k martin, and darren j wallace. 2015. 125 years of legacy data at the geological survey of western australia: capture and delivery. georesj, 6:175-194. leslie n. smith. 2015. no more pesky learning rate guessing games. corr, abs/1506.01186. alexandre tessarollo and alexandre rademaker. 2020. inclusion of lithological terms (rocks and minerals) in the open wordnet for english. in proceedings of the lrec 2020 workshop on multimodal wordnets (mmw2020), pages 33-38, marseille, france. the european language resources association (elra).