title: Domain-specific language models for multi-label classification of medical text

authors: Yogarajan, Vithya

subjects: Machine learning

abstract: Recent advancements in machine learning-based medical text multi-label classifications can be used to enhance the understanding of the human body and aid the need for patient care. This research considers predicting medical codes from electronic health records (EHRs) as multi-label problems, where the number of labels ranged from 15 to 923. It is motivated by the advancements in domain-specific language models to better understand and represent electronic health records and improve the predictive accuracy of medical codes.    

The thesis presents an extensive empirical study of language models for binary and multi-label medical text classifications. Domain-specific multi-sourced fastText pre-trained embeddings are introduced. Experimental results show considerable improvements to predictive accuracy when such embeddings are used to represent medical text. It is shown that using domain-specific transformer models outperforms results for multi-label problems with fixed sequence length. If processing time is not an issue for a long medical text, then TransformerXL will be the best model to use. Experimental results show significant improvements over other models, including state-of-the-art results, when TransformerXL is used for down-streaming tasks such as predicting medical codes. 

The thesis considers concatenated language models to handle long medical documents and text data from multiple sources of EHRs. Experimental results show improvements in overall micro and macro F1 scores, and such improvements are achieved with fewer resources. In addition, it is shown that concatenated domain-specific transformers improve F1 scores of infrequent labels across several multi-label problems, especially with long-tail labels.

text: Domain-specific language models for multi-label classification of medical text Recent advancements in machine learning-based medical text multi-label classifications can be used to enhance the understanding of the human body and aid the need for patient care. This research considers predicting medical codes from electronic health records (EHRs) as multi-label problems, where the number of labels ranged from 15 to 923. It is motivated by the advancements in domain-specific language models to better understand and represent electronic health records and improve the predictive accuracy of medical codes.    

The thesis presents an extensive empirical study of language models for binary and multi-label medical text classifications. Domain-specific multi-sourced fastText pre-trained embeddings are introduced. Experimental results show considerable improvements to predictive accuracy when such embeddings are used to represent medical text. It is shown that using domain-specific transformer models outperforms results for multi-label problems with fixed sequence length. If processing time is not an issue for a long medical text, then TransformerXL will be the best model to use. Experimental results show significant improvements over other models, including state-of-the-art results, when TransformerXL is used for down-streaming tasks such as predicting medical codes. 

The thesis considers concatenated language models to handle long medical documents and text data from multiple sources of EHRs. Experimental results show improvements in overall micro and macro F1 scores, and such improvements are achieved with fewer resources. In addition, it is shown that concatenated domain-specific transformers improve F1 scores of infrequent labels across several multi-label problems, especially with long-tail labels.

year: 2022

title: Alum dosing at Waitangi Soda Springs – bioavailability of aluminium in 2016

authors: Ling, Nicholas

abstract: Continuous alum dosing of the Waitangi Soda Springs at Lake Rotoehu commenced in 2010 to reduce inflows of dissolved reactive phosphorus to Lake Rotoehu. Analyses of bioaccumulation in the tissues of kōura and goldfish from Lake Rotoehu were undertaken on animals collected in November 2016 for comparison with data from previous years and control locations to determine the bioavailability and bioaccumulation of aluminium to satisfy annual resource consent conditions 11.7 and 11.8 of resource consent 65966. Lake Rotoehu goldfish showed slightly elevated aluminium concentrations in the gills indicating enhanced exposure to aluminium as a result of alum discharge from the Waitangi Soda Springs but no evidence of significant bioaccumulation in internal tissues.

text: Alum dosing at Waitangi Soda Springs – bioavailability of aluminium in 2016 Continuous alum dosing of the Waitangi Soda Springs at Lake Rotoehu commenced in 2010 to reduce inflows of dissolved reactive phosphorus to Lake Rotoehu. Analyses of bioaccumulation in the tissues of kōura and goldfish from Lake Rotoehu were undertaken on animals collected in November 2016 for comparison with data from previous years and control locations to determine the bioavailability and bioaccumulation of aluminium to satisfy annual resource consent conditions 11.7 and 11.8 of resource consent 65966. Lake Rotoehu goldfish showed slightly elevated aluminium concentrations in the gills indicating enhanced exposure to aluminium as a result of alum discharge from the Waitangi Soda Springs but no evidence of significant bioaccumulation in internal tissues.

year: 2017

title: Tephras and New Zealand archaeology

authors: Lowe, David J.

subjects: tephrochronology

abstract: Establishing an accurate date for earliest Polynesian settlement in New Zealand is essential for understanding patterns of settlement and associated environmental impacts, and the processes and rates of cultural change in Eastern Polynesia. Tephra deposits from five volcanic centres, together with exotic sea-rafted pumice, provide isochronous constraints on the timing of earliest settlement and human impacts in northern New Zealand. A local basaltic tephra from Rangitoto Island (Auckland) and locally distributed andesitic tephras from Egmont volcano directly date human occupation to c. AD 1400–1450. Distal andesitic tephras (Tufa Trig Formation) from Mt Ruapehu, Tongariro volcanic centre, help constrain the timing of earliest anthropogenic deforestation signals in Hawke's Bay. Sea-rafted Loisels Pumice(s), although of uncertain stratigraphic reliability in places, overlies cultural remains that can be no younger than c. AD 1350 along the east coast, North Island. The regionally extensive rhyolitic Kaharoa Tephra, which erupted from Okataina volcano between c. AD  1300–1390, is the critical “settlement layer” datum for dating prehistory in the North Island: no human artefacts are recorded beneath it and the earliest inferred environmental impacts by humans are dated to c. AD  1280, just prior to its deposition. This maximum date matches the earliest radiocarbon dates derived for both settlement and human impacts from archaeological and natural sites (c. AD 1250), and implies that the onset of deforestation was essentially contemporaneous with initial settlement.
The widespread rhyolitic Taupo Tephra, which erupted from Taupo volcano c. AD 200, provides an isochronous benchmark well before earliest settlement. The tephra may coincide approximately with a putative earlier transient contact in New Zealand based on Pacific rat-bone (Rattus exulans) dates. More precise calendrical dates on the tephras—via dendrochronology or ice-core records or other dating methods—would help refine assessment of the timing of earliest settlement, while extending the distributional range of critical tephra layers, through application of crypto-tephra analysis, could lead to a greater understanding of settlement patterns.

text: Tephras and New Zealand archaeology Establishing an accurate date for earliest Polynesian settlement in New Zealand is essential for understanding patterns of settlement and associated environmental impacts, and the processes and rates of cultural change in Eastern Polynesia. Tephra deposits from five volcanic centres, together with exotic sea-rafted pumice, provide isochronous constraints on the timing of earliest settlement and human impacts in northern New Zealand. A local basaltic tephra from Rangitoto Island (Auckland) and locally distributed andesitic tephras from Egmont volcano directly date human occupation to c. AD 1400–1450. Distal andesitic tephras (Tufa Trig Formation) from Mt Ruapehu, Tongariro volcanic centre, help constrain the timing of earliest anthropogenic deforestation signals in Hawke's Bay. Sea-rafted Loisels Pumice(s), although of uncertain stratigraphic reliability in places, overlies cultural remains that can be no younger than c. AD 1350 along the east coast, North Island. The regionally extensive rhyolitic Kaharoa Tephra, which erupted from Okataina volcano between c. AD  1300–1390, is the critical “settlement layer” datum for dating prehistory in the North Island: no human artefacts are recorded beneath it and the earliest inferred environmental impacts by humans are dated to c. AD  1280, just prior to its deposition. This maximum date matches the earliest radiocarbon dates derived for both settlement and human impacts from archaeological and natural sites (c. AD 1250), and implies that the onset of deforestation was essentially contemporaneous with initial settlement.
The widespread rhyolitic Taupo Tephra, which erupted from Taupo volcano c. AD 200, provides an isochronous benchmark well before earliest settlement. The tephra may coincide approximately with a putative earlier transient contact in New Zealand based on Pacific rat-bone (Rattus exulans) dates. More precise calendrical dates on the tephras—via dendrochronology or ice-core records or other dating methods—would help refine assessment of the timing of earliest settlement, while extending the distributional range of critical tephra layers, through application of crypto-tephra analysis, could lead to a greater understanding of settlement patterns.

year: 2000