title: Studying and exploiting the relationship between model accuracy and explanation quality

authors: Jia, Yunzhe

subjects: computer science

abstract: Many explanation methods have been proposed to reveal insights about the internal procedures of black-box models like deep neural networks. Although these methods are able to generate explanations for individual predictions, little research has been conducted to investigate the relationship of model accuracy and explanation quality, or how to use explanations to improve model performance. In this paper, we evaluate explanations using a metric based on area under the ROC curve (AUC), treating expert-provided image annotations as ground-truth explanations, and quantify the correlation between model accuracy and explanation quality when performing image classifications with deep neural networks. The experiments are conducted using two image datasets: the CUB-200-2011 dataset and a Kahikatea dataset that we publish with this paper. For each dataset, we compare and evaluate seven different neural networks with four different explainers in terms of both accuracy and explanation quality. We also investigate how explanation quality evolves as loss metrics change through the training iterations of each model. The experiments suggest a strong correlation between model accuracy and explanation quality. Based on this observation, we demonstrate how explanations can be exploited to benefit the model selection process—even if simply maximising accuracy on test data is the primary goal.

text: Studying and exploiting the relationship between model accuracy and explanation quality Many explanation methods have been proposed to reveal insights about the internal procedures of black-box models like deep neural networks. Although these methods are able to generate explanations for individual predictions, little research has been conducted to investigate the relationship of model accuracy and explanation quality, or how to use explanations to improve model performance. In this paper, we evaluate explanations using a metric based on area under the ROC curve (AUC), treating expert-provided image annotations as ground-truth explanations, and quantify the correlation between model accuracy and explanation quality when performing image classifications with deep neural networks. The experiments are conducted using two image datasets: the CUB-200-2011 dataset and a Kahikatea dataset that we publish with this paper. For each dataset, we compare and evaluate seven different neural networks with four different explainers in terms of both accuracy and explanation quality. We also investigate how explanation quality evolves as loss metrics change through the training iterations of each model. The experiments suggest a strong correlation between model accuracy and explanation quality. Based on this observation, we demonstrate how explanations can be exploited to benefit the model selection process—even if simply maximising accuracy on test data is the primary goal.

year: 2021

title: A small-N study of body weight during free-feeding in the brushtail possum (Trichosurus vulpecula)

authors: Cameron, Kristie Elizabeth

abstract: Animals in captivity can develop physiological characteristics such as obesity that could negatively affect their health. In the captive brushtail possum (Trichosurus vulpecula), it is unknown what effect long-term free-feeding has on body weight. In this study, which took place over 12 months, three possums were fed according to a free-feeding regime. Food intake was measured daily, and the possums were weighed every four days. The possums all gained weight but showed bouts of stable body weights across the year. Stability was evaluated using mathematical models. It was concluded that long-term free-feeding is not recommended for animals-in-waiting for use in operant experiments to mitigate weight gain due to overeating.

text: A small-N study of body weight during free-feeding in the brushtail possum (Trichosurus vulpecula) Animals in captivity can develop physiological characteristics such as obesity that could negatively affect their health. In the captive brushtail possum (Trichosurus vulpecula), it is unknown what effect long-term free-feeding has on body weight. In this study, which took place over 12 months, three possums were fed according to a free-feeding regime. Food intake was measured daily, and the possums were weighed every four days. The possums all gained weight but showed bouts of stable body weights across the year. Stability was evaluated using mathematical models. It was concluded that long-term free-feeding is not recommended for animals-in-waiting for use in operant experiments to mitigate weight gain due to overeating.

year: 2022

title: Viewing the world through a wider lens: Māori and council planning documents

authors: Kennedy, Nathan

subjects: New Zealand

abstract: My effort to convey an iwi perspective on environmental resource management - what we call kaitiakitanga – should highlight for new planners about to enter the profession that the environmental perspectives of hapū  and iwi (which are provided for in the RMA), are currently not well covered in either mainstream local government planning or education.

text: Viewing the world through a wider lens: Māori and council planning documents My effort to convey an iwi perspective on environmental resource management - what we call kaitiakitanga – should highlight for new planners about to enter the profession that the environmental perspectives of hapū  and iwi (which are provided for in the RMA), are currently not well covered in either mainstream local government planning or education.

year: 2008-07-30