title: Hitting the target: Stopping active learning at the cost-based optimum

authors: Pullar-Strecker, Zac

subjects: computer science

abstract: Active learning allows machine learning models to be trained using fewer labels while retaining similar performance to traditional supervised learning. An active learner selects the most informative data points, requests their labels, and retrains itself. While this approach is promising, it raises the question of how to determine when the model is ‘good enough’ without the additional labels required for traditional evaluation. Previously, different stopping criteria have been proposed aiming to identify the optimal stopping point. Yet, optimality can only be expressed as a domain-dependent trade-off between accuracy and the number of labels, and no criterion is superior in all applications. As a further complication, a comparison of criteria for a particular real-world application would require practitioners to collect additional labelled data they are aiming to avoid by using active learning in the first place. This work enables practitioners to employ active learning by providing actionable recommendations for which stopping criteria are best for a given real-world scenario. We contribute the first large-scale comparison of stopping criteria for pool-based active learning, using a cost measure to quantify the accuracy/label trade-off, public implementations of all stopping criteria we evaluate, and an open-source framework for evaluating stopping criteria. Our research enables practitioners to substantially reduce labelling costs by utilizing the stopping criterion which best suits their domain.

text: Hitting the target: Stopping active learning at the cost-based optimum Active learning allows machine learning models to be trained using fewer labels while retaining similar performance to traditional supervised learning. An active learner selects the most informative data points, requests their labels, and retrains itself. While this approach is promising, it raises the question of how to determine when the model is ‘good enough’ without the additional labels required for traditional evaluation. Previously, different stopping criteria have been proposed aiming to identify the optimal stopping point. Yet, optimality can only be expressed as a domain-dependent trade-off between accuracy and the number of labels, and no criterion is superior in all applications. As a further complication, a comparison of criteria for a particular real-world application would require practitioners to collect additional labelled data they are aiming to avoid by using active learning in the first place. This work enables practitioners to employ active learning by providing actionable recommendations for which stopping criteria are best for a given real-world scenario. We contribute the first large-scale comparison of stopping criteria for pool-based active learning, using a cost measure to quantify the accuracy/label trade-off, public implementations of all stopping criteria we evaluate, and an open-source framework for evaluating stopping criteria. Our research enables practitioners to substantially reduce labelling costs by utilizing the stopping criterion which best suits their domain.

year: 2022

title: Genre, academic writing and e-learning: An integrated tertiary level Taiwan-based study

authors: Lin, Hsiu-Chen (Antonia)

subjects: genre

abstract: The research reported here has two main focus points: online learning and the teaching of academic writing to learners of English as an additional language. At its core is a study involving an intensive genre-centered writing course conducted in a tertiary educational institution in Taiwan and delivered in three modes - face-to-face, fully online and blended. That study, preceded by a pilot study conducted in New Zealand, involved a writing course that focused on cognitive genres (e.g. argument) that have been identified as being fundamental to academic writing. It included model texts (constructed in segments with accompanying discussion of their language and structure) and writing exercises. Analysis of post-course questionnaires and focus group discussions revealed a high level of satisfaction with the course. Analysis of pre-test and post-test writing tasks in terms of a wide range of criteria provided evidence of improvement in the writing of course participants in a range of areas. Although those involved in blended and face-to-face modes were most positive about the advantages of the course, it was not necessarily always the case that they outperformed online group members in terms of improvement in writing. 

Also included are two questionnaire-based surveys of samples of teachers of English in tertiary level educational institutions in Taiwan. The first investigated attitudes and practices in relation to the integration of instructional technology into teaching. Although the vast majority of survey participants believed that it was important to incorporate instructional technology into their teaching, this was not necessarily reflected in their more specific beliefs and practices. Very few reported having spent more than a few hours attending instructional technology-related workshops, more than half indicated that very little or none of the interaction in their language classes was computer-mediated, only approximately one third reported having used a learning platform in the six weeks prior to the survey, and over one third reported that they had never used a learning platform. The second questionnaire-based survey investigated attitudes and practices in relation to the teaching and assessment of writing. Although survey participants were familiar with process-centered approaches to the teaching of writing, they appeared to be much less familiar with genre-centered approaches. Using model texts as a way of introducing, demonstrating and explaining language in use seemed to be the exception rather than the rule. Additionally, although they reported spending a considerable amount of time grading and commenting on their students' writing, most of them indicated that they did not design grading criteria that related specifically to course content, and many of the sample comments on student writing that they provided were of a type that is unlikely to help students to improve their writing. 

Overall, the study provides evidence that a genre-centered academic writing course can be associated with a high level of student satisfaction and can lead to demonstrable improvement in student writing. However, it also demonstrates that teachers of English at tertiary level in Taiwan are generally unfamiliar with this sort of approach and that many of them are not yet ready to provide their students with options in terms of delivery modes.

text: Genre, academic writing and e-learning: An integrated tertiary level Taiwan-based study The research reported here has two main focus points: online learning and the teaching of academic writing to learners of English as an additional language. At its core is a study involving an intensive genre-centered writing course conducted in a tertiary educational institution in Taiwan and delivered in three modes - face-to-face, fully online and blended. That study, preceded by a pilot study conducted in New Zealand, involved a writing course that focused on cognitive genres (e.g. argument) that have been identified as being fundamental to academic writing. It included model texts (constructed in segments with accompanying discussion of their language and structure) and writing exercises. Analysis of post-course questionnaires and focus group discussions revealed a high level of satisfaction with the course. Analysis of pre-test and post-test writing tasks in terms of a wide range of criteria provided evidence of improvement in the writing of course participants in a range of areas. Although those involved in blended and face-to-face modes were most positive about the advantages of the course, it was not necessarily always the case that they outperformed online group members in terms of improvement in writing. 

Also included are two questionnaire-based surveys of samples of teachers of English in tertiary level educational institutions in Taiwan. The first investigated attitudes and practices in relation to the integration of instructional technology into teaching. Although the vast majority of survey participants believed that it was important to incorporate instructional technology into their teaching, this was not necessarily reflected in their more specific beliefs and practices. Very few reported having spent more than a few hours attending instructional technology-related workshops, more than half indicated that very little or none of the interaction in their language classes was computer-mediated, only approximately one third reported having used a learning platform in the six weeks prior to the survey, and over one third reported that they had never used a learning platform. The second questionnaire-based survey investigated attitudes and practices in relation to the teaching and assessment of writing. Although survey participants were familiar with process-centered approaches to the teaching of writing, they appeared to be much less familiar with genre-centered approaches. Using model texts as a way of introducing, demonstrating and explaining language in use seemed to be the exception rather than the rule. Additionally, although they reported spending a considerable amount of time grading and commenting on their students' writing, most of them indicated that they did not design grading criteria that related specifically to course content, and many of the sample comments on student writing that they provided were of a type that is unlikely to help students to improve their writing. 

Overall, the study provides evidence that a genre-centered academic writing course can be associated with a high level of student satisfaction and can lead to demonstrable improvement in student writing. However, it also demonstrates that teachers of English at tertiary level in Taiwan are generally unfamiliar with this sort of approach and that many of them are not yet ready to provide their students with options in terms of delivery modes.

year: 2010

title: “Landlords wouldn’t give my application a second look.” Discrimination exacerbates inequalities in access to private rental housing

authors: Terruhn, Jessica

abstract: This report and the key insights are based on findings from a housing survey that was conducted with 800 residents in neighbourhoods in Auckland, Hamilton and Christchurch. 
• Perceptions of unfairness in Aotearoa’s housing sector are widespread. Renters, Māori, younger people and low-income groups as well as residents in neighbourhoods with high levels of housing deprivation are most likely to think that people are treated unfairly when trying to rent or buy a home in Aotearoa. 
• Nearly one in three respondents reported having experienced discrimination when trying to rent or buy a home in Aotearoa. Renters, Māori, younger people and low-income groups as well as residents in neighbourhoods with high levels of housing deprivation are most likely to report experiences of discrimination. 
• Advantage and disadvantage in securing a home are determined by a combination of interlocking factors, including income and employment status, age, family status, and race/ethnicity or skin colour. These patterns suggest widespread experiences of potentially unlawful housing discrimination. 
• People strategically try to avoid and mitigate discrimination. Expectations of being discriminated against and of being advantaged influence where and how home seekers search for housing. This finding signals that experiences of rejection play a role in constraining people’s housing choices. 
• The survey findings suggest that discrimination, as part of tenant selection, contributes to housing precarity and inequalities in access to rental housing. Therefore, this research points to an urgent need to address housing discrimination, especially in the context of high levels of residential mobility among renters and intense competition for rental properties.

text: “Landlords wouldn’t give my application a second look.” Discrimination exacerbates inequalities in access to private rental housing This report and the key insights are based on findings from a housing survey that was conducted with 800 residents in neighbourhoods in Auckland, Hamilton and Christchurch. 
• Perceptions of unfairness in Aotearoa’s housing sector are widespread. Renters, Māori, younger people and low-income groups as well as residents in neighbourhoods with high levels of housing deprivation are most likely to think that people are treated unfairly when trying to rent or buy a home in Aotearoa. 
• Nearly one in three respondents reported having experienced discrimination when trying to rent or buy a home in Aotearoa. Renters, Māori, younger people and low-income groups as well as residents in neighbourhoods with high levels of housing deprivation are most likely to report experiences of discrimination. 
• Advantage and disadvantage in securing a home are determined by a combination of interlocking factors, including income and employment status, age, family status, and race/ethnicity or skin colour. These patterns suggest widespread experiences of potentially unlawful housing discrimination. 
• People strategically try to avoid and mitigate discrimination. Expectations of being discriminated against and of being advantaged influence where and how home seekers search for housing. This finding signals that experiences of rejection play a role in constraining people’s housing choices. 
• The survey findings suggest that discrimination, as part of tenant selection, contributes to housing precarity and inequalities in access to rental housing. Therefore, this research points to an urgent need to address housing discrimination, especially in the context of high levels of residential mobility among renters and intense competition for rental properties.

year: 2024-08