bluuebunny/crossref_metadata_2025_split huggingface dataset:
Issues concerning the Delivery of Surveys: An Institutional View
Authors: Nair Chenicheri S , Adams Phillip , Mertova Patricie 
Published: January 2018

This article presents a case study that investigates several issues concerning the delivery of evaluation surveys in a large research-intensive university in Australia. The article touches on the issue of ‘survey fatigue’, which has been highlighted in relation to the conduct of an increased number and frequency of surveys in tertiary institutions. Despite these warnings, the university described in this case study has recently increased the number and frequency of evaluation surveys, while its survey response rates have been gradually decreasing since the late 1990s. The case study further outlines two strategies developed in the institution with the aim of engaging students better, subsequently increasing student survey response rates and ultimately improving educational provision in the institution. These strategies were supported further by adopting a more user-friendly platform for submission of online evaluation surveys and by providing students with alternative means of survey submission. The ease and coherence of the evaluation process in the institution was assisted further by the introduction of a new Survey Management System (SMS). All these strategies and initiatives have proved successful in the institution. Therefore, the article aims to provide some suggestions for other tertiary institutions dealing with similar issues. At the same time, it highlights the fact that individual institutional circumstances ought to be considered and such strategies need to be planned carefully according to the needs of the individual institution.
Maximising Evaluation Influence in an International Development Donor Agency
Authors: Tennant Stacey 
Published: January 2018

This article examines how current theories of evaluation use and evaluation influence can assist an international development donor agency, namely the Australian Agency for International Development (AusAID), maximise the outcomes and track the consequences of evaluations of its country programs. It considers those theories most relevant to AusAID's evaluation interests and contexts, that is: instrumental use; conceptual (or enlightenment) use; process use; and evaluation influence. It finds that evaluation influence has much to offer in terms of linking evaluation outcomes to the achievement of program outcomes. By considering the mechanisms that can mediate evaluation influence, the framework underpinning evaluation influence theory builds on many of the central tenets identified in evaluation use theories and encourages the consideration of an evaluation as an intervention in itself. The principal benefit of doing so is to understand when, and why, some use of evaluation occurs and when and why it does not. Evaluation influence theory holds the potential for AusAID to maximise its evaluation efforts and strengthen its country programs. The systematic consideration of which mechanisms can influence the intermediate and long-term outcomes that lead to social betterment is the key to realising this potential. Empirical research into these mechanisms could both assist and be assisted by AusAID embracing an approach of valuing evaluations as an intervention in themselves.
Maximising Participation in International Community-level Project Evaluation: A Strength-based Approach
Authors: Donnelly John 
Published: January 2018

The evaluation of international aid-funded development projects implemented primarily by NGOs usually relies heavily on input from community members. However, the evaluation design and conduct generally does not provide the time, or the opportunity, for the participation of all community members. Informants are often selected according to a statistical sampling method or they are the ‘leaders’ of the community or groups within the community. In addition, methods used for data collection, such as questionnaires and surveys, can be intimidating to people who are illiterate or who have limited literacy skills, thus alienating large numbers of potential community stakeholders further. The community can be alienated from the evaluation process even more when the data collected from the community, or its representatives, at each evaluation site, is taken away for collation and analysis and is rarely seen again by those who provided it. This article demonstrates that the situation outlined above need not be the case and that evaluations can be both inclusive and informing for all stakeholders, including donors/commissioners, implementing agencies and the community. The article also contends that this approach to project evaluation, as well as being inclusive and transparent, is also empowering to communities and enhances the sustainability of project outcomes.
Negotiating and Planning an Evaluation: An Example from a Youth Services Work for the Dole Program Activity
Authors: Scott Theresa L , Darlington Yvonne 
Published: January 2018

This article describes the negotiating and planning of an evaluation proposal for a client, a job network agency situated in Northern New South Wales. The evaluand is a training-based Work for the Dole program activity targeted at unemployed youth and overseen by a not-for-profit community organisation. The article begins by examining the social, political and historical context in which the Work for the Dole program is embedded, and considers the potential impact of context on the evaluation process and utilisation of the results. The evaluation proposal itself covers negotiation, ethical issues, key evaluation questions and methodology. The last section of the article provides a critical review of the proposed plan and discusses potential process issues. Throughout, it is emphasised that developing an understanding of the competing values of the many stakeholders involved is fundamental to successful negotiation and planning with all those involved in the evaluation process.
Goal Attainment Scaling in Early Childhood Intervention
Authors: Sharp Colin , Read Phillip 
Published: January 2018

One of the problems about evaluations of outcomes of therapeutic services is the need for quantitative and/or qualitative measurement of attainment of intended goals that is sensitive to small changes over short periods. Goal Attainment Scaling (GAS) was developed by Dr Tom Kiresuk and colleagues (in the United States) in the 1960s to address this issue. While there are many variations and applications of GAS in education and community services, there is little available literature on its application in the field of therapeutic interventions with visually impaired young children. The project described in this article entailed research on the efficacy of GAS in an evaluation of family-centred early intervention programs in an organisation focusing on young children with a vision impairment. It emerged that the preferred version of GAS was different in early intervention from the main applications of this instrument. In addition to the initial focus on GAS, was the need to develop a complementary global assessment measure. A repeated measures comparison of three intervention groups of visually impaired children showed a statistically significant effect, indicating that GAS was sufficiently clinically sensitive to detect small changes in individuals and small groups over several weeks of intervention and observation. Meanwhile, the qualitative data revealed a positive appreciation by all stakeholders of the opportunity for parents to become involved in goal setting and evaluation of their child's program.
Learning from Evaluations of School–Family Strengthening Programs: Lessons for All
Authors: Guenther John , Galbraith Mal 
Published: January 2018

For more than 10 years, a program called Families And Schools Together (FAST) has been run in schools across the Northern Territory. These programs have always had an evaluation component built in. However, over time, the evaluations have changed. Initially, they were summative, built around a quantitative psychometric tool (with a positivist research paradigm). The intent of the summative evaluation was to demonstrate the effectiveness of the program. However, as the program was rolled out in remote contexts, the need for adaptation was recognised. Changes were made, but it was soon recognised that other outcomes, not captured or explored in the methodology, were emerging. After six years of working together, the evaluator and the program manager felt that it was time to explore effectiveness in a different way with a more qualitative evaluation process (based on naturalistic and pragmatic paradigms). This article explores the six-year learning journey through evaluation that the evaluator and program manager have undertaken. It describes tensions between the need for reliable and generalisable objective quantitative data and the need for authentic and credible data based on participant experience.
Cost-benefit analysis explained
Authors: Liddle John , Wright Matt , Koop Bronte 
Published: January 2018

In every evaluation context, the question of worth underpins the motivation or role of the evaluation, even if it is not explicitly stated. Often, this question cannot be answered fully, simply because people or parties may make varying subjective judgments about the value and likelihood of different outcomes—it is at this juncture that cost–benefit analysis (CBA) can be of use. For evaluations where most significant inputs and outcomes can be valued in monetary terms, CBA provides a systematic methodology for reaching conclusions on ‘worth’. CBA is best used in conjunction with traditional evaluation approaches rather than as a replacement approach. The use of CBA can help an evaluator to be more confident in stating the evaluation findings, particularly in social policy settings where the use and importance of CBA is growing. This article aims to help evaluators learn more about CBA by demonstrating its use to analyse an intervention designed to help young people disengaged from learning.
Evaluation of Pacific cancer care navigation patient facilitation and support service
Authors: Tanuvasa Ausaga Faasalele , Neale Jenny 
Published: January 2018

From 2010, a not-for-profit community services’ provider has delivered a Pacific cancer navigation patient facilitation and support service (hereafter ‘the Service’) for Pacific peoples residing in an area of West Auckland, New Zealand. The Service was evaluated in 2013. As part of the evaluation, both group and individual interviews were held with sixteen cancer patients, eight family members and seven key informants; in conjunction, observations and informal discussions were held with two navigators. Service personnel were closely involved from the inception of the evaluation, and this practice will be discussed as part of building an evaluation culture in the organisation. There was general agreement that the implementation of the facilitation and support Service had proceeded well, this was characterised by effective engagement with health professionals; the establishment of a support group; the provision of transport; assistance in dealings with Work and Income New Zealand (WINZ); and the provision of timely and accurate information to assist patients and their families. From this common consensus, suggestions were made around increasing and ensuring ongoing funding to sustain the supporting roles of navigators and the services offered. Other invaluable features of the Service included the ability of navigators to translate from English to Samoan, and from medical to lay language, as well as the provision of the Service beyond central metro areas.
Project evaluation in higher education: A study of contextual issues
Authors: Huber Elaine , Harvey Marina 
Published: January 2018

This paper reports on the second phase of a wider study on evaluation of locally funded learning and teaching projects in higher education. A review of the extant literature in this area shows that there is little evidence of the extent to which the alignment between theory and practice, or praxis, of such evaluation is achieved. The first phase established that the project leader's perception and conceptualisation of evaluation can inhibit achieving praxis. Consequently, the aim of the second phase was to investigate in detail the contextual factors impacting on project evaluation. Three case examples of internally funded learning and teaching projects provide the data for analysis using an explanatory approach. The projects share a focus on curriculum design in arts and humanities disciplines at an Australian university. A case study approach enabled the in-depth investigation of this contemporary phenomenon, within its real-life context. Data collection included interviews with each project manager and with the project lead; this was triangulated with the data of grant applications, final reports and meeting minutes. A thematic approach to coding the research data was adopted. One key finding indicates a disjunct of how evaluation is conceptualised between the project lead and the project manager. In addition, factors that influence the praxis of evaluation were also identified including timeframe; previous experience of leading a project; and the requirement (or lack thereof) from the projects’ grant funding body. Informed by the research findings, four strategies to enhance the adoption of systematic evaluation in small learning and teaching projects are presented; these strategies offer the potential to reach across disciplinary and sector boundaries in their application.
Bridging the Gap Both-Ways: Enhancing Evaluation Quality and Utilisation in a Study of Remote Community Safety and Wellbeing with Indigenous Australians
Authors: Grey Kim , Putt Judy , Baxter Nathalie , Sutton Sue 
Published: January 2018

This paper demonstrates how the voices of people affected by a policy can contribute to evaluation quality and utilisation. In this participatory mixed-methods approach, Indigenous Australians in remote parts of the Northern Territory of Australia were involved in evaluation of an intervention that significantly affected their daily lives. The evaluation was inspired by the ideas and values demonstrated in a both-ways learning model. This mixed-methods approach combines a people-centered participatory methodology with a more standardised policy-focused application of survey methods to contribute to policy development and local planning. The aim of the study was to hear from local people in communities and encourage them to share their views about changes in community safety. It provided a voice for Indigenous Australians affected by the Northern Territory Emergency Response (NTER), which sought to protect children in 73 Indigenous communities across the Northern Territory. The aim of the NTER was to create safer communities and to improve health, nutrition, educational outcomes, service delivery and engagement with Indigenous people. Regardless of views about the NTER, a clear gap in the evidence base for evaluation was to understand if people felt safer after the intervention. Increased police, safe houses, night patrols, health services, family support services and management of income support payments were introduced, but what did Indigenous people living in remote communities think? What was their experience and what will happen to the information they may choose to share? Adopting a mixed-methods approach that included a large-scale survey and qualitative research conducted with local Indigenous residents of remote communities strengthened evaluative findings, particularly for policy makers and community residents. A participatory research cycle was used to engage local people and return results in an ethical feedback process that aimed to build capacity on the ground and link into future planning. We sought to promote the role of evaluation in remote Northern Territory communities as a worthwhile way for local people to share their voice, stories and experiences with the policy makers whose decisions can so significantly affect their lives.
Challenges in Evaluating Aboriginal Healing Programs: Definitions, Diversity and Data
Authors: Blignault Ilse , Williams Megan 
Published: January 2018

Indigenous people around the world have long healing traditions. Contemporary Aboriginal and Torres Strait Islander healing projects are designed to empower individuals, families and communities; strengthen connections to culture; and reduce the damaging effects of colonisation and government policies such as the forcible removal of children (the Stolen Generations). Evidence on the conditions necessary for healing to occur, and how healing works for different people and in different contexts, is limited. Evaluations that will help identify good practice and document the full range of outcomes are sorely needed. This paper is based largely on experiences and learnings from Stolen Generations projects around Australia funded by the Aboriginal and Torres Strait Islander Healing Foundation, and the reflections of experienced scholar-practitioners. It argues that evaluations that are responsive to, and ultimately owned and led by, Aboriginal and Torres Strait Islander communities need to be designed and implemented differently to mainstream evaluations. Timeframes, methods, relationships between evaluators and stakeholders, and the identification and measurement of outcomes all need to be carefully considered. Challenges include definitions of healing, diversity of landscapes and programs, and data collection. Qualitative methods that preference and support Indigenous cultural frameworks and ways of creating and sharing knowledge work well. In addition to ensuring culturally sensitive methodologies and tools, working ethically and effectively in the Indigenous healing space means emphasising and enabling safety for participants, workers and organisations.
Evaluating Women's Empowerment: Experimenting with a Creative Participatory Self-evaluation Methodology in Papua New Guinea
Authors: Waffi Joy Marie 
Published: January 2018

This article introduces a participatory evaluation methodology that can be used with very low literacy groups of women to capture change experienced in their voice, participation, and decision-making abilities at the household and community levels. Central to this methodology is the process utilised to enable a personal determination of the level of change that has taken place according to individual baselines and circumstances, and using this information to present a more accurate picture of how much change has occurred for some women versus others and use percentage break-ups to show the different change-level-groups of women in a community. Whereas one can easily make the mistake of generalising change levels for all the women based on the responses of a few outspoken ones, this evaluation methodology captures the richness and diversity that exists even among a small group of women. Utilising such an evaluation approach makes one recognise the complexities that come with measuring change in women's voice, participation and decision-making in very low literacy settings. Developed in haste and used for an internal CARE International end-of-project evaluation in early 2014, the author shares her original experience of utilising this new self-developed participatory evaluation methodology. The article is sequenced so that the reader is presented with the circumstances out of which this evaluation methodology was developed, the benefits discovered from utilising it, shortfalls of the original methodology, and improvements that will be made to further develop and strengthen the methodology.
Factors Associated With Adherence to Nasoalveolar Molding (NAM) by Caregivers of Infants Born With Cleft Lip and Palate
Authors: Esmonde Nick O. , Garfinkle Judah S. , Chen Yiyi , Lambert William E. , Kuang Anna A. 
Published: January 2018

Objective: Identify factors associated with adherence to nasoalveolar molding (NAM) therapy. Design: Retrospective case-control study. Setting: Tertiary referral center. Patients, Participants: Infants with cleft lip, with or without cleft palate, referred for NAM. One hundred thirty-five patients met criteria. Main Outcome Measure(s): Adherence to NAM therapy, defined as continuous use of the appliance and attendance of NAM adjustment visits. Results: Female sex (OR = 2.85, 95% CI 1.21-6.74), bilateral cleft (OR = 2.88, 95% CI 1.29-6.46), and travel distance (OR = 1.01, 95% CI 1.00-1.01) were independent predictors of nonadherence. Bilateral clefts (OR = 8.35, 95% CI 2.72-25.64) and public-payer insurance (OR = 3.67, 95% CI 1.13-11.91) for male patients were significantly associated with nonadherence, in our sex-stratified multivariate model. The majority of the families (58%) had public health insurance. Males comprised 77.0% of the cohort. Conclusions: NAM treatment adherence is impaired by bilateral clefts, female sex, increased travel distance, and public insurance. Further studies are warranted to investigate how these factors affect adherence, and to develop interventions to improve adherence in families at risk due to economic or psychosocial barriers.
Female and Male Differences in Academic Achievement in Individuals With Cleft
Authors: Persson Martin , Becker Magnus , Conrad Amy L. , Svensson Henry 
Published: January 2018

Objective: The focus of this study was to determine if there is any significant difference in academic achievement for girls and boys with a cleft compared to the general population of Swedish students at graduation from compulsory school. Design: A retrospective population-based study using data obtained from the Swedish Medical Birth Register that was linked to the Swedish School–Grade Register. Participants: Two hundred seventy girls and 241 boys with cleft palate (CP), 222 girls and 429 boys with cleft lip (CL), and 299 girls and 531 boys with cleft lip and palate (CLP) were compared with the compulsory school population comprising 609,397 girls and 640,007 boys. Main Outcome Measures: (1) Odds of receiving the lowest grade and reduced odds in receiving high grades in Mathematics, English, and Swedish. (2) grade point average (GPA). Results: In all 3 subject grades, for boys with cleft there was no difference when compared to the male population. Girls with cleft were similar to their peers with a few exceptions. Girls with CLP had lower Math grades, and girls with CP had lower Math, English, and Swedish grades. Girls with CP and CLP achieved a significantly lower GPA in comparison to the female population and boys with CP and CL achieved lower GPAs in comparison to the male population. Conclusions: This study indicates that educational outcomes for girls with cleft are more negatively affected than for boys with cleft.