bluuebunny/crossref_metadata_2025 huggingface dataset:
How did conservation agriculture go to scale?
Authors: Ramirez Ricardo , Neudoerffer Cynthia , Salomons Michael
Published: April 2023

Background

The Foodgrains Bank has an established record working in agriculture and food security with resource constrained, marginalized farmers in sub-Saharan Africa. The three outcome areas of the Scaling-Up Conservation Agriculture in East Africa (SUCA) Program were: the adoption of conservation agriculture systems, an enabling institutional environment, and the promotion of enabling policies. These program areas were expected to yield intermediate outcomes that, together, would lead to the ultimate outcome of improved food security and sustainable livelihoods for smallholder farming households in East Africa. This case study reports on the end-line evaluation of the five-year program.

Purpose

To illustrate the overlap between utilization-focused evaluation (UFE) and collaborative approaches to evaluation (CAE). The case study profiles an agricultural intervention, and explores how the evaluation design accommodated the systemic nature of the program.

Setting

Scaling-Up Conservation Agriculture in East Africa (SUCA) was a five-year program of the Canadian Foodgrains Bank implemented from 2015-2020 to expand the size and scope of Foodgrains Bank’s work in conservation agriculture in East Africa. The program supported local partners with a target of 50,000 male and female farmers practicing a minimum of 2 of 3 conservation agriculture principles, and to improve food security and sustainable livelihoods for 18,000 of these farmers’ households across three countries. 

Research design

The Foodgrains Bank was directly involved in the evaluation design through the definition of evaluation uses and key evaluation questions. Eleven implementing partners in East Africa were involved in primary data collection and some initial analysis.

Data collection and analysis

A mixed method approach was used combining quantitative, qualitative, and participatory / visual data collection tools. A robust, intersectional gender lens was applied to the data collection instruments in the form of gender disaggregated data collection and gender-focused questions across most data collection instruments. 

Findings

The collaborative process confirmed a sense of ownership by the primary evaluation users over the evaluation design. The evaluation design combined outcome and learning uses that took advantages of the implementing organizations’ commitment to learning. The findings demonstrated the value of the program and produced a framework illustrating the multi-disciplinary approach underlying its success.
Conditions to Consider in the Use of Randomized Experimental Designs in Evaluation
Authors: Julnes George , Mark Melvin , Shipman Stephanie
Published: April 2023

Debates about the role of randomized experiments in evaluation have been heated at times, which likely has not facilitated and possibly has hindered thoughtful judgments about whether and when to use a randomized experimental design. The challenges of thoughtful deliberation may be especially great for funders and others who influence the choice of an evaluation design but are not immersed in methodological literatures. The current paper offers a non-technical summary of general factors to take into consideration when determining the appropriateness of a randomized design in a forthcoming evaluation or set of evaluations. Four general conditions are described that should be considered with respect to the specific context for the upcoming evaluation(s). These are, first, the expected value of the information that a well-implemented experiment can provide in the specific context; second, the legal and ethical issues that apply in the circumstances at hand; third, the practical constraints (or facilitating factors) that would apply to a randomized experiment in that context; and fourth, the likely value of the experimental findings in relation to and as part of a portfolio of evaluative studies in the specific context.
In This Issue
Authors: Scriven Michael
Published: April 2023

The journal homepage has had over 6,000 hits, and there have been around 2,500 downloads of all or part of the first issue. Our list of 932 people who want to benotified of new issues now includes residents from more than 100 countries. The current issue is a bit longer: it runs over 170pp. but you can download just the parts that interest you.
Network Evaluation as a Complex Learning Process
Authors: Weber Susanne
Published: April 2023

The following contribution will explicate, based on an understanding of networking as a reflexive process and on an approach working from a theory of regulation, one set of criteria for the development of evaluation designs in a networking context. Needs for evaluation and monitoring that is action- and future-oriented lead to other needs already established by social-ecological planning theory. From these can be generated questions for decisions in monitoring and evaluation within complex actor settings as well as criteria for concepts of evaluation and monitoring in a networking context. On this basis, four dimensions of network evaluation and monitoring are suggested and they are embedded in the multi-dimensional design approach of the “learning network” which puts collective competence development and future- and effect-orientation at the center of the developmental process.
Managing Extreme Evaluation Anxiety Through Nonverbal Communication
Authors: Schinker Regina S.
Published: April 2023

“Many evaluative situations cause people to fear that they will be found to bedeficient or inadequate by others...” (Donaldson, Gooler, & Scriven, 2002, p. 261). Donaldson, et al. (2002) use the acronym XEA to describe excessive anxiety and explain that “...there are people who are very upset by, and sometimes rendered virtually dysfunctional by, any prospect of evaluation, or who attack the evaluation without regards to how well conceived it might be” (ibid). A common technique or ‘magic bullet’ to prevent excessive anxiety would not exist in program evaluation.
The Problem of Free Will in Program Evaluation
Authors: Scriven Michael
Published: April 2023

Free will is the freedom to start a program: will power is what it takes to complete it.
Japan Evaluation Society: Pilot Test of an Accreditation Scheme for Evaluation Training
Authors: Nagao Masafumi
Published: April 2023

The Japan Evaluation Society (JES) has conducted a pilot test of an accreditation scheme for short-term evaluation training programs. The program chosen for this test was a 4-day school evaluation training course organized by a public teacher training center in cooperation with Hiroshima University.
Actualización sobre la evaluación en la región de América Latina y el Caribe
Authors: Chianca Thomaz
Published: April 2023

Este breve documento es una actualización sobre el ámbito de la evaluación en la región LAC (América Latina y el Caribe). Más específicamente, incide sobre cuatro aspectos de dicho ámbito: (1) evidencia sobre el número cada vez mayor de oportunidades de empleo para evaluadores/as en la región, (2) noticias sobre organizaciones nacionales de evaluación, (3) cursos de especialización disponibles en el 2005, y (4) el sistema de evaluación interna impulsado por el gobierno federal de Brasil.
Summary of American Journal of Evaluation, Volume 25(4), 2004
Authors: Mark Melvin M.
Published: April 2023

The following is excerpted from the introduction to Volume 25, Issue 4 of the American Journal of Evaluation, by former AJE editor Dr. Melvin M. Mark. It is reprinted here with permission from Dr. Mark; AJE’s current editor, Dr. Robin Miller; and the American Evaluation Association (AEA).The American Journal of Evaluation is the official journal of the American Evaluation Association and is distributed to AEA members as part of their membership package. To learn more about AEA and how to receive AJE, please go to www.eval.org.
Evaluation Capacity Building and Humanitarian Organization
Authors: Valéry Ridde , Shakir Sahibullah
Published: April 2023

This paper documents a process of evaluation capacity building in a humanitarianorganization in Afghanistan between 2001 and 2003. The authors carried out an annual evaluation and they undertook evaluation capacity building activities. The analysis of the empirical data shows that in the context of humanitarian organizations, the capacity building process would be improved if it would i) employ a mix of participative and utilization-focused approach, ii) organize participative workshops and on-the-job training, with the continuity of collaborators ensured, iii) use a myriad of dissemination/advocacy activities for a varied public.
Evaluation--Making it Real in Aotearoa New Zealand: Leading by Example, Leading by Association
Authors: Oliver Pam , McKegg Kate , Stone Geoff , Jakob-Hoff Maggie
Published: April 2023

The second Aotearoa New Zealand Evaluation Conference, sponsored by the Auckland Evaluation Group, will be held 18-20 July, 2005 at the Tauhara Centre, Acacia Bay, Taupo.
A Review of the Chinese National Center for Science and Technology Evaluation
Authors: Pan Luo Laura
Published: April 2023

China is now in a transitioning stage from planned economy to market economy. There is a growing interest in China to have a strong evaluation process in place so that planning and decisions can be based on valid and credible information. Evaluation also provides a guide for resource allocation.
Formative Evaluation of an Educational Technology Innovation: Developer's Insights into Assessment Tools for Teaching and Learning
Authors: Hattie John A. , Brown Gavin T. L. , Ward Lorrae , Gavin S. Earl , Keegan Peter J.
Published: April 2023

Formative evaluations conducted by the development team, focusing on the consequences of choices to be made by the developers, during the development of asTTle, a new ICT2-mediated educational assessment resource, are reported. The evaluations focused on the validity, accuracy, added value, training, and utility standards identified as important in the deployment of educational technology. As a result of in-house utilization of the evaluations, it is argued that users can have confidence that asTTle has demonstrated validity through alignment with the curriculum and classroom practice, that the reporting systems add value to teachers’ work and that accuracy of understanding is enhanced by professional development. Insights into effective PD have been obtained, though further improvements in the training component of the system are required. The utility of the software has been constantly increased, but there are still unresolved issues that are currently being responded to. The gradual implementation mechanism has been successfully used to obtain user buy-in as well as identification and response to identified needs. Users and funding agencies can have confidence that asTTle has been designed, developed, and implemented in a fashion consistent with maximizing benefits to the end-user. The software has been developed in such a way that the validity, added value, accuracy, training and utility standards for educational technology have been met, at least in part. The data reported here demonstrate again that formative evaluations which maximize end-user benefits can add significant value to an educational technology innovation.
A Scale for Describing Levels of Confirmation/Replication in Evaluation
Authors: Scriven Michael
Published: April 2023

In order to support both the validity and credibility of claims about program or product merit or worth, we often turn to evaluation. There are various ways in which evaluation can be conducted or managed, which themselves have differing levels of validity and credibility. The following scale can be used descriptively to indicate these levels, following the usual view of scientific method which identifies confirmation by others as a major criterion of validity. The higher numbers are not necessarily more valid, though they are generally more credible. Keep your sense of humor when reading this; it’s not only meant to be serious!
Contexts and Constraints: An Analysis of the Evolution of Evaluation in Ireland with Particular Reference to the Education System
Authors: O'Hara Joe , McNamara Gerry , Boyle Richard , Sullivan Connor
Published: April 2023

This paper is a case study of the emergence of an evaluation culture in the public sector and particularly in education in Ireland over the past three decades. It suggests that the emergence of this culture was strongly influenced by external factors, particularly the European Union (EU), and to a lesser but significant degree, the Organisation for Economic Cooperation and Development (OECD). Indeed, it can be argued that without these external influences no culture of evaluation would have emerged. Moreover it is further suggested that even after three decades the continuation of systematic evaluation is still probably dependent on external forces, since a belief in or commitment to evaluation as a tool of governance has not taken hold among key policy makers in Ireland. A second point made in the paper is that, while evaluation practice in Ireland was at first largely concentrated around EU funded projects and economic transfers, in more recent times evaluation has migrated to programmes and policies. In particular, it has become an element of the quality assurance processes institutionalised as part of the ‘reform agenda’ of the public service. Finally, the paper makes the point that an evaluation culture in a particular country—in this case, Ireland, but the same applies elsewhere—is hugely contextualised and influenced by the constraints of existing ideologies, traditions, practices and relationships between different interest groups. Thus, in Ireland, in line with the corporatist and partnership-driven approaches to economic policy and industrial relations which have been dominant in the past two decades, the form of evaluation which has emerged is consensual, collaborative and negotiated.
Culturally Competent Evaluation for Aboriginal Communities: A Review of the Empirical Literature
Authors: Chouinard Jill A. , Cousins J. Bradley
Published: April 2023

Much of program evaluation is concerned with understanding and improving social programs so that they are ultimately more responsive and more reflective of program participant needs. At the same time, these programs exist and are embedded within specific social, cultural and historical contexts which impact program development, implementation, and eventual outcomes. Evaluations that attempt to address responsiveness to contextual and cultural specificity are often referred to as culturally competent, culturally responsive, inclusive, multicultural, or cross-cultural, among other terms. While there are no agreed upon terminologies, definitions, or even methodologies, what these approaches all share is the recognition that culture and context matter, and that there are no universally agreed upon rules or abstractions that can be applicable in all contexts (Guba & Lincoln, 2005). The recognition of culture and context thus becomes “an explicit criterion rather than an unspoken expectation” (SenGupta, Hopson, & Thompson-Robinson, 2004, p. 15) in evaluations of this type.
Unlearning Some of our Social Scientist Habits
Authors: Davidson E. Jane
Published: April 2023

According to the organizational learning literature, some of the most important learning an organization can do is in the form of organizational unlearning—getting rid of erroneous or dysfunctional knowledge, habits, and practices. Simply adding new skills, behaviors, and ways of thinking isn’t enough. Without some explicit attention to the process of unlearning, existing knowledge and know-how becomes a barrier to constructive change and new learning.
A Summative Evaluation of RCT Methodology: & An Alternative Approach to Causal Research
Authors: Scriven Michael
Published: April 2023

This review focuses on what might be called a reconsideration of the working credentials of the RCT design, and includes some radical new perspectives on these, with relatively brief coverage of the usual suspects.
Logic Modeling Methods in Program Evaluation
Authors: Schröter Daniela C. , Urschel Jessica L.
Published: April 2023

A review of the book Logic Modeling Methods in Program Evaluation by Joy A. Frechtling published in 2007 by Jossey-Bass.
Cost Analysis in Evaluation
Authors: Scriven Michael
Published: April 2023

Program, product, and policy evaluation can rarely be done seriously without serious attention to costs. Yet the surveys of journal literature and meeting papers show that it is still very rarely considered.
The Economist's Fallacy
Authors: Scriven Michael
Published: April 2023

The economist’s definition of cost is totally insensitive to major differences in true cost.
Why Government and Non-Governmental Policies and Projects Fail Despite ‘Evaluations’: An Indicator to Measure whether Evaluation Systems Incorporate the Rules of Good Governance
Authors: Lempert David H.
Published: April 2023

Background: While much has been written in the evaluation literature on the theory of evaluations and on specific cases, there is still no comprehensive and easy to use indicator that can be used to hold organizations to the principles of effective evaluation, to score their quality in several areas, and to offer an immediate diagnostic for improvements.
Purpose: The article offers an easy-to-use indicator for measuring whether organizational evaluation policies for government and non- governmental organization spending actually protect the public interest in line with good governance and management principles or whether they serve, instead, to protect bureaucracies and hide wrongdoing under the cover of an “outside” evaluation.
Setting: The primary focus of the piece is on international development evaluations but the author shows how the same indicator and model can be used for other government agencies as well as businesses, with modifications.
Intervention: The article examines failures of evaluation systems in light of the principles for quality and shows how an indicator can be used to measure and prevent those failures.

Research Design: The piece defines the principles of evaluation systems and accountability using both the frameworks of international agencies, themselves, and professional texts, places them in a model framework for effective evaluations, and then turns this framework into a set of questions to derive an indicator.
Data Collection and Analysis: The article offers a sample detailed test of the indicator using the United Nations Development Programme (UNDP) as a case study. Several other organizations of different types are also tested briefly to yield a variety of results on the quality of their evaluation systems.
Findings: Use of this indicator on several organizations including those of the UN system, the EC, U.S.A.I.D., reveals that a number of governmental agencies and contractors (and particularly in the field of international development) are actually failing to protect the public interest and are using evaluation processes as tools to cover up abuses and mistakes and to advocate or advertise for more funding.
Using Abductive Research Logic to Construct a Rigorous Explanation of Amorphous Evaluation Findings
Authors: Levin-Rozalis Miri
Published: April 2023

Background: Two kinds of research logic prevail in scientific research: deductive research logic and inductive research logic. However, both fail in the field of evaluation, especially evaluation conducted in unfamiliar environments.
Purpose: In this article I wish to suggest the application of a research logic—abduction—the logic of discovery—which is powerful and very effective in constructing and validating explanations of new phenomena (evaluation findings, in particular).
Setting: The primary focus of the article is theoretic with a case example illustrating the practice of using the logic of discovery.
Intervention: Not applicable.Research Design: Not applicable.Data Collection and Analysis: Not applicable.Findings: Not applicable.
The 2009 Claremont Debates: The Promise and Pitfalls of Utilization-Focused and Empowerment Evaluation
Authors: Donaldson Stewart I. , Patton Michael Q. , Fetterman David M. , Scriven Michael
Published: April 2023

Background: Hundreds of evaluators visit the Claremont Colleges in southern California each year to discuss a wide range of topics related to improving the quality of evaluation practice. Debates between thought leaders in the field have been one of the most popular and informative ways to advance understanding about how best to practice evaluation in contemporary times.
Purpose: The purpose of this article is to provide a written transcript of the 2009 Claremont Evaluation Debates. The first debate is between Michael Quinn Patton and Michael Scriven on the promise and pitfalls of utilization-focused evaluation. The second debate is between David Fetterman, Michael Quinn Patton, and Michael Scriven on the promise and pitfalls of empowerment evaluation.

Setting: The debates occurred at the Claremont Graduate University on August 23-24, 2009. Several hundred evaluators from around the world also viewed and participated in the debates via a live webcast.
Intervention: Not applicable.Research Design: Not applicable.Data Collection and Analysis: Not applicable.Findings: Not applicable.
Rethinking Evaluation Methodology
Authors: Scriven Michael
Published: April 2023

Medicine, engineering, and evaluation have a highly significant common feature: they completely ignored the ban on evaluation that controlled the social sciences through most of the last century. What is it that doctors do in their core practice? They diagnose disease and malfunction, they recommend treatment, they encourage good health. What do engineers do? Amongst other tasks, they work out why the bridge failed, why the plane crashed, and how to correct the underlying errors and build better structures thereafter. And evaluators do the same with programs or policies or products or personnel—find the best, improve the flawed, report on the worst. It is the core nature of these essentially practical enterprises to be evaluative; they were not just describing or explaining or predicting how the world is, but trying to improve it. They simply didn’t take seriously that the essential nature of science had to be ‘value-free.’ Are there any lessons to be learnt from the methods used by our fellow-practitioners in these highly evaluative disciplines?
Quantitative Methods for Estimating the Reliability of Qualitative Data
Authors: Davey Jason W. , Gugiu P. Cristian , Coryn Chris L. S.
Published: April 2023

Background: Measurement is an indispensable aspect of conducting both quantitative and qualitative research and evaluation. With respect to qualitative research, measurement typically occurs during the coding process.
Purpose: This paper presents quantitative methods for determining the reliability of conclusions from qualitative data sources. Although some qualitative researchers disagree with such applications, a link between the qualitative and quantitative fields is successfully established through data collection and coding procedures.
Setting: Not applicable.
Intervention: Not applicable.
Research Design: Case study.
Data Collection and Analysis: Narrative data were collected from a random sample of 528 undergraduate students and 28 professors.
Findings: The calculation of the kappa statistic, weighted kappa statistic, ANOVA Binary Intraclass Correlation, and Kuder-Richardson 20 is illustrated through a fictitious example. Formulae are presented so that the researcher can calculate these estimators without the use of sophisticated statistical software.
Evaluating Faculty Perceptions of Student Learning Outcomes: A Rasch Measurement Analysis
Authors: Royal Kenneth
Published: April 2023

Background: The importance of assessing student learning outcomes has demanded attention from most everyone involved in the higher education enterprise, as accreditation and funding implications are often linked to the results. Faculty, however, are often critical of the assessment process because outcomes assessment is costly with regard to time, energy and other resources, and evidence of its effectiveness is not always noticeable.
Purpose: The purpose of this article is to evaluate faculty perceptions of various student learning outcomes in order to determine which types outcomes are most valued by faculty.
Setting: United States.
Intervention: Not applicable.
Research Design: The HERI Faculty Survey was utilized to capture a nationally representative snapshot of faculty perceptions of student learning outcomes.
Data Collection and Analysis: The Rating Scale Model, a Rasch measurement model, was used to analyze survey data of 7,356 respondents.
Findings: Faculty from virtually all disciplines are primarily concerned with the intellectual growth of students. All other types of student learning outcomes (i.e., social, emotional, cultural growth) are of lesser concern. These findings suggest higher education institutions seeking “faculty buy-in” may want to consider focusing more on intellectual types of outcomes as these outcomes appear to best resonate with faculty, and in turn may result in less faculty resistance.
Evaluation and Limitations of Social Interventions: The Case of Spain
Authors: Ligero Lasa Juan A.
Published: April 2023

Background: Although the number and practice of evaluation varies enormously among policy areas, there are very few studies about the unequal evaluation development in the different policy sectors.
Purpose: This article aims to (1) acknowledge the different evaluation development among different policy sectors in Spain (2) identify the factors and causes that provoke this disproportion, and (3) explore potential consequences of this unequal distribution of evaluation studies among policy areas.
Setting: Spain.
Intervention: Public polices in Spain.
Research Design: A sample of evaluations is classified by policy sector and the number of evaluations in each sector is analyzed and compared. Then, other significant variables are identified for explaining differences among sectors.

Data Collection and Analysis: The cases (evaluation studies) are drawn from two samples: (1) a data base of evaluation studies and (2) a survey to Spanish evaluators held in 2009. The comparison was done with difference in proportions, adjusted standardised residuals and crosstabs.
Findings: Analysis of Spanish evaluations shows that program evaluations are much more frequent in the social policies’ area than in the areas of security, defense or justice. A variable with a high ability to predict whether or not evaluations will be carried out is identified: the selective versus universal nature of the policies being evaluated. Selective interventions are more frequently evaluated than universal policies. This lack of balance makes selective interventions more prone to severe critical analysis. This evaluation bias, in turn, produces a series of perverse effects such as a greater probability of cutting down programs based on selective application strategies.
Integrating Evaluation and Agent-Based Modeling: Rationale and an Example for Adopting Evidence-Based Practices
Authors: Morell Jonathan A. , Hilscher Rainer , Magura Stephen , Ford Jay
Published: April 2023

Background: While there is a great deal of discussion about complex adaptive systems in the field of evaluation, little has been written about the expression of complex adaptive systems in terms of agent-based models, the execution of those models as computer simulations, or the tight integration of agent-based modeling with traditional evaluation methods. These topics need to be explored if evaluation is to move beyond using complex adaptive systems in an exclusively heuristic fashion.
Purpose: This paper advocates for the integration of agent based modeling into traditional evaluation activities. We advance this position in order to spark a movement toward incorporating the formal application of complex adaptive systems into the theory and methods of evaluation. To make our case, we provide a hypothetical example of how interactions between an agent-based model and evaluation can provide unique and powerful understanding about the adoption of evidence-based practices in the field of addiction treatment. We advance our argument by addressing four questions: Why combine evaluation methods with simulation methods? Why use a complex adaptive systems approach? Why add agent-based modeling to the mix of evaluation tools? What is the relationship between agent-based modeling and the theories and methods of evaluation that are being developed to deal with surprise, unintended consequences, and program evolution?
Setting: Not applicable.
Intervention: Not applicable.
Research Design: Not applicable.
Data Collection and Analysis: Not applicable.
Findings: Not applicable.
Theory Construction and Model-Building Skills: A Practical Guide for Social Scientists
Authors: Pratt Brandy
Published: April 2023

A review of the book Theory Construction and Model-Building Skills: A Practical Guide for Social Scientists by James Jaccard & Jacob Jacoby published in 2010 by Guilford.
Theory Construction and Model-Building Skills: A Practical Guide for Social Scientists
Authors: French Stacy
Published: April 2023

A review of the book Theory Construction and Model-Building Skills: A Practical Guide for Social Scientists by James Jaccard & Jacob Jacoby published in 2010 by Guilford.
Credibility and Independence of Evaluation in UNDP: A Response to David Lempert
Authors: Uitto Juha I.
Published: April 2023

The paper contains a response to the article by David Lempert on 'Why Government and Non-governmental Policies and Projects Fail Despite Evaluation' (JMDE, Vol. 6, No. 13, 2010). The paper focuses on the role of the evaluation function in public organizations, using the example of the United Nations Development Programme. It makes broader observations regarding the independence and credibility of evaluation. Finally, the role of evaluation in oversight is discussed.
An Interview with Ralph Tyler
Authors: Nowakowski Jeri Ridings
Published: April 2023

Originally published as Paper #13, Occasional Paper Series, November, 1981
Benefits of the Person-Oriented Perspective for Program Evaluation: Analyzing the Differential Treatment Effects of the Vienna E-Lecturing Program
Authors: Lapka Dominik , Wagner Petra , Schober Barbara , Gradinger Petra , Spiel Christiane
Published: April 2023

Background: In program evaluation interventions are usually examined for global effects but not always for differential effects. The reasoning behind the focus on global effects is that most of the concepts applied in data analyses, and most of the hypotheses being assessed, are variable-oriented. Consequently, existing differential effects will not be detected through global analyses based on a variable-oriented perspective. Accordingly, the intervention under evaluation will be wrongly considered to be ineffective.
Purpose: This contribution, therefore, argues for the application of person-oriented analyses in addition to the common variable-oriented approach in order to identify the differential effects of an intervention during evaluation.
Setting: Both approaches will be applied in the evaluation of a showcase program conducted in real life setting. The results of the two perspectives will be compared.
Intervention: An internet supported program for the promotion of self-regulated learning in an academic context called Vienna E-Lecturing (VEL).
Research Design: The evaluation of the VEL was conducted with a treatment-control-design. Data was collected at the beginning as well as at the end of the intervention.
Data Collection and Analysis: Data regarding motivation as well as factual knowledge was collected. All motivational variables were assessed using previously published scales consisting of 3-4 items. Factual knowledge in statistics and methodology was tested regarding the aspects of reproduction, comprehension and the production of methodological content.
Findings: The application of a variable-oriented perspective showed small effects with low practical relevance. Through the application of a person-oriented approach, and taking into account the theoretical framework of the intervention, three students groups which differed systematically in their learning motivation could be identified. For two of these groups the intervention was very successful. The discussion provides recommendations on how person-oriented approaches can be used systematically in program evaluation.
Empathy Training: Methods, Evaluation Practices, and Validity
Authors: Lam Tony Chiu Ming , Kolomitro Klodiana , Alamparambil Flanny C.
Published: April 2023

Background: Empathy is an individual’s capacity to understand the behavior of others, to experience their feelings, and to express that understanding to them. Empathic ability is an asset professionally for individuals, such as teachers, physicians and social workers, who work with people.  Being empathetic is also critical to our being able to live with others in general, and ultimately to leading happier lives. Subsequently it seems imperative to examine if and how it is possible to enhance people’s empathic ability.
Purpose: The purpose of this article is to use narrative review method to analyze studies of empathy training in human service and social science disciplines over the past thirty years to address the questions: “How have people been trained in empathy and what are the findings?” and “How was empathy training evaluated and how valid are these evaluation findings?” 
Setting: Not applicable.
Intervention: Not applicable.
Research Design: Not applicable.
Data Collection and Analysis: Narrative review.
Findings: Twenty-nine articles pertaining to empathy training evaluation research were identified based on an advanced computer search on the following databases: “Education Full Text,” “ProQuest Education Journals,” “Web of Knowledge” and “Educational Resources Information Center (ERIC). Seven types of training methods were noted in these 29 evaluations with the most popular being didactic related (42%). All but two studies (93%) reported positive findings, mainly in regard to learning (86%), or the cognitive component of empathy. These findings suggest that regardless of the training method, individuals can learn about the concept of empathy. Unfortunately, information pertaining to the effects of training on individuals’ feeling for others, and their ability and propensity to take the perspective of others and to demonstrate it in the natural environments is lacking. Consequently, very little is known about the trainability of the affective and behavioral components of empathy. Also, some of the findings were moderated by gender, age, education level, and time of measurement. Regarding evaluation research designs, most of the studies used self-reporting to collect trainees’ knowledge about empathy and most of the quantitative studies used a control group and pretesting to examine training impact. Construct validity of both empathy measurement and training is very problematic. A majority of the studies did not clearly define empathy, provide training as defined, and/or measure what is being trained; conceptualization of empathy across studies was not consistent either.  In sum, data from the studies reviewed were neither complete nor valid enough to provide a clear and full understanding of the trainability of empathy. More research is apparently needed and hopefully lessons learned from our review will be considered in designing future studies.
Return on Investment: A Placebo for the Chief Financial Officer… And Other Paradoxes
Authors: Andru Peter , Botchkarev Alexei
Published: April 2023

Background: Return on investment (ROI) is one of the most popular evaluation metrics. ROI analysis (when applied correctly) is a powerful tool of evaluating existing information systems and making informed decisions on the acquisitions. However, practical use of the ROI is complicated by a number of uncertainties and controversies. The article reveals some of these controversies in an engaging and thought-provocative manner.
Purpose: The intent of this note is to highlight several of the ROI paradoxes in a format of an opinion or a viewpoint with a hope that drawing attention of the ROI practitioners and researchers to these issues will contribute to more transparent and responsible application of the ROI evaluation.
Setting: Not applicable.
Intervention: Not applicable.
Research Design: Not applicable.
Data Collection and Analysis: Review of current practice.
Findings: The article reveals three weaknesses of the ROI evaluations, which in the absence of the commonly accepted ROI standard, can make results of the ROI evaluations uncertain or questionable.
Evaluation Essentials: From A to Z
Authors: Hobson Kristin A.
Published: April 2023

A review of the book Evaluation Essentials: From A to Z by Marvin C. Alkin published in 2011 by Guilford Press.
Conducting Democratic Evaluations where Democratic Principles are not always Practiced
Authors: Kurt Serhat
Published: April 2023

Background: Turkey is a fast-developing country facing a lot of different problems. One of the problems is inadequate education. A large project was started to improve Turkish elementary education. The project includes many different purposes from physical refurbishment of school buildings, to increasing technology use in education. The author of this paper carried out a democratic study to evaluate this project. This article reports on what was learned from this evaluation study conducted in Turkey.   Purpose: The purpose of this article is to examine how democratic evaluations can be used to help understand the complex realities where undemocratic ideology has a long history.    Setting: This evaluation study was conducted in an elementary school from a lower-middle class neighborhood.   Intervention: Not applicable.   Research Design: Qualitative method of inquiry was applied.   Data Collection and Analysis: The data mostly came from the interviews. However document analysis and observations were also conducted. Qualitative data from the interviews and observations were analyzed to interpret meaningful patterns or themes.   Findings: It was concluded that democratic evaluations can serve an important mission by informing the public when the public’s role is limited in the decision making process. Further, democratic evaluators can serve as middle men who transfer information among the stakeholders, helping the evaluated program’s or project’s success. Finally, democratic evaluations can help increase democracy where democratic values are not always obeyed.   Keywords: Democratic evaluation; Turkey; education; democracy; educational technology
Thinking Beyond Measurement, Description and Judgement: Fourth Generation Evaluation in Family-Centered Pediatric Healthcare Organizations
Authors: Moreau Katherine Ann , Clarkin Chantalle Louise
Published: April 2023

Background: Although pediatric healthcare organizations have widely implemented the philosophy of family-centered care (FCC), evaluators and health professionals have not explored how to preserve the philosophy of FCC in evaluation processes. Purpose:  To illustrate how fourth generation evaluation, in theory, could facilitate collaboration between evaluators and families and uphold the philosophy of FCC in evaluation. Exploration focuses on describing the ways in which fourth generation evaluation is consistent with FCC and outlining a strategy for implementing it within pediatric healthcare organizations.  Discussion: Current evaluation practices used in healthcare organizations reflect what some describe as the first three generations of evaluation: measurement-, descriptive-, and judgment-oriented evaluation. While these generations encourage evaluators and health professionals to use systematic and rigorous approaches and techniques, they negate opportunities to explore issues that may surface in more flexible evaluation processes and do little to promote FCC in evaluation. Fourth generation evaluation is based on the constructivist paradigm, and its hermeneutic dialectic process moves beyond these generations, as well as the problems associated with them, to reflect the FCC notions of family participation, partnership, collaboration, respect, and joint decision-making. Conclusion: The collaborative and dialogue-oriented environment of pediatric healthcare organizations provides an ideal context for fourth generation evaluation. Although this evaluation approach is consistent with the philosophy of FCC, more research is required to understand the strengths and limitations of using it in these organizations.Keywords: family-centered care; pediatrics; fourth generation evaluation; constructivist paradigm
Stakeholder Involvement in Evaluation: Three Decades of the American Journal of Evaluation
Authors: Rodriguez-Campos Liliana
Published: April 2023

Background: Stakeholder involvement in various phases of evaluation has received increasing attention over the past three decades. Indeed, the American Journal of Evaluation (AJE) has reflected this overall philosophy and strategy through a number of publications about several theoretical frameworks and practical applications. Those AJE articles outline the primary assumptions of the stakeholder approaches to evaluation, their practical applications, constraints, and benefits for providing a new direction in evaluation.   Purpose: What lessons are there for our field concerning the way that stakeholder involvement has been conceptualized and applied in AJE? This article focuses on the ways in which AJE authors have approached the notion of stakeholder involvement over the past three decades, drawing on key articles from scholarship on this topic. Setting: Not applicable.   Intervention: Not applicable.   Research Design: Not applicable.   Data Collection and Analysis: Desk review.   Findings: This review shows how AJE has enhanced our understanding of the evolution of these approaches to evaluation, from a global perspective to differentiated approaches with a shared theme.   Keywords: collaborative evaluation; participatory evaluation; empowerment evaluation; stakeholder approaches to evaluation; stakeholder involvement
Assessing Implementation Integrity of a National Nutrition Education Program: A Case Study of Share Our Strength's Operation Frontline
Authors: Coryn Chris L. S. , Schröter Daniela C. , Cullen Anne , Semen Laura , McLaughlin Janet
Published: April 2023

Background: Treatment implementation is not just one thing but rather is a multifaceted process that includes treatment delivery, treatment receipt, and treatment adherence. As such, local variations in implementation and service delivery of interventions are an inevitable. Purpose: To assess implementation fidelity of a multi-site experiential nutrition education program. Setting: Multiple sites throughout the continental United States. Intervention: An experiential nutrition education program. Research Design: A concurrent mixed methods design was used to assess implementation fidelity. Data Collection and Analysis: Multiple methods of data collection and analysis were used including observations, interviews, survey questionnaires, and extant data. Findings: Although implementation fidelity varied over program sites, overall implementation fidelity was very good and when it varied, it varied to local site needs and context. Keywords: program fidelity; program implementation; nutrition education
La gamificación como eje motivador para el aprendizaje de la matemática
Authors: Delgado José , Espinoza Mariuxi , Vivanco Cristina , Medina Niorka , Ayala Marco
Published: April 2023

La motivación dentro de la educación tiene impacto en el rendimiento académico de los estudiantes, pues se trata de un conjunto de factores internos y externos que estimulan el deseo y la energía para interesarse y comprometerse con un tema. Para mejorar la motivación de los estudiantes en el aprendizaje de matemática, se tiene que implementar estrategias didácticas innovadoras y acordes al contexto educativo. Ante ello, la gamificación es una estrategia diseñada para motivar el proceso de enseñanza aprendizaje y fomentar el desarrollo de habilidades. La presente investigación tiene como objetivo general analizar el uso de la gamificación como eje motivador para el aprendizaje de las matemáticas. Para ello se implementó una propuesta de intervención pedagógica incorporando la gamificación con sus tres elementos: mecánica, dinámica y estética. Se enmarca en un estudio de tipo descriptivo, enfoque cuantitativo, diseño de campo y transversal. La muestra se conformó de 28 estudiantes, las técnicas que se aplicó fueron, la observación y prueba objetiva, los instrumentos, una rúbrica para medir el grado de motivación de los alumnos, un pre y post test. En los resultados se observó un nivel alto de motivación en los estudiantes mediante el uso del monopolio como recurso didáctico. La conclusión más relevante es que, mejoró el rendimiento académico de los estudiantes del sexto grado de educación general básica.
PUBLICATIONS DU SECTEUR ACADEMIQUE DE MEDECINE GENERALE FRANCAIS ENTRE 2011 ET 2018
Authors: TAHA A , ORCEL V , ROUGE-BUGAT M-E , BEIS J-N , RENARD V , SAINT-LARY O
Published: April 2023

Contexte. Les publications de médecine générale progressent dans tous les pays, en particulier ceux dont le système de santé est structuré autour des soins premiers. Aucune donnée n’était disponible pour les publications du secteur académique de médecine générale français. Objectif. Décrire les caractéristiques des publications scientifiques du secteur académique de médecine générale entre 2011 et 2018. Méthode. Un double recueil des publications a été réalisé chaque année entre 2011 et 2018 : un autoquestionnaire a été adressé à chaque département de médecine générale qui devait déclarer ses publications ; une requête PubMed® a été réalisée pour trouver les publications dont un auteur était affilié au secteur académique de médecine générale. Les caractéristiques de chaque publication ont été recherchées. Résultats. Parmi les 1 645 publications recensées, 911 provenaient de revues indexées dans PubMed®. D’après la CISP-2, les publications indexées concernaient principalement la thématique « générale et non spécifiée » (26,9 %) suivie par les thématiques « psychologique » (13 %) et « circulatoire » (9,1 %). Entre 2011 et 2018, la médiane des facteurs d’impact variait entre 0,899 et 2,302. La part des publications avec un premier auteur variait entre 55,9 et 73,9 %, celle avec un dernier auteur affilié au secteur académique variait entre 23,5 et 39,2 %. Conclusion. Les publications ont progressé entre 2011 et 2018, suivant ainsi la tendance internationale. Le secteur académique de médecine générale français témoigne ainsi de sa structuration.
SITUATION EN SUISS
Authors: GOUVEIA A , COHIDON C
Published: April 2023

La Confédération suisse est un État fédéral composé de 26 cantons, qui jouissent d’une importante autonomie administrative et juridique (Office fédéral de la statistique [OFS], 2022). En 2020, la Suisse comptait 8 670 300 habitants, dont la majorité parlait l’allemand (62,3 %), et des minorités le français (22,8 %), l’italien (8 %) et le romanche (0,5 %). Après l’annonce des premiers cas de Covid-19 en Suisse fin février 2020, le Conseil fédéral a d’abord décidé d’interdire les manifestations de plus de 1 000 personnes dans tout le territoire. Quelques semaines plus tard, la Suisse est formellement placée en « état d’urgence », selon la loi sur les épidémies, et ceci de façon concomitante avec d’autres pays européens. La situation exceptionnelle d’état d’urgence a permis temporairement au Conseil fédéral de prendre des décisions sans besoin d’approbation parlementaire ou cantonale, comme le début du semi-confinement, l’interruption des activités éducatives et la généralisation de l’éloignement social.