bluuebunny/crossref_metadata_2025 huggingface dataset:
Using Cognitive Interviewing to Test Youth Survey and Interview Items in Evaluation: A Case Example
Authors: LaPietra Elisa , Brown Urban Jennifer , Linver Miriam R.
Published: April 2023

Background: Cognitive interviewing is a pretesting tool used by evaluators to increase item and response option validity. Cognitive interviewing techniques are used to assess the cognitive processes utilized by participants to respond to items. This approach is particularly appropriate for testing items with children and adolescents who have more limited cognitive capacities than adults, vary in their cognitive development, and have a unique perspective on their life experiences and context.

Purpose: This paper presents a case example of cognitive interviewing with youth as part of a national program evaluation, and aims to expand the use of cognitive interviewing as a pretesting tool for both quantitative and qualitative items in evaluation studies involving youth.

Setting: Youth participants were located in four regions of the United States: Northeast, Central, Southern, and Western. Interviewers were located at Montclair State University.

Intervention: Not applicable.

Research design: A cognitive interview measure was designed to include a subset of survey items, interview questions, and verbal probes, to evaluate if these items and questions would be understood as intended by both younger and older youth participants. An iterative design was used with cognitive interviewing testing rounds, analysis, and revisions.

Data Collection and Analysis: The cognitive interview was administered by phone to 10 male youth, five from the 10-13-year-old age range and five from the 15-17-year-old age range. Interviews were audio-recorded, transcribed, reviewed, and coded. Survey items and interview questions were revised based on feedback from the participants and consensus agreement among the evaluation team. Item revisions were included in further testing rounds with new participants.

Findings: As a result of using cognitive interviewing to pretest survey and interview items with youth, response errors were identified. Participants did not understand some of the items and response options as intended, indicating problems with validity. These findings support the use of cognitive interviewing for testing and modifying survey items adapted for use with youth, as well as qualitative interview items. Additionally, the perspective of the youth participants was valuable for informing decisions to modify items and helping the evaluators learn the participants’ program culture and experiences. Based on the findings and limitations of the study, we give practice recommendations for future studies using cognitive interviewing with a youth sample.

Keywords: cognitive interviewing; item validity; response error; verbal probes; pre-testing surveys; qualitative evaluation; interviewing children and adolescents; survey development
Dialogic and Generative Reflection: An Application of the Critical Appreciative Process in Program Evaluation
Authors: He Ye , Smith Tiffany L. , Moteane Malitsitso
Published: April 2023

Background: Even though the positive potential of reflective practice is widely acknowledged across professional fields, it has been recognized that reflective practice may be carried out primarily as an individual-based exercise, and at the technical or descriptive level without generative impact. Dialogic reflective processes involving both evaluators and program directors are far from being systematically implemented or examined.

Purpose: The purpose of this article is to share our experiences engaging in dialogic and generative reflections as the project director and program evaluators of a K-12 teacher education program using the critical appreciative process. Building upon the reflective practice traditions in both disciplinary areas, we introduce the use of the critical appreciative process as a promising model to guide dialogic and generative reflection to support the co-design and improvement of the program and accompanying evaluation efforts.

Setting: The project director and evaluators are engaged in a grant-funded teacher preparation project designed to prepare teachers for K-12 English learners and dual language learners. The project builds upon partnerships between the university teacher preparation program and two local school districts. The evaluation plan was designed based on culturally responsive, collaborative, and use-focused evaluation approaches and theory. In 2020, the project team faced critical decisions in the context of the COVID-19 pandemic.

Intervention: Not applicable.

Research Design: We applied self-study methodology to guide data collection and analysis in this study. The primary data source included individual written reflections and group critical friend dialogues guided by the critical appreciative process. Both the reflections and meeting notes were analyzed to identify convergent and divergent perspectives shared throughout the critical appreciative process and to highlight implications for both the evaluation and the program moving forward.

Findings: Convergent and divergent perspectives from both the project director and the evaluators were shared based on the 4-D critical appreciative process: Discover, Dream, Design, and Deliver. Based on this shared experience, we illustrate how the dialogic reflective process entails reflexivity and requires pausing; how reflective practice in program evaluation situates our dialogues as learning-oriented rather than a mere accountability discussion; and how reflective action can create a dialogic and generative virtuous cycle.

Keywords: reflective practice; evaluation; critical appreciative process
Unpacking the Participatory Process
Authors: Weaver Linda , Cousins J. Bradley
Published: April 2023

This paper questions the adequacy of the process dimensions of the earlier version of the participatory framework. Their ongoing analysis of process dimensions reveals that the stakeholder selection dimension is problematic and needs reconsideration. This paper represents the framework and describes enhancements to the process dimension component. By way of illustration, the framework is applied to two separate case examples of practical participatory evaluation.
Evaluation Activities in Africa
Authors: Lamphear Paul A.
Published: April 2023

This article discusses the recent growth of evaluation associations and activities in Africa, namely the founding of the African Evaluation Association (AfrEA) in 1999. AfrEA’s initiatives are highlighted, including developing cultural evaluation guidelines, establishing training programs, and connecting to international organizations that support evaluation in Africa.
Evaluation in Latin America and the Caribbean: An Overview of Recent Developments
Authors: Chianca Thomaz , Youker Brandon W.
Published: April 2023

This article discusses the state of evaluation in Latin America and the Caribbean and outlines the professional organizations and publications in the region.
Washington, DC: Evaluation of Driver Education
Authors: Northport Associates
Published: April 2023

This is intended to provide a brief overview of this project and of the consultative workshop which took place last week in Washington, D.C.
African Evaluation Association
Authors: AfrEA
Published: April 2023

The African Evaluation Association (AfrEA) held its Third Conference in CapeTown, South Africa, on December 1-4, 2004. The conference was attended by 479 people from 61 countries, of which 47 were in Africa. Around 250 participants also attended the 10 pre-conference professional development workshops held on 29-30 November.
An Update on Evaluation in Europe
Authors: Schröter Daniela C.
Published: April 2023

Details of events and news items listed below can be found on the corresponding Web sites.
An Update on Evaluation in the Latin American and Caribbean Region
Authors: Chianca Thomaz
Published: April 2023

This short paper updates the evaluation scene in the Latin American and Caribbean (LAC) region. Specifically, it covers four aspects of that scene: (1) evidence on the growing number of job opportunities for evaluators in the region, (2) news about national-level evaluation organizations, (3) professional development courses available in 2005, and (4) the internal evaluation system supported by the Brazilian federal government.
The Evaluation Exchange--Harvard Family Research Project
Authors: Youker Brandon W.
Published: April 2023

Harvard Family Research Project (HFRP) was founded by the Harvard Graduate School of Education in 1983. The HFRP aims to help strengthen family, school, and community partnerships of early childhood care and education; promote evaluation and accountability; and offer professional development to those who work with children and/or their families. The project has aided philanthropies, policymakers, and practitioners by collecting, analyzing, and synthesizing research and information.
Evaluation: The International Journal of Theory, Research and Practice, Volume 10(4), October 2004
Authors: Schröter Daniela C.
Published: April 2023

The most recent issue of Evaluation contains six articles, one contribution to A Visit to the World of Practice, and News from the Community respectively.
Ethnography and Evaluation: Their Relationship and Three Anthropological Models of Evaluation
Authors: Youker Brandon W.
Published: April 2023

This paper examines the relationship between ethnographic research methods and evaluation theory and methodology. It is divided into two main sections: (a) ethnography in evaluation and (b) anthropological models of evaluation. Three levels of the leading anthropological models of evaluation are summarized, which include responsive evaluation, goal-free evaluation, and constructivist evaluation. In conclusion, (a) there is no consensual definition of ethnography; (b) in many circumstances, ethnographic evaluation models may be beneficial; and (c) ethnography can be used in evaluation but requires a high level of analysis to transform ethnographic data into useful information for eliciting an evaluative conclusion.
A Call to Action: The First International Congress of Qualitative Inquiry
Authors: Coryn Chris L. S. , Schröter Daniela C. , Scriven Michael
Published: April 2023

The First International Congress of Qualitative Inquiry, held at the University of Illinois at Urbana-Champaign from May 5-7, 2005, was assembled so that the international community of qualitative researchers could address the implications of attempts by federal funding agencies to “regulate scientific inquiry by defining what is good science” (Denzin, 2005b). The Congress was attended by more than 800 persons from more than 45 nations. More than 160 sessions consisting of more than 650 papers authored by more than 750 persons were presented. The complete Congress program, including session and paper abstracts, complete papers, and other information is available at http://www.qi2005.org/.
American Journal of Evaluation
Authors: Wingate Lori
Published: April 2023

Volume 26, number 1 was the first issue of the American Journal of Evaluation (Volume 26, Number) with Robin Miller at the helm as Editor. There are 5 articles, plus 1 contribution each in “Forum,” “Methods Notes,” “The Historical Record,” and “Ethical Challenges.”
In This Issue
Authors: Scriven Michael
Published: April 2023

This is a particularly interesting issue, which is just as well since it’s also our longest to date—over 220 pages, and I doubt you can find a way to shorten it without a hundred readers feeling seriously deprived!
Using Test Standard-Setting Methods in Educational Program Evaluation: Addressing the Issue of How Good is Good Enough
Authors: Brandon Paul R.
Published: April 2023

School districts in the United States and elsewhere commonly use standard setting to assign value to student test and assessment scores. That is, they set standards to show “how good is good enough.” This paper presents a summary of the empirical findings on the most widely-studied test standard-setting method and describes what the conclusions of the summary suggest about the use of test standard-setting in educational program evaluations.
In This Issue
Authors: Scriven Michael
Published: April 2023

In this issue, we have some methodological reflections from another valorous (female) evaluator working successfully on the HIV/AIDS front in the darkest reaches of the Amazon jungle, with a largely law-breaking clientele whose general attitude about any breach of confidentiality seems to have been that it would probably cause their death, so why not yours as well. The article is a kind of handbook of hazards for evaluators working ‘in harm’s way’ and hence a good guide to planning for others. Interestingly, it may provide the strongest argument for empowerment evaluation: do it when it’s the only possibility.
Using Culturally Sensitive Methodologies When Researching Diverse Cultures
Authors: Baugh Eboni J. , Guion Lisa
Published: April 2023

This article introduces additional information in the ongoing conversation about using culturally sensitive research methods with diverse research populations. Research, including evaluation research, examining ethnic minorities, international, tribal members, and individuals within diverse cultures should be performed within a context of cultural understanding. Several methodological examples will be presented, expanding the discussion of contemporary research with diverse cultures.
Evaluating a Health Education Project in Maine
Authors: Hubley Teresa A. , Amrich MaryAnn
Published: April 2023

The following paper details the evaluation of a public health education project in the state of Maine. Evaluation of education projects presents a challenge, in that the effects of the intervention are not easy to trace and outside influences difficult to impossible to control. This study approached this difficult issue at the start by focusing on one variable for which data are readily available (namely blood lead testing rates). The evaluation was further enhanced by use of a model called "RE-AIM", which measures the reach, efficacy, adoption, implementation and maintenance of educational projects (Glasgow, et.al. 1999). Measurements centered on data tracked through newly created databases and focused on elements directly attributable to the project (i.e. behavior of medical personnel trained through project activities). Finally, small focus groups and interactions with families served by the program were used to derive qualitative data that provided a broader perspective on the success of activities. As the program ultimately seeks to entirely eliminate childhood lead poisoning, this paper concludes with a discussion of areas that continue to need attention for future education projects.
The Politics of Local Control and Program Evaluation in Community Context
Authors: Rosenberg Helen , Statham Anne , Wang George , Letven Esther
Published: April 2023

This is a case study of an evaluation of a Youth Fair Chance (YFC) program that discusses multiple challenges faced by our evaluation team. We consider the role of evaluators. in relation to the overall program structure, struggles among organizational stakeholders with special interests, and power differentials which impacted policy decision making, control of information, and ultimately, the outcomes of this evaluation. In identifying the particular issues that arose throughout the evaluation process, we hope others who face similar issues will benefit from our insights. We realize the issues we face are endemic to any evaluation process.
Asymmetrical Tests
Authors: Scriven Michael
Published: April 2023

It is often thought, or presupposed, that investigative tests done for evaluative purposes should be symmetrical—that is, they should be equally capable of giving positive and negative results. This is an error in the logic of evaluation, and it arises from confusion between the asymmetrical ability of a test to yield information and its propensity to yield biased information. Although ‘asymmetrical’ does connote one-sidedness and we often use ‘one-sided’ to mean biased, we can and should use the terms more carefully. Some tests can only yield information about the faults of a program (or product, or person, etc.)—or only about its virtues—while others can yield one of these more reliably than the other, and yet others are symmetrical in their treatment of merits and demerits. All of them gather relevant information for evaluative purposes, and one cannot conclude that any of them are biased simply because they are asymmetrical. The point is important because in many situations, one may only have access to asymmetrical tests and this does not support the claim of a biased approach as long as there is more than one test in the battery used to evaluate, and one test’s asymmetry is balanced out by the other test(s).
Progress Report on JMDE
Authors: Scriven Michael
Published: April 2023

We have now put out 5 issues, which is conventionally an occasion for summary and reflection. I’ll keep it brief. We now have 1,750 readers who have asked to be notified of the appearance of each issue: we’ll call them ‘committed readers,’ perhaps our closest approximation to subscribers to hardcopy journals. We have had over 19,000 hits on the website, according to our counter, but do not know how many of these read, or download, or make copies for others, of the material. An independent estimate of the number of hits from Lars Balzar’s website (q.v. via Google) places the numbers a little higher, at 12,000 to 30,000 per day (!). Our issues are now widely announced in a considerable number of journals and websites-and many thanks for their help in dissemination.
Collaborative Evaluation
Authors: Cousins J. Bradley
Published: April 2023

When I was initially approached by Michael Scriven to review Collaborative Evaluations I was entirely enthusiastic about the invitation. It was only after having received the book that I took note of subtitle ‘A step-by-step model for the evaluator.’ Hmmm, I thought. Now this looks like an interesting twist. To be completely honest, my enthusiasm turned to skepticism, even before I had cracked the cover. Let me explain.
Evaluation of the Wetland in Yencheng, China
Authors: Pan Luo Laura
Published: April 2023

The wetlands of China are components of hydrological systems that, when well managed, provide water in periods of shortage, important flood controls in periods of heavy rain or snowmelt, food for millions of people, and breeding and feeding habitat for many wild species. In July 2006, I participated in a monitoring and evaluation mission of a UNDP-GEF project on Conservation and Sustainable Use of Wetlands in China. The project in Yancheng aims at establishing wetland biodiversity conservation as a routine consideration in national, provincial and local government decision making and action. Started in June 2005, the project ends by the end of 2007. The evaluation team reviewed the progress achieved so far and identified implementation problems.
A Study of the Family and Community Support Project in China
Authors: Pan Luo Laura , Qiu Aijun , Zhang Yali
Published: April 2023

The United Nations Children’s Fund (UNICEF) sponsored a Family and Community Support Project in China from 2001 through 2005. Implemented by the China Center for Town Reform and Development local Chinese governments and relevant organizations, the project aimed at protecting children’s rights and well-being, and at developing communities that help children participate fully in school and family life. Three communities, from Beijing, Tianjing and Shanghai, were selected as pilot communities for this project. At the conclusion of the project, UNICEF invited a team of specialists in the field of sociology, program evaluation and other related disciplines to review and document experiences and lessons learned from the project.
Review of Japanese Journal of Evaluation Studies (Volume 6, Number 1)--Special Topic: “Evidence-Based Evaluation”
Authors: Sasaki Ryoh
Published: April 2023

The Japan Evaluation Society (JES) was established in 2000. Since then, JES has published its professional journal twice per year (see the archives at http://www.idcj.or.jp/jes). The most recent issue—Volume 6, Number 1—was published in March 2006. It includes 12 papers and the summary report of the 6thannual conference held at Hiroshima University. With more than 200 pages, this issue marks the largest volume since the journal started. We can say that evaluation is becoming popular and receiving significant attention by academia and practitioners in Japan.
Review of Social Network Analysis in Program Evaluation
Authors: Martens Krystin
Published: April 2023

The fall, 2005, New Directions for Evaluation journal included nine articles dedicated to social network analysis (SNA) in Program Evaluation. Four of the nine articles in the journal explored the academics of SNA methodology, while the other five articles presented examples of quite diverse applications of SNA in program evaluation. This review will outline the major arguments for the use of SNA, applications of SNA in program evaluation, and a critique of the SNA content contained in this journal.
Program Evaluation Development in the Newly Independent States
Authors: Karimov Afar , Borovykh Alexander , Kuzmin Alexey , Abdykadyrova Asel , Efendiev Djahangir , Greshnova Ekaterina , Konovalova Elena , Frants Inessa , Palivoda Liubov , Usifli Seymour , Balakirev Vladimir
Published: April 2023

This paper provides a general overview of the development of program evaluation in CIS countries. We start by telling a story that describes how evaluation appeared in the scene, how it developed and who the key players were in its development. We discuss the issue of demand for and supply of evaluation services. In the conclusion, we describe our view of the stages in the emergence of evaluation in the CIS and the perspective of each. The four stages we identify are: “An External Phenomenon” (1991–1995), “Initiation” (1996–2000), “Assimilation” (2000–2004), and “Implementation” (2005 to the present).
Review of Evaluation and Program Planning: Volume 30, Issue 1
Authors: Watts Brad
Published: April 2023

The latest issue of Evaluation and Program Planning offers eleven general-interest articles covering a range of topics related to professional and academic evaluation activities. Unlike some previous issues that have focused on a special topic or interest area, this quarter’s publication offers articles from a wide range of subject areas and disciplines. As such, the following review provides a brief overview of each article, in publication order, instead of providing a more in-depth review of the work of select authors.
Addendum to "A Review of the History and the Current Practice of Aid Evaluation"
Authors: Sasaki Ryoh
Published: April 2023

In the article in the previous issue of JMDE, September 2006, “A Review of the History and the Current Practice of Aid Evaluation,” I stated “the EU does not have a unified evaluation approach, because a different EU member agency takes the lead in each evaluation project.” At the time when I accessed the webpage of Europe Aid Co-operation Office (AIDCO), the guidelines of evaluation were not available. The Webpage simply stated “the methodology pages will be available in English, French and Spanish and will be accessible early 2006.” It can be found at: http://ec.europa.eu/europeaid/evaluation/methodology)
A Feasibility Study of Microfinance for Chinese Disabled Farmers
Authors: Pan Luo Laura , Lin Liu
Published: April 2023

The Chinese government supports development aid to persons with disabilities (PWDs) in the form of microfinance. The nation’s Poverty Alleviation and Development Plan for PWDs (2001-2010) and the Eleventh Five-Year Plan for Rural Disabled Persons (2006-2010) spell providing quality rehabilitation out that the main methods to help poor rural disabled persons include significantly increasing the amount of funds for their use as well as ion services for them. The local government at every level has the responsibility to assist disabled persons.
Comparison Group Identification for Difficult-to-Evaluate Populations: Lessons from Evaluating 1n10 LGBTQ Youth Support Services
Authors: Friesner Donald E. , Peck Laura R.
Published: April 2023

The evaluation literature on Lesbian, Gay, Bisexual, Transgender, and Queer (LGBTQ) youth programs is largely absent of quantitative studies, likely in part due to the challenge of using experimental or quasi-experimental evaluation designs. This paper proposes the creative use of a national data set to overcome the problem of estimating a counterfactual for this population. In addition to discussing lessons from this approach, we describe the program and its impacts. Evidence suggests that the program under study—1n10, a local support group for LGBTQ youth in the Phoenix, AZ, metropolitan area—draws a relatively more disadvantaged group than the national average but that their levels of suicide ideation are lower. By addressing key threats to the internal validity of evaluation designs, we rule out plausible rival explanations for program impacts.
“Predictive Evaluation”: Some Comments on Michael Scriven’s Article in the June 2007 Issue of JMDE
Authors: Eggers Hellmut
Published: April 2023

Response to Michael Scriven's "Predictive Evaluation" from Journal of MultiDisciplinary Evaluation, 4(7), https://doi.org/10.56645/jmde.v4i7.26
Why Does Running an Electronic Journal Need Any Money?
Authors: Scriven Michael
Published: April 2023

Comment on the following quotation from the latest issue (9/07) of a fairly new (1995) journal, Sociological Research Online published in the UK.
The Concept of a Transdiscipline: And of Evaluation as a Transdiscipline
Authors: Scriven Michael
Published: April 2023

There are two main meanings of this term that can be disinterred from the 8,000+ references to their definition in Google (at December 12, 2007), both of them sharing the idea of a discipline that crosses over the boundaries between many other disciplines: we can distinguish them as the ‘point of view’ sense of the term and the ‘method’ sense of the term. These senses are not sharply distinct from each other, but are distinct from ‘interdiscipline’ meaning a compound approach drawing from or working in the boundary area between two or sometimes more other disciplines; and from ‘multidiscipline’ meaning a compound approach involving more than two other disciplines.
Advocacy Impact Evaluation
Authors: Patton Michael Q.
Published: April 2023

Advocacy and policy change evaluations focus on policy as the unit of analysis rather than the more traditional program or project. Advocacy evaluation, like all evaluation, is guided by the profession’s Principles and Standards. Advocacy evaluation can be, and should be, utilization-focused. That means focusing the evaluation on intended use by intended users, and evaluating the evaluation by that standard.
Analysis of Paired Dichotomous Data: A Gentle Introduction to the McNemar Test in SPSS
Authors: Adedokun Omolola A. , Burgess Wilella D.
Published: April 2023

Background: Although McNemar Test is the most appropriate tool for analyzing pre-post differences in dichotomous items (e.g., “yes” or “no”, “correct” or “incorrect”, etc.), many scholars have noted the inappropriate use of Pearson’s Chi-square Test by researchers, including social scientists and evaluators, for the analysis of related or dependent dichotomous variables.   Purpose: The goal of this paper is to promote the use of McNemar Test among evaluators by providing a gentle introduction to the method. Setting: Not applicable.   Intervention: Not applicable.   Research Design: Not applicable.   Data Collection and Analysis: Using data from 506 6th grade students’ responses to a pre-post science test; this contribution illustrates how to conduct McNemar Test in SPSS..   Findings: This contribution provides a non-technical introduction to McNemar test and illustrates its use in an applied research/evaluation context.   Keywords: McNemar Test; paired dichotomous data; Chi-square test; equality of proportions.
‘Policing Schools’ Strategies: A Review of the Evaluation Evidence
Authors: Petrosino Anthony , Guckenburg Sarah , Fronius Trevor
Published: April 2023

Background: Schools experience a wide range of crime and disorder, victimizing students and staff, and undermining attempts to create a safe and orderly environment for student learning. Police have long established programs with schools, but there has been no systematic review of evaluations of these programs, outside of police-led prevention classroom curriculum programs such as D.A.R.E.     Purpose: This paper documents a systematic search to identify experimental and quasi-experimental evaluations that assess the effectiveness of non-educational policing strategies and programs in schools.     Setting: Included studies took place in or around K-12 schools in the United States, Canada, and the United Kingdom.   Intervention: Studies were included if they reported on a specific school-based strategy that heavily involved police and did not exclusively involve the police teaching a curriculum or program such as Drug Abuse Resistance Education (D.A.R.E.).   Research Design:  Systematic review of experimental or quasi-experimental evaluations   Data Collection and Analysis: Only those impact studies that used experimental or quasi-experimental design, had at least one outcome measure of school crime or disorder, and were available through December 2009 were eligible.  Electronic searches and other methods were used to identify published and unpublished evaluation reports.       Findings: The searches identified a total of eleven quasi-experimental studies. Ten of the eleven studies would likely have received a “3” on the Maryland Scientific Methods Rating Scale, a common approach to classifying studies on the basis of internal validity.  If evidence rating criteria from the U.S. Department of Education’s What Works Clearinghouse (WWC) were applied, only one study would likely receive a grade of “Level 2” evidence (acceptable with reservations) and the other ten studies would likely not meet WWC evidence screening criteria.   Keywords: systematic review; police and schools; crime prevention
Methodological Pathways to Improvements of Evaluation Approaches: The Case of Irrigated Agriculture Performance Evaluation
Authors: Chaponnière Anne , Marlet Serge , Perret Sylvain R. , Bouleau Gabrielle , Zaïri Abdelaziz
Published: April 2023

Background:  Irrigated agriculture is often evaluated but few reviews of evaluation methodology adapted to this object are available in the literature. Besides, recommendations to improve evaluation in this field are lacking.   Purpose: The purpose of the paper is to contribute filling this gap.   Setting: Not applicable.   Intervention: Not applicable. Research Design: Not applicable.   Data Collection and Analysis: Desk review.   Findings: This review shows the evolution of evaluation methodology in the field of irrigated agriculture pointing out a trend towards more comprehensive methodologies. The review also suggests some methodological tools to improve evaluation process. Keywords: irrigated agriculture; evaluation methodology; plurality of evaluation; intended users; viewpoint
Utilizing Generalizability Theory to Investigate the Reliability of Grades Assigned to Undergraduate Research Papers
Authors: Gugiu Mihaiela R. , Gugiu Paul C. , Baldus Robert
Published: April 2023

Background:  Educational researchers have long espoused the virtues of writing with regard to student cognitive skills. However, research on the reliability of the grades assigned to written papers reveals a high degree of contradiction, with some researchers concluding that the grades assigned are very reliable whereas others suggesting that they are so unreliable that random assignment of grades would have been almost as helpful.   Purpose: The primary purpose of the study was to investigate the reliability of grades assigned to written reports. The secondary purpose was to illustrate the use of Generalizability Theory, specifically the fully-crossed two-facet model, for computing interrater reliability coefficients.   Setting: The participants for this study were 29 undergraduate students enrolled in an introductory-level course on Political Behavior in Spring 2011 at a Midwest university.   Intervention: Not applicable. Research Design: Students were randomly assigned to one of nine groups. Two-facet fully crossed G-study and D-study designs were used wherein two raters graded four assignments for 9 student groups—72 evaluations in total. The universe of admissible observations was deemed to be random for both raters and assignments, whereas the universe of generalization was deemed to be mixed (random for two raters but fixed for four assignments).   Data Collection and Analysis: The semester-long project was assigned to groups consisting of an annotated bibliography, survey development, sampling design, and analysis and final report. Four grading rubrics were developed and utilized to evaluate the quality of each written report. Two-facet generalizability analyses were conducted to assess interrater reliability using software developed by one of the authors.   Findings: This study found a very high interrater reliability coefficient (0.929) for only two raters who received no training in how to use the four grading rubrics. Keywords: grading; reliability; Generalizability Theory; writing
What’s in a Scriven Number?
Authors: Roy Abhik R. , Hobson Kristin A. , Coryn Chris L. S.
Published: April 2023

Background: Other than Leonhard Euler, one of the greatest physicists and mathematicians ever to have lived and dubbed “…c'est notre maître à tous” or “…the master of us all” by the French mathematician and astronomer Pierre-Simon Laplace, Paul Erdös was a mathematician who published more papers in mathematics than anyone else (Hoffman, 1998). Because of this feat, friends and colleagues created what has been known as an Erdös number. An Erdös number describes a person’s degree of separation from Erdös through collaborations directly with him or with others who have collaborated with him.   Purpose: In this paper, we present reasoning similar to that of an Erdös number for Michael Scriven who is widely considered to be one of the leading theorists in evaluation. With his numerous publications and influence on the theory and practice of evaluation, we present not only the hope, but also the need for a Scriven number. Setting: Not applicable.   Intervention: Scriven number.   Research Design: Not applicable.   Data Collection and Analysis: Not applicable.   Findings: A collaborative count such as a Scriven number would centralize evaluation. A Scriven number would form new connections and collaborations, thus yielding a robust connectivity. Keywords: Scriven number
Ciberbullying a través de WhatsApp en jóvenes migrantes de la frontera norte de Ecuador: análisis de sus formas y consecuencias
Authors: Moreno Gudiño Bryan Patricio , Ruiz Onofre Diana
Published: April 2023

Esta investigación tiene como objetivo profundizar en el ciberbullying y su práctica contra jóvenes migrantes de entre 15 y 19 años en Imbabura y Carchi, provincias de la Sierra Norte del Ecuador. Un diseño metodológico sencillo pero preciso, permite detectar y caracterizar detalladamente aquellas representaciones del ciberacoso, por medio de una aproximación a muestras homogéneas por oportunidad, en correspondencia con el grupo de estudio definido. La aplicación de entrevistas semiestructuradas en profundidad recopila los testimonios de jóvenes colombianos y venezolanos como víctimas directas o indirectas de ciberacoso, específicamente a través de los formatos soportados por el sistema de mensajería instantánea WhatsApp.
Importancia de las intervenciones educativas y gerontológicas en cuidadores primarios de pacientes geriátricos
Authors: Chávez de la Luz Haidee Aislinn , Méndez Salazar Vianey , Carpio Rodríguez María de los Ángeles
Published: April 2023

Las intervenciones educativas y gerontológicas en cuidadores primarios de pacientes geriátricos representan una oportunidad para mejorar diversas actividades cotidianas por parte del cuidador. Objetivo: Demostrar la disminución de la sobrecarga en el cuidador a través de la aplicación de intervenciones gerontológicas sociales y educativas efectuadas en cuidadores primarios de pacientes geriátricos, evaluando dimensiones como calidad de vida, capacidad de autocuidado, redes de apoyo social y competencias para afrontar problemas conductuales y clínicos del paciente a través de instrumentos de valoración. Se trata de un estudio cuantitativo de tipo exploratorio y transversal, el muestreo fue de tipo no probabilístico, para la captura de los datos y validación de la información se utilizó el programa estadístico de cómputo SPSS versión 20, se empleó la estadística descriptiva para el análisis de los datos, utilizando de medidas de tendencia central, de los resultados obtenidos se observó que, de un total de 36 cuidadores primarios, el 79.2% presentó sobrecarga y el 64.8% reportó una disminución de la misma, debido a la aplicación de intervenciones educativas y gerontológicas logrando asimismo la disminución del estrés, ansiedad y depresión.
Pengaruh penempatan terhadap kinerja pegawai melalui komitmen organisasional di Pondok Pesantren
Authors: Mastur Mastur
Published: April 2023

Islamic boarding schools as educational institutions that are very much needed by the community need to be maintained for their sustainability. The sustainability of the pesantren is determined by how well the existing employees are able to perform. This article examines how to improve performance in Islamic boarding schools, resulting in improved employee performance. Using PLS analysis software with saturated sampling method. Forging employees has a positive and insignificant effect on performance, meaning that the placement process must be adjusted to the competence of employees, so that it will give birth to good performance. Employee placement has a positive and significant effect on organizational commitment, this means that the placement will increase organizational commitment. Organizational commitment has a positive and significant effect on employee performance, the existence of organizational commitment will result in increased employee performance. Organizational commitment is capable of mediating between the placement of employees on employee performance, the placement will give birth to organizational commitment and will have implications for increasing employee performance.
Efek lingkungan kerja dan stres kerja terhadap kinerja karyawan melalui kepuasan kerja sebagai variabel intervening
Authors: Fauji Ade , Damaihati Salsabila
Published: April 2023

One of the factors that influence the level of success of an organization is employee performance. This research was made to find out how the effect of work environment and work stress on employee performance through job satisfaction as an intervening variable at PT. Cilegon Fabricators. The research method used is correlational quantitative and the sample technique used is purposive sampling with 54 respondents. Data were obtained by distributing questionnaires and processing data using SmartPLS version 3.3.3. In this research it can be concluded that the work environment and work stress have an effect on employee performance, job satisfaction has no effect on employee performance, the work environment has no effect on job satisfaction, job stress has an effect on job satisfaction, and job satisfaction is not able to be an intervening variable in an unrelated relationship direct work environment and work stress on employee performance.
Pengaruh kompensasi, disiplin kerja dan motivasi kerja terhadap kinerja karyawan
Authors: Sari Indah Silvia , Lisnawati Lilis , Paramita Mellysa Padma
Published: April 2023

The success achieved by Bank BCA cannot be separated from the important role of HR. Therefore, the Bank is committed to developing a highly competent workforce that has the agility, mindset to continue to grow, and adheres to BCA's values to run the business amidst dynamic industrial developments. The concept of Branch Service Quality (BSQ) is used by Bank BCA to assess employee performance based on respondents' perceptions which is done independently. This is done by the bank as an effort to maintain the quality of service which is currently good, where this is done by evaluating employee performance based on customer perceptions. The number of respondents in this study were 42 employees of Bank BCA KCU Kedoya Permai. This research is quantitative by using analytical methods with the SPSS program. The results of this study are that there is an influence between compensation, work discipline and work motivation on employee performance partially and simultaneously.
EMOTIONS
Authors: LEBEAU J-P
Published: April 2023

Les émotions ne sont pas de simples conséquences du traitement que nos personnalités proposent des informations sensorielles qui arrivent à nos cortex. Elles ne se contentent pas de moduler, avec plus ou moins de subtilité, les réactions que nous attribuons à notre « caractère ». Elles nous définissent. À tel point que nous en arrivons à classer les personnes que nous rencontrons non pas en fonction de leurs caractéristiques propres, mais en fonction des émotions que nous-mêmes ressentons face à elles. Et cette façon de considérer l’autre n’est pas réservée à la sphère privée – comment pourrait-elle l’être ? – elle intervient de façon parfois cruciale dans la relation médecin-patient. Qu’est-ce donc qu’un « patient difficile », sinon un patient qui génère en nous des émotions que nous ne souhaiterions pas avoir à gérer dans le cadre de notre exercice ? Il est d’autant plus surprenant qu’alors que nous recherchons sans cesse, dans les arts, les divertissements et les multiples modalités des relations interhumaines, à créer et à ressentir des émotions, qu’elles aient pu faire l’objet, dans la formation médicale « classique », d’une censure quasi absolue posée comme un principe : il faut se blinder ! Nous savons tous maintenant à quel point cette subtilité toute militaire, qui ambitionnait de transformer l’étudiant en soins et sa vocation en char d’assaut, trouve ses limites face à des patients difficilement assimilables à des ennemis à exterminer… La complexité des ressentis de la relation thérapeutique ne peut à l’évidence se contenter de réponses simples et toutes faites, d’autant que ces réponses voudraient nous protéger de ce qui viendrait de l’extérieur (le « blindage »…) alors qu’il nous faut apprendre à reconnaître et à vivre avec ce qui vient de nous-mêmes, ce qui est nous-mêmes. Les chercheurs québécois, spécialistes de la communication professionnelle en santé, nous montrent que c’est par la maîtrise du langage que nos émotions peuvent devenir des outils1. Bouchet et al., de leur côté, nous suggèrent qu’il reste beaucoup à inventer et à faire dans ce domaine dans la formation initiale à la médecine générale2. Faire face à ces questionnements complexes oblige à une exploration minutieuse des éléments de la complexité relationnelle, oblige à la maîtrise de méthodes pertinentes pour des recherches conduites dans le cadre spécifique des soins premiers. Ce n’est par hasard, par facilité ou par opportunisme que la médecine générale a fait siennes les méthodes de recherche des sciences humaines et sociales. La médecine générale est une science humaine et sociale.
Expanding Literacy Horizons: Inclusive, Diverse, and Innovative Approaches
Authors: Scullin Bethany , Griffin Robert
Published: April 2023

The spring 2023 issue of the Georgia Journal of Literacy, “Expanding Literacy Horizons: Inclusive, Diverse, and Innovative Approaches,” aims to inspire and inform K–12 literacy educators, researchers, and stakeholders by offering valuable insights, research, and resources. The issue emphasizes collaboration, creativity, and critical reflection and is divided into sections by article type, including a literature review on inclusive literacy ideologies; practitioner articles exploring interdisciplinary approaches, reading aloud, and the use of cluster texts to celebrate diversity in education; and teaching tips offering guidance on organizing author visits, fostering classroom community, and transitioning to digital interactive journals. The issue encourages readers to reflect on how inclusive, diverse, and innovative approaches can be integrated into their practices to create more equitable, dynamic, and engaging learning environments.
2021 CLINICAL PRACTICE GUIDELINES IN THE EVALUATION AND MANAGEMENT OF PEDIATRIC COMMUNITY-ACQUIRED PNEUMONIA
Authors: Jalandoni-Cabahug Ma. Vina , Capeding Maria Rosario , Alvarado-Dela Cruz Kristine , Castellano Mark Joseph , De Leon Maria Nerissa , Padua Jay Ron
Published: April 2023

The Clinical Practice Guidelines (CPG) for the Diagnosis and Management of Pediatric Community-Acquired Pneumonia (PCAP) was initiated by the Philippine Academy of Pediatric Pulmonologists, Inc. (PAPP) and the Pediatric Infectious Disease Society of the Philippines (PIDSP), in cooperation with Philippine Pediatric Society, Inc. (PPS) way back in 2004. Several CPG updates were then undertaken by the PAPP PCAP CPG Task Force from 2008 to 2016. Clinically-relevant research questions were answered with recent and current recommendations based on evidence from local and international data. The 2021 PCAP CPG initiative was envisioned in March 2018 upon the recommendations of the 2018 PAPP Board for the purpose of updating the evidence in the PCAP CPG 2016 clinical questions. This led to the collaboration of PAPP and PIDSP to develop this CPG. Individual members were identified from each society as content experts to form the Steering Committee along with a clinical epidemiologist and technical writer as review experts. The committee identified the scope and target end user of the CPG as well as additional clinical questions to be included in the 2021 update aside from the questions on the previous CPGs. Selected members from the two societies formed the Technical Working Group (TWG) who did the literature search, appraisal of evidences, and formulation of recommendations. These recommendations were then presented to the stakeholders who became part of the consensus panel. There was no identified conflict of interest among the CPG developers, TWG members and stakeholders. A survey to determine potential competing interests were conducted during the development of this CPG. This initiative was fully funded by the PAPP and PIDSP societies. The 2021 PCAP CPG significantly differs from the previous CPGs in several aspects. First, the current guideline is a consensus between two pediatric societies. Second, much of the literature review has been centered on meta-analyses or systematic reviews instead of individual studies. Finally, appraisal of published literature was based on Grading of Recommendations, Assessment, Development and Evaluation (GRADE) criteria. Such methodological differences may provide difficulties in defining evolution of care through the years.