bluuebunny/crossref_metadata_2025 huggingface dataset:
Optimizing Visual Properties of Game Content Through Neuroevolution
Authors: Liapis Antonios , Yannakakis Georgios , Togelius Julian
Published: October 2022

This paper presents a search-based approach to generating game content that satisfies both gameplay requirements and user-expressed aesthetic criteria. Using evolutionary constraint satisfaction, we search for spaceships (for a space combat game) represented as compositional pattern-producing networks. While the gameplay requirements are satisfied by ad-hoc defined constraints, the aesthetic evaluation function can also be informed by human aesthetic judgement. This is achieved using indirect interactive evolution, where an evaluation function re-weights an array of aesthetic criteria based on the choices of a human player. Early results show that we can create aesthetically diverse and interesting spaceships while retaining in-game functionality.
Robust and Authorable Multiplayer Storytelling Experiences
Authors: Riedl Mark , Li Boyang , Ai Hua , Ram Ashwin
Published: October 2022

Interactive narrative systems attempt to tell stories to players capable of changing the direction and/or outcome of the story. Despite the growing importance of multiplayer social experiences in games, little research has focused on multiplayer interactive narrative experiences. We performed a preliminary study to determine how human directors design and execute multiplayer interactive story experiences in online and real world environments. Based on our observations, we developed the Multiplayer Storytelling Engine that manages a story world at the individual and group levels. Our flexible story representation enables human authors to naturally model multiplayer narrative experiences. An intelligent execution algorithm detects when the author's story representation fails to account for player behaviors and automatically generates a branch to restore the story to the authors' original intent, thus balancing authorability against robust multiplayer execution.
An Object-Oriented Approach to Reinforcement Learning in an Action Game
Authors: Mohan Shiwali , Laird John
Published: October 2022

In this work, we look at the challenge of learning in an action game,Infinite Mario. Learning to play an action game can be divided intotwo distinct but related problems, learning an object-relatedbehavior and selecting a primitive action. We propose a framework that allows for the use of reinforcement learning for both ofthese problems. We present promising results in some instances of thegame and identify some problems that might affect learning.
Comme il Faut: A System for Authoring Playable Social Models
Authors: McCoy Joshua , Treanor Mike , Samuel Ben , Wardrip-Fruin Noah , Mateas Michael
Published: October 2022

Authoring interactive stories where the player is afforded a wide range of social interactions results in a very large space of possible social and story situations. The amount of effort required to individually author for each of these circumstances can quickly become intractable. The social AI system Comme il Faut (CiF) aims to reduce the burden on the author by providing a playable model of social interaction where the author provides reusable and recombinable representations of social norms and social interactions. Motivated through examples from an in-development video game, Prom Week, this paper provides a detailed description of the structures with which CiF represents social knowledge and how this knowledge is employed to simulate social interactions between characters.
Real-Time Adaptive A∗ with Depression Avoidance
Authors: Hernandez Carlos , Baier Jorge
Published: October 2022

RTAA* is probably the best-performing real-time heuristic search algorithm at path-finding tasks in which the environ- ment is not known in advance or in which the environment is known and there is no time for pre-processing. As most real- time search algorithms do, RTAA∗ performs poorly in presence of heuristic depressions, which are bounded areas of the search space in which the heuristic is too low with respect to their border. Recently, it has been shown that LSS-LRTA∗, a well-known real-time search algorithm, can be improved when search is actively guided away of depressions. In this paper we investigate whether or not RTAA∗ can be improved in the same manner. We propose aRTAA∗ and daRTAA∗, two algorithms based on RTAA∗ that avoid heuristic depressions. Both algorithms outperform RTAA∗ on standard path-finding tasks, obtaining better-quality solutions when the same time deadline is imposed on the duration of the planning episode. We prove, in addition, that both algorithms have good theoretical properties
A Discrete Event Calculus Implementation of the OCC Theory of Emotion
Authors: Sarlej Margaret , Ryan Malcolm
Published: October 2022

Characters are a critical part of storytelling and emotion is a vital part of character. Readers generally credit characters with human emotions, and it is these emotions which bring meaning to stories. To computationally construct interesting and meaningful stories we need a model of emotion which allows us to predict characters’ reactions to events in the world. There are many different psychological theories of emotion; the most popular to date for computational applications is the OCC theory. This paper describes a Discrete Event Calculus implementation of the OCC Theory of Emotion. To evaluate our system, we apply it to a selection of Aesop’s fables, and compare the output to the emotions readers expect in the same situations based on a survey.
Representing Morals in Terms of Emotion
Authors: Sarlej Margaret , Ryan Malcolm
Published: October 2022

Morals are an important part of many stories, and central to why storytelling developed in the first place as a means of communication. They have the potential to provide a framework for developing story structure, which could be utilised by modern storytelling systems. To achieve this we need a general representation for morals. We propose patterns of character emotion as a suitable foundation. In this paper, we categorise Aesop’s fables based on the morals they convey, and use them as a source of emotion data corresponding to those morals. We use inductive logic programming to identify relationships between particular patterns of emotion and the morals of the stories in which they arise.
A Temporal Data-Driven Player Model for Dynamic Difficulty Adjustment
Authors: Zook Alexander , Riedl Mark
Published: October 2022

Many computer games of all genres pit the player against a succession of increasingly difficult challenges such as combat with computer-controlled enemies and puzzles. Part of the fun of computer games is to master the skills necessary to complete the game. Challenge tailoring is the problem of matching the difficulty of skill-based events over the course of a game to a specific player's abilities. We present a tensor factorization approach to predicting player performance in skill-based computer games. Our tensor factorization approach is data-driven and can predict changes in players' skill mastery over time, allowing more accurate tailoring of challenges. We demonstrate the efficacy and scalability of tensor factorization models through an empirical study of human players in a simple role-playing combat game. We further find a significant correlation between these performance ratings and player subjective experiences of difficulty and discuss ways our model can be used to optimize player enjoyment.
Plan-Based Character Diversity
Authors: Coman Alexandra , Munoz-Avila Hector
Published: October 2022

Non-player character diversity enriches game environments increasing their replay value. We propose a method for obtaining character behavior diversity based on the diversity of plans enacted by characters, and demonstrate this method in a scenario in which characters have multiple choices. Using case-based planning techniques, we reuse plans for varied character behavior, which simulate different personality traits.
Player Profiling with Fallout 3
Authors: Spronck Pieter , Balemans Iris , Van Lankveld Giel
Published: October 2022

In previous research we concluded that a personality profile, based on the Five Factor Model, can be constructed from observations of a player’s behavior in a module that we designed for Neverwinter Nights (Lankveld et al. 2011a). In the present research, we investigate whether we can do the same thing in an actual modern commercial video game, in this case the game Fallout 3. We stored automatic observations on 36 participants who played the introductory stages of Fallout 3. We then correlated these observations with the participants’ personality profiles, expressed by values for five personality traits as measured by the standard NEO-FFI questionnaire. Our analysis shows correlations between all five personality traits and the game observations. These results validate and generalize the results from our previous research (Lankveld et al. 2011a). We may conclude that Fallout 3, and by extension other modern video games, allows players to express their personality, and can therefore be used to create personality profiles.
Automatic Orchestration for Automatic Composition
Authors: Handelman Eliot , Sigler Andie , Donna David
Published: October 2022

The automatic orchestration problem is that of assigning instruments or sounds to the notes of an unorchestrated score. This is related to, but distinct from, problems of automatic expressive interpretation. A simple algorithm is described that successfully orchestrates scores based on analysis of one musical structure -- the "Z-chain."
The Combinatorial Multi-Armed Bandit Problem and Its Application to Real-Time Strategy Games
Authors: Ontanon Santiago
Published: October 2022

Game tree search in games with large branching factors is a notoriously hard problem. In this paper, we address this problem with a new sampling strategy for Monte Carlo Tree Search (MCTS) algorithms, called "Naive Sampling", based on a variant of the Multi-armed Bandit problem called the "Combinatorial Multi-armed Bandit" (CMAB) problem. We present a new MCTS algorithm based on Naive Sampling called NaiveMCTS, and evaluate it in the context of real-time strategy (RTS) games. Our results show that as the branching factor grows, NaiveMCTS performs significantly better than other algorithms.
Performance of Paid and Volunteer Image Labeling in Citizen Science — A Retrospective Analysis
Authors: Gandhi Kutub , Spatharioti Sofia Eleni , Eustis Scott , Wylie Sara , Cooper Seth
Published: October 2022

Citizen science projects that rely on human computation can attempt to solicit volunteers or use paid microwork platforms such as Amazon Mechanical Turk. To better understand these approaches, this paper analyzes crowdsourced image label data sourced from an environmental justice project looking at wetland loss off the coast of Louisiana. This retrospective analysis identifies key differences between the two populations: while Mechanical Turk workers are accessible, cost-efficient, and rate more images than volunteers (on average), their labels are of lower quality, whereas volunteers can achieve high accuracy with comparably few votes. Volunteer organizations can also interface with the educational or outreach goals of an organization in ways that the limited context of microwork prevents.
SignUpCrowd: Using Sign-Language as an Input Modality for Microtask Crowdsourcing
Authors: Singh Aayush , Wehkamp Sebastian , Gadiraju Ujwal
Published: October 2022

Different input modalities have been proposed and employed in technological landscapes like microtask crowdsourcing. However, sign language remains an input modality that has received little attention. Despite the fact that thousands of people around the world primarily use sign language, very little has been done to include them in such technological landscapes. We aim to address this gap and take a step towards the inclusion of deaf and mute people in microtask crowdsourcing. We first identify various microtasks which can be adapted to use sign language as input, while elucidating the challenges it introduces. We built a system called ‘SignUpCrowd’ that can be used to support sign language input for microtask crowdsourcing. We carried out a between-subjects study (N=240) to understand the effectiveness of sign language as an input modality for microtask crowdsourcing in comparison to prevalent textual and click input modalities. We explored this through the lens of visual question answering and sentiment analysis tasks by recruiting workers from the Prolific crowdsourcing platform. Our results indicate that sign language as an input modality in microtask crowdsourcing is comparable to the prevalent standards of using text and click input. Although people with no knowledge of sign language found it difficult to use, this input modality has the potential to broaden participation in crowd work. We highlight evidence suggesting the scope for sign language as a viable input type for microtask crowdsourcing. Our findings pave the way for further research to introduce sign language in real-world applications and create an inclusive technological landscape that more people can benefit from.
DataSift: An Expressive and Accurate Crowd-Powered Search Toolkit
Authors: Parameswaran Aditya , Teh Ming Han , Garcia-Molina Hector , Widom Jennifer
Published: October 2022

Traditional information retrieval systems have limited functionality. For instance, they are not able to adequately support queries containing non-textual fragments such as images or videos, queries that are very long or ambiguous, or semantically-rich queries over non-textual corpora. In this paper, we present DataSift, an expressive and accurate crowd-powered search toolkit that can connect to any corpus. We provide a number of alternative configurations for DataSift using crowdsourced and automated components, and demonstrate gains of 2–3x on precision over traditional retrieval schemes using experiments on real corpora. We also present our results on determining suitable values for parameters in those configurations, along with a number of interesting insights learned along the way.
The Crowd-Median Algorithm
Authors: Heikinheimo Hannes , Ukkonen Antti
Published: October 2022

The power of human computation is founded on the capabilities of humans to process qualitative information in a manner that is hard to reproduce with a computer. However, all machine learning algorithms rely on mathematical operations, such as sums, averages, least squares etc. that are less suitable for human computation. This paper is an effort to combine these two aspects of data processing. We consider the problem of computing a centroid of a data set, a key component in many data-analysis applications such as clustering, using a very simple human intelligence task (HIT). In this task the workers must choose the outlier from a set of three items. After presenting a number of such triplets to the workers, the item chosen the least number of times as the outlier is selected as the centroid. We provide a proof that the centroid determined by this procedure is equal the mean of a univariate normal distribution. Furthermore, as a demonstration of the viability of our method, we implement a human computation based variant of the k-means clustering algorithm. We present experiments where the proposed method is used to find an "average" image in a collection, and cluster images to semantic categories.
Crowdsourcing a HIT: Measuring Workers' Pre-Task Interactions on Microtask Markets
Authors: Jacques Jason , Kristensson Per Ola
Published: October 2022

The ability to entice and engage crowd workers to participate in human intelligence tasks (HITs) is critical for many human computation systems and large-scale experiments. While various metrics have been devised to measure and improve the quality of worker output via task designs, effective recruitment of crowd workers is often overlooked. To help us gain a better understanding of crowd recruitment strategies we propose three new metrics for measuring crowd workers' willingness to participate in advertised HITs: conversion rate, conversion rate over time, and nominal conversion rate. We discuss how the conversion rate of workers—the number of potential workers aware of a task that choose to accept the task—can affect the quantity, quality, and validity of any data collected via crowdsourcing. We also contribute a tool — turkmill — that enables requesters on Amazon Mechanical Turk to easily measure the conversion rate of HITs. We then present the results of two experiments that demonstrate how conversion rate metrics can be used to evaluate the effect of different HIT designs. We investigate how four HIT design features (value proposition, branding, quality of presentation, and intrinsic motivation) affect conversion rates. Among other things, we find that including a clear value proposition has a strong significant, positive effect on the nominal conversion rate. We also find that crowd workers prefer commercial entities to non-profit or university requesters.
A Human-Centered Framework for Ensuring Reliability on Crowdsourced Labeling Tasks
Authors: Alonso Omar , Marshall Catherine , Najork Marc
Published: October 2022

This paper describes an approach to improving the reliability of a crowdsourced labeling task for which there is no objective right answer. Our approach focuses on three contingent elements of the labeling task: data quality, worker reliability, and task design. We describe how we developed and applied this framework to the task of labeling tweets according to their interestingness. We use in-task CAPTCHAs to identify unreliable workers, and measure inter-rater agreement to decide whether subtasks have objective or merely subjective answers.
Incentives to Counter Bias in Human Computation
Authors: Faltings Boi , Jurca Radu , Pu Pearl , Tran Bao Duy
Published: October 2022

In online labor platforms such as Amazon Mechanical Turk, a good strategy to obtain quality answers is to take aggregate answers submitted by multiple workers, exploiting the wisdom of crowds. However, human computation issusceptible to systematic biases which cannot be corrected by using multiple workers.We investigate a game-theoretic bonus scheme, called peer truth serum (PTS), to overcome this problem. We report on the design and outcomes of a set of experiments to validate this scheme. Results show  peer truth serum can indeed correct the biases and increase the answer accuracy by up to 80%.
Groupsourcing: Distributed Problem Solving Using Social Networks
Authors: Chamberlain Jon
Published: October 2022

Crowdsourcing and citizen science have established themselves in the mainstream of research methodology in recent years, employing a variety of methods to solve problems using human computation. An approach described here, termed "groupsourcing", uses social networks to present problems and collect solutions. This paper details a method for archiving social network messages and investigates messages containing an image classification task in the domain of marine biology. In comparison to other methods, groupsourcing offers a high accuracy, data-driven and low cost approach.
Tuning the Diversity of Open-Ended Responses From the Crowd
Authors: Lasecki Walter , Homan Christopher , Bigham Jeffrey
Published: October 2022

Crowdsourcing can solve problems beyond the reach of state-of-the-art fully automated systems. A common pattern found in many such systems is for the workers to discover, in parallel, a number of candidate solutions and then vote on the best one to pass forward, often within a fixed amount of time. We present the propose-vote-abstain mechanism for eliciting from crowd workers the proper balance between solution discovery and selection. Each crowd worker is given a choice among proposing an answer, voting among the answers proposed so far, or abstaining, i.e., doing nothing. When a stopping condition is reached, the mechanism returns the answer with the most votes. Workers are paid a base amount, with bonuses if they propose or vote for the winning answer.
Crowd Access Path Optimization: Diversity Matters
Authors: Nushi Besmira , Singla Adish , Gruenheid Anja , Zamanian Erfan , Krause Andreas , Kossmann Donald
Published: October 2022

Quality assurance is one the most important challenges in crowdsourcing. Assigning tasks to several workers to increase quality through redundant answers can be expensive if asking homogeneous sources. This limitation has been overlooked by current crowdsourcing platforms resulting therefore in costly solutions. In order to achieve desirable cost-quality tradeoffs it is essential to apply efficient crowd access optimization techniques. Our work argues that optimization needs to be aware of diversity and correlation of information within groups of individuals so that crowdsourcing redundancy can be adequately planned beforehand. Based on this intuitive idea, we introduce the Access Path Model (APM), a novel crowd model that leverages the notion of access paths as an alternative way of retrieving information. APM aggregates answers ensuring high quality and meaningful confidence. Moreover, we devise a greedy optimization algorithm for this model that finds a provably good approximate plan to access the crowd. We evaluate our approach on three crowdsourced datasets that illustrate various aspects of the problem. Our results show that the Access Path Model combined with greedy optimization is cost-efficient and practical to overcome common difficulties in large-scale crowdsourcing like data sparsity and anonymity.
Using Anonymity and Communal Efforts to Improve Quality of Crowdsourced Feedback
Authors: Hui Julie , Glenn Amos , Jue Rachel , Gerber Elizabeth , Dow Steven
Published: October 2022

Student entrepreneurs struggle to collect feedback on their product pitches in a classroom setting due to a lack of time, money, and access to motivated feedback providers. Online social networks present a unique opportunity for entrepreneurial students to quickly access feedback providers by leveraging their online social capital. In order to better understand how to improve crowdsourced online pitch feedback, we perform an experiment to test the effect of online anonymity on pitch feedback quality and quantity. We also test a communal feedback method—evenly distributing between teams feedback providers from the class’s collective online social networks—which would help all teams benefit from a useful amount of feedback rather than having some teams receive much more feedback than others. We found that feedback providers in the anonymous condition provided significantly more specific criticism and specific praise, which students rated as more useful. Furthermore, we found that the communal feedback method helped all teams receive sufficient feedback to edit their pitches. This research contributes an empirical investigation to the crowdsourcing community of how crowds through online social networks can help student entrepreneurs obtain authentic feedback to improve their work.
Publishable Humanly Usable Secure Password Creation Schemas
Authors: Blum Manuel , Vempala Santosh
Published: October 2022

What can a human compute in his/her head that a powerful adversary cannot infer? To answer this question, we define a model of human computation and a measure of security. Then, motivated by the special case of password creation, we propose a collection of well-defined password-generation methods. We show that our password generation methods are humanly computable and, to a well-defined extent, machine uncrackable. For the proof of security, we posit that password generation methods are public, but that the human’s privately chosen seed is not, and that the adversary will have observed only a few input-output pairs. Besides the application to password generation, our proposed Human Usability Model (HUM) will have other applications.
Surpassing Humans and Computers with JELLYBEAN: Crowd-Vision-Hybrid Counting Algorithms
Authors: Sarma Akash , Jain Ayush , Nandi Arnab , Parameswaran Aditya , Widom Jennifer
Published: October 2022

Counting objects is a fundamental image processisng primitive, and has many scientific, health, surveillance, security, and military applications. Existing supervised computer vision techniques typically require large quantities of labeled training data, and even with that, fail to return accurate results in all but the most stylized settings. Using vanilla crowdsourcing, on the other hand, can lead to significant errors, especially on images with many objects. In this paper, we present our JellyBean suite of algorithms, that combines the best of crowds and computer vision to count objects in images, and uses judicious decomposition of images to greatly improve accuracy at low cost. Our algorithms have several desirable properties: (i) they are theoretically optimal or near-optimal, in that they ask as few questions as possible to humans (under certain intuitively reasonable assumptions that we justify in our paper experimentally); (ii) they operate under stand-alone or hybrid modes, in that they can either work independent of computer vision algorithms, or work in concert with them, depending on whether the computer vision techniques are available or useful for the given setting; (iii) they perform very well in practice, returning accurate counts on images that no individual worker or computer vision algorithm can count correctly, while not incurring a high cost.
Click Carving: Segmenting Objects in Video with Point Clicks
Authors: Jain Suyog , Grauman Kristen
Published: October 2022

We present a novel form of interactive video object segmentation where a few clicks by the user helps the system produce a full spatio-temporal segmentation of the object of interest. Whereas conventional interactive pipelines take the user's initialization as a starting point, we show the value in the system taking the lead even in initialization. In particular, for a given video frame, the system precomputes a ranked list of thousands of possible segmentation hypotheses (also referred to as object region proposals) using image and motion cues. Then, the user looks at the top ranked proposals, and clicks on the object boundary to carve away erroneous ones. This process iterates (typically 2-3 times), and each time the system revises the top ranked proposal set, until the user is satisfied with a resulting segmentation mask. Finally, the mask is propagated across the video to produce a spatio-temporal object tube. On three challenging datasets, we provide extensive comparisons with both existing work and simpler alternative methods. In all, the proposed Click Carving approach strikes an excellent balance of accuracy and human effort. It outperforms all similarly fast methods, and is competitive or better than those requiring 2 to 12 times the effort.
Interactive Consensus Agreement Games for Labeling Images
Authors: Upchurch Paul , Sedra Daniel , Mullen Andrew , Hirsh Haym , Bala Kavita
Published: October 2022

Scene understanding algorithms in computer vision are improving dramatically by training deep convolutional neural networks on millions of accurately annotated images. Collecting large-scale datasets for this kind of training is challenging, and the learning algorithms are only as good as the data they train on. Training annotations are often obtained by taking the majority label from independent crowdsourced workers using platforms such as Amazon Mechanical Turk. However, the accuracy of the resulting annotations can vary, with the hardest-to-annotate samples having prohibitively low accuracy. Our insight is that in cases where independent worker annotations are poor more accurate results can be obtained by having workers collaborate. This paper introduces consensus agreement games, a novel method for assigning annotations to images by the agreement of multiple consensuses of small cliques of workers. We demonstrate that this approach reduces error by 37.8% on two different datasets at a cost of $0.10 or $0.17 per annotation. The higher cost is justified because our method does not need to be run on the entire dataset. Ultimately, our method enables us to more accurately annotate images and build more challenging training datasets for learning algorithms.
Confusing the Crowd: Task Instruction Quality on Amazon Mechanical Turk
Authors: Wu Meng-Han , Quinn Alexander
Published: October 2022

Task instruction quality is widely presumed to affect outcomes, such as accuracy, throughput, trust, and worker satisfaction. Best practices guides written by experienced requesters share their advice about how to craft task interfaces. However, there is little evidence of how specific task design attributes affect actual outcomes. This paper presents a set of studies that expose the relationship between three sets of measures: (a) workers’ perceptions of task quality, (b) adherence to popular best practices, and (c) actual outcomes when tasks are posted (including accuracy, throughput, trust, and worker satisfaction). These were investigated using collected task interfaces, along with a model task that we systematically mutated to test the effects of specific task design guidelines.
Skill-and-Stress-Aware Assignment of Crowd-Worker Groups to Task Streams
Authors: Kumai Katsumi , Matsubara Masaki , Shiraishi Yuhki , Wakatsuki Daisuke , Zhang Jianwei , Shionome Takeaki , Kitagawa Hiroyuki , Morishima Atsuyuki
Published: October 2022

Worker-task assignments represent one of the critical issues in crowdsourcing, as they affect the quality of task results. This study addresses the problem of forming worker groups assigned to the same task in a task stream that requires more than one worker. We introduce a worker-group queue model that covers practical and common scenarios for task-stream crowdsourcing, and compare three strategies in terms of the skill balance among worker groups, the quality of the final outputs, the number of worker re-assignments of workers, and psychological stress felt by workers. We found that one of the compared strategies that employs multiple worker queues yields good results based on these measures.
Active Learning with Unbalanced Classes and Example-Generation Queries
Authors: Lin Christopher , Mausam Mausam , Weld Daniel
Published: October 2022

Machine learning in real-world high-skew domains is difficult, because traditional strategies for crowdsourcing labeled training examples are ineffective at locating the scarce minority-class examples. For example, both random sampling and traditional active learning (which reduces to random sampling when just starting) will most likely recover very few minority-class examples. To bootstrap the machine learning process, researchers have proposed tasking the crowd with finding or generating minority-class examples, but such strategies have their weaknesses as well. They are unnecessarily expensive in well-balanced domains, and they often yield samples from a biased distribution that is unrepresentative of the one being learned.This paper extends the traditional active learning framework by investigating the problem of intelligently switching between various crowdsourcing strategies for obtaining labeled training examples in order to optimally train a classifier. We start by analyzing several such strategies (e.g., annotate an example, generate a minority-class example, etc.), and then develop a novel, skew-robust algorithm, called MB-CB, for the control problem. Experiments show that our method outperforms state-of-the-art GL-Hybrid by up to 14.3 points in F1 AUC, across various domains and class-frequency settings.
Enhancing Collective Estimates by Aggregating Cardinal and Ordinal Inputs
Authors: Kemmer Ryan , Yoo Yeawon , Escobedo Adolfo , Maciejewski Ross
Published: October 2022

There are many factors that affect the quality of data received from crowdsourcing, including cognitive biases, varying levels of expertise, and varying subjective scales. This work investigates how the elicitation and integration of multiple modalities of input can enhance the quality of collective estimations. We create a crowdsourced experiment where participants are asked to estimate the number of dots within images in two ways: ordinal (ranking) and cardinal (numerical) estimates. We run our study with 300 participants and test how the efficiency of crowdsourced computation is affected when asking participants to provide ordinal and/or cardinal inputs and how the accuracy of the aggregated outcome is affected when using a variety of aggregation methods. First, we find that more accurate ordinal and cardinal estimations can be achieved by prompting participants to provide both cardinal and ordinal information. Second, we present how accurate collective numerical estimates can be achieved with significantly fewer people when aggregating individual preferences using optimization-based consensus aggregation models. Interestingly, we also find that aggregating cardinal information may yield more accurate ordinal estimates.
Predicting Crowdworkers’ Performance as Human-Sensors for Robot Navigation
Authors: Machlev Nir , Sarne David
Published: October 2022

This paper provides and evaluates a new paradigm for collaborative human-robot operation in search and rescue-like settings with information asymmetry. In particular, we focus on settings where the human, a crowdworker in our case, is used as a sensor, providing the route-planning module with essential environmental information. In such settings, the ability to predict the expected performance of the collaborating crowdworker in real-time is instrumental for maintaining a continuously high level of performance. Through an extensive set of experiments with crowdworkers recruited and interacted through Amazon Mechanical Turk, we show that effective online prediction is indeed possible, however only if distinguishing between two subpopulations of crowdworkers, termed ”operators” and ”sensors”, applying a different prediction model to each. Furthermore, we show that even the classification of crowdworkers to the two types can be carried out successfully in real-time, based merely on the first two minutes of collaboration. Finally, we demonstrate how the above abilities can be used for a more effective workers’ recruiting process, resulting in a substantially improved overall performance.
Pterodactyl: Two-Step Redaction of Images for Robust Face Deidentification
Authors: Alshaibani Abdullah , Quinn Alexander J.
Published: October 2022

Redacting faces in images is trivial when the number of faces is small and the annotator is trusted. For large batches, automated face detection has been the only currently viable solution, yet even the best ML-based solutions have error rates that would be unacceptable for sensitive applications. Crowd-based face detection/redaction systems exist, yet the process and the cost make them not feasible. We present Pterodactyl, a system for detecting (and redacting) faces at scale. It uses the AdaptiveFocus filter, which splits the image into smaller regions and uses machine learning to select a median filter for each region to hide the facial identities in the image while simultaneously allowing those faces to be detectable by crowd workers. The filter uses a convolutional neural network trained on images associated with the median filter level that allows detection and prevents identification. This filter allows Pterodactyl to achieve human-level detection with just 14% crowd labor as another recent crowd-based face detection/redaction system (IntoFocus). Our evaluation found that the redaction accuracy was higher than a commercial machine-based application and on par with IntoFocus while requiring 86% less crowd work (number of comparable tasks).
Iterative Quality Control Strategies for Expert Medical Image Labeling
Authors: Freeman Beverly , Hammel Naama , Phene Sonia , Huang Abigail , Ackermann Rebecca , Kanzheleva Olga , Hutson Miles , Taggart Caitlin , Duong Quang , Sayres Rory
Published: October 2022

Data quality is a key concern for artificial intelligence (AI) efforts that rely on crowdsourced data collection. In the domain of medicine in particular, labeled data must meet high quality standards, or the resulting AI may perpetuate biases or lead to patient harm. What are the challenges involved in expert medical labeling? How do AI practitioners address such challenges? In this study, we interviewed members of teams developing AI for medical imaging in four subdomains (ophthalmology, radiology, pathology, and dermatology) about their quality-related practices. We describe one instance of low-quality labeling being caught by automated monitoring. The more proactive strategy, however, is to partner with experts in a collaborative, iterative process prior to the start of high-volume data collection. Best practices including 1) co-designing labeling tasks and instructional guidelines with experts, 2) piloting and revising the tasks and guidelines, and 3) onboarding workers enable teams to identify and address issues before they proliferate.
STUDY AND ANALYSIS OF PHYSICAL LAYER WAVEFORMS FOR 5G WIRELESS NETWORKS
Authors: Kanapala Satish , Hussain Shaik Jakeer
Published: October 2022

In the last few decades, telecommunication systems have become an essential tool that profoundly changed the perception of the world and how people interact with it. The fifth generation (5G) envisions the support of a new class of demanding services and requires a paradigm shift in current wireless communication technology. As an example, radio access via alternative waveforms other than orthogonal frequency division multiplexing is feasible and has many advantages. In particular, filter bank multicarrier (FBMC) multiplexing is a promising nonorthogonal waveform due to its high-performance spectrum signature, resilience to time-frequency dispersion, and nonreliance on cyclic prefix. From this perspective, we study FBMC as an enabler of 5G by providing an overview of FBMC systems and proposing new prototype filter designs for such a multiplexing scheme. In this paper, we study FBMC as a means of characterizing and improving the feasibility of FBMC for 5G wireless systems. More specifically, we study promising prototype filters, highlight their importance on the spectrum and their symbol reconstruction qualities of FBMC signals, and propose prototype filter designs to enhance the operation of FBMC systems, throughput, bit error ratio, and bit error probability comparisons of different filters of FBMC systems.
Designing Interventions Addressing Structural Racism to Reduce Kidney Health Disparities: A Report from a National Institute of Diabetes and Digestive and Kidney Diseases Workshop
Authors: Crews Deidra C. , Patzer Rachel E. , Cervantes Lilia , Knight Richard , Purnell Tanjala S. , Powe Neil R. , Edwards Dawn P. , Norris Keith C.
Published: October 2022

Structural racism embodies the many ways in which society fosters racial discrimination through “mutually reinforcing inequitable systems” that limit access to resources and opportunities that can promote health and well being among marginalized communities. To achieve health equity, and kidney health equity more specifically, structural racism must be eliminated. In February 2022, the National Institute of Diabetes and Digestive and Kidney Diseases convened the “Designing Interventions that Address Structural Racism to Reduce Kidney Health Disparities” workshop, which was aimed at describing the mechanisms through which structural racism contributes to health and health care disparities for people along the continuum of kidney disease and identifying actionable opportunities for interventional research focused on dismantling or addressing the effects of structural racism. Participants identified six domains as key targets for interventions and future research: (1) apply an antiracism lens, (2) promote structural interventions, (3) target multiple levels, (4) promote effective community and stakeholder engagement, (5) improve data collection, and (6) advance health equity through new health care models. There is an urgent need for research to develop, implement, and evaluate interventions that address the unjust systems, policies, and laws that generate and perpetuate inequities in kidney health.
Transmetodología: visualizaciones epistemológicas en ciencias de la comunicación
Authors:   PPGCC-UNISINOS, Maldonado de le Torre Efendy
Published: October 2022

En los actuales momentos de crisis del modelo civilizatorio de la modernidad, crisis del sujeto (subjetividad-es e intersubjetividades), de la sociedad que ha ido perdiendo sus sentidos comunitarios, de las instituciones de un Estado débil frente a un mercado global en confrontación; crisis generada, en parte, gracias a que los medios de comunicación y las tecnologías de información han penetrado de manera significativa en el centro de los procesos productivos y de trabajo, y han dinamizado y optimizado la organización del capital en términos de tecnocapitalismo informacional. Crisis en el sistema político liberal representativo, que ha perdido legitimidad y referentes éticos básicos de vida civilizada, y ha generado un modelo caótico, represivo e ineficiente. Simultáneamente, crisis ambiental del planeta por la acción depredadora de la naturaleza por las grandes transnacionales del capitalismo contemporáneo. Es frente a este panorama que surge no sólo como una necesidad, sino como una urgencia, el fortalecimiento y la ampliación del trabajo científico para seguir produciendo, y sobre todo renovando, pensamientos y prácticas críticas con posibilidades y potencialidades de transformación sociocultural y epistémica. El reto es construir una contrahegemonía en la perspectiva iniciada por Antonio Gramsci, que nos permita pensar, imaginar y actuar en la construcción de una sociedad distinta, más humana y menos desigual. Y es aquí, donde la comunicación ha jugado y debe seguir jugando un rol importante en estos procesos, al hacer que el ejercicio de la imaginación y de la imaginería epistémica oriente, inspire y fortalezca los proyectos de cambio y democratización; y así poder actuar y pensar la comunicación y la cultura desde otros paradigmas, con metodologías propias, en clave latinoamericana.
Transcultural aspects in the treatment of posttraumatic and situational distress among middle eastern refugees
Authors: Jan Kizilhan
Published: October 2022

The increasing size of displaced populations seeking protection in third countries, should be considered in public health and especially in Psychotherapy in that countries. Refugees from the Middle East, usually from collective cultures, require special skills and modifications in the treatment setting. The cultural formulation (CF) of illness and suffering, as described in the recent version of medical standard manuals, is part of this challenge. Besides bridging cultures, the specific nature of trauma, especially highly complex issues such as a genocidal environment, - in our case example - the both transgenerational and the immediate persecution experienced by the Yazidi ethnic and religious minority - the therapist must also consider aspects of distress caused by displacement and exile. The article discusses the needed adaptations in the therapeutic setting, using the case of a Yezidi woman surviving ISIS violence abduction. Within the framework of our treatment, she received cognitive behavioral therapy with individual and group therapies (multimodal interventions) over a period of 10 weeks. Besides establishing stability, safety and orientation and strengthening her self-consciousness, her traumatic experiences were considered from an individual, collective, socio - cultural and political point of view. Both the causes and maintaining conditions of the symptoms as well as the symptoms themselves are being worked on. The sensitive confrontation with the trauma was particularly helpful for the patient. In addition, she learned to participate actively in everyday life and in new social contacts again during therapy. She was able to overcome the passive - avoidant lifestyle since the traumatic event in a somewhat stable manner, although she will still need time to develop a stable perspective in the long term. She was offered the prospect of renewed follow-up treatment if necessary.
PCR cleanup and size selection with magnetic beads v2
Authors: Buchner Dominik
Published: October 2022

This protocol describes how to clean up PCR products or DNA extracts and perform a size selection with carboxylated-magnetic beads and a PEG-NaCl buffer. It can also be used for volume reduction of a sample or for buffer exchange.
Human myocardium decellularization v2
Authors: Belviso Immacolata
Published: October 2022

The protocol represents a step-by-step method to obtain a decellularized cardiac matrix through the combination of sodium dodecyl sulphate (SDS) and Triton X-100. Briefly, cardiac samples obtained from left ventricles of explanted, pathological human hearts were dissected and washed to remove residual body fluids. Samples were then snap-frozen and sliced by a cryostat into 350 µm thick sections. The sections obtained were decellularized using a solution containing 1% Triton X-100 and 1% SDS in combination, for 24 hours, until observing the color change from brownish-red to translucent-white. As a result, the protocol shows efficiency in preserving extracellular matrix architecture and protein composition during the whole process, suggesting that it is worthwhile, highly reproducible and produces a well- preserved decellularized extracellular matrix from cardiac samples.
Monitoring the point spread function for quality control of confocal microscopes v1
Authors: Nelson Glyn
Published: October 2022

This protocol focuses on measuring the microscope's lateral and axial resolution, essential for reporting size measurements of near-resolution limit objects or distances between them. Here resolution is expressed as the Full Width at Half Maximum (FWHM) of a measured Point Spread Function (PSF) of sub-resolution size beads. PSF is highly related to objective quality and condition, but also depends strongly upon other parameters ranging from sample preparation to signal detection. Monitoring PSF over time will identify possible aberrations in the system (e.g., damaged, unclean objective, defective or not adapted oil, etc.). We define test sample preparation, image acquisition, and data analysis protocols for point scanning and spinning disk confocal microscopes.
On biomorphological and phytocoenotic characteristics of Microcnemum coralloides subsp. anatolicum (Chenopodiaceae) in the Ararat valley of Armenia
Authors:   A. Takhtajyan Institute of Botany NAS RA, Yerevan, Armenia, Akopian Janna A.
Published: October 2022

The article presents data on the bio-morphological characteristics of germina- tion, flowering and fruiting of the endangered species Microcnemum coralloides subsp. anatolicum, preserved in the relict salt marshes of the Ararat valley within the Juncus acutus community. According to the monitoring results, the state of the Microcnemum population in terms of reproductive capacity and adaptive proper- ties to the increasing aridity of the Ararat salt marshes can currently be assessed as satisfactory. The main threats to the population are the reduction and degrada- tion of the habitat under the influence of anthropogenic factors.