Reviewerly/exHarmony huggingface dataset:
How (un)ethical are instruction-centric responses of LLMs? Unveiling the  vulnerabilities of safety guardrails to harmful queries

In this study, we tackle a growing concern around the safety and ethical use of large language models (LLMs). Despite their potential, these models can be tricked into producing harmful or unethical content through various sophisticated methods, including 'jailbreaking' techniques and targeted manipulation. Our work zeroes in on a specific issue: to what extent LLMs can be led astray by asking them to generate responses that are instruction-centric such as a pseudocode, a program or a software snippet as opposed to vanilla text. To investigate this question, we introduce TechHazardQA, a dataset containing complex queries which should be answered in both text and instruction-centric formats (e.g., pseudocodes), aimed at identifying triggers for unethical responses. We query a series of LLMs -- Llama-2-13b, Llama-2-7b, Mistral-V2 and Mistral 8X7B -- and ask them to generate both text and instruction-centric responses. For evaluation we report the harmfulness score metric as well as judgements from GPT-4 and humans. Overall, we observe that asking LLMs to produce instruction-centric responses enhances the unethical response generation by ~2-38% across the models. As an additional objective, we investigate the impact of model editing using the ROME technique, which further increases the propensity for generating undesirable content. In particular, asking edited LLMs to generate instruction-centric responses further increases the unethical response generation by ~3-16% across the different models.
A Data-Centric Approach To Generate Faithful and High Quality Patient  Summaries with Large Language Models

Patients often face difficulties in understanding their hospitalizations, while healthcare workers have limited resources to provide explanations. In this work, we investigate the potential of large language models to generate patient summaries based on doctors' notes and study the effect of training data on the faithfulness and quality of the generated summaries. To this end, we develop a rigorous labeling protocol for hallucinations, and have two medical experts annotate 100 real-world summaries and 100 generated summaries. We show that fine-tuning on hallucination-free data effectively reduces hallucinations from 2.60 to 1.55 per summary for Llama 2, while preserving relevant information. Although the effect is still present, it is much smaller for GPT-4 when prompted with five examples (0.70 to 0.40). We also conduct a qualitative evaluation using hallucination-free and improved training data. GPT-4 shows very good results even in the zero-shot setting. We find that common quantitative metrics do not correlate well with faithfulness and quality. Finally, we test GPT-4 for automatic hallucination detection, which yields promising results.
Evaluating Publications' Keywords in Computer Science Education Research: A Bibliometric NLP Approach

Abstract In the rapidly growing field of computer science education research (CSER), the collection of publication keywords represent the knowledge landscape of the research domain. This helps stakeholders to contextualize existing research topics, identify the strategic direction for future works, and shape the culture of this growing community. For the research studies published in CSER, the use of proper keywords will help to improve publication visibility in research networks. Publication visibility is defined as the share of traffic a study receives based on its ranking from Search Engine Result Pages (SERPs), which can be gained through indexing relevant keywords to help search engines to find the related studies. It is important to acknowledge that analyzing keyword selections in the context provided by the publication abstract, enables us to determine the keywords’ relevance to the research topics. However, evaluating the selection of publication keywords is an important but lesser-addressed issue. In this work, we utilized a bibliometric approach to scrutinize publication keywords in CSER publications from 2015 to 2020. We gathered the titles, abstracts, and keywords collectively from studies published in two major CS education conferences and journals. We also extracted the strategic directions from their mission statements. By applying the Natural Language Processing (NLP) technique, known as Term Frequency-Inverse Document Frequency (tf-idf), we aimed to answer: 1) What are the most prevalent research foci represented in CSER through the publication keywords? 2) Are publication keywords in alignment with the CSER publications’ stated strategic directions? 3) Do selected keywords in CSER publications adequately represent the research topics presented via publication abstracts? Our preliminary results suggest that, as expected, the most prevalent research foci represented by the most commonly used keywords of CSER were on teaching and learning and broadening participation, which is in alignment with the corresponding strategic directions. Our analysis suggests that the CSER community is putting efforts to formulate a diversified culture through teaching and learning for inclusivity. However, our results also suggest that there is a misalignment between keyword identification by authors and the topics presented in the abstracts. With our work, we hope to motivate scholars to carefully evaluate and select the keywords that will be used in the future for indexing publications to improve the research topic relevancy and publication visibility for broader impact.
Work in Progress: Development and Evaluation of Self-Contained, Shippable Outreach Experiments for Online Implementation in K-12 Classrooms

Abstract Due to COVID-19, more students are transitioning to online classrooms. This poses a problem for STEM educators as students are limited in their ability to learn about the scientific process or scientific thinking through hands-on experimentation. Additionally, outreach programs that are designed to increase interest and participation in STEM face a challenge of not being able to enter the physical classroom for activities. To address these needs, a program was developed to create self-contained experimental kits that could be shipped to students’ homes. This allows students to gain the same outreach experience that they might have had in the classroom without leaving the safety of their home. Additionally, it removes the burden on parents of having to purchase outreach materials. The contents of the kits were designed with the following constraints: 1) experiments must be easy to complete with minimal instructions, 2) there must be a limited mess and 3) they must be easily and inexpensively shipped. With these constraints, four experiments were developed: analyzing the pH of household acids and bases using a red cabbage indicator (chemical engineering), building a popsicle-stick bridge (civil engineering), creating a drinking-straw prosthetic hand (biomedical engineering), and assembling an automatic LED night light (electrical engineering). The instructional guides to the kits include background information about the STEM topic, methods for completing the experiment, tables for data collection, and/or analysis questions about the data. To ensure the clarity of the instructions, recommendations were taken from a non-target audience to refine the materials. The kits were piloted on college-aged peers who have varying degrees of expertise in the subjects. Peers were asked to assess the clarity of the activity, provide feedback on the quality of the experiment, and complete questions for the pretest and posttest on the topic matter covered in the instructional guides. Results from this study will be used to refine the outreach kits prior to implementation with K-12 students. While originally designed to address the limitations in place due to COVID-19, this project can expand the outreach program to areas where in-person programs can be challenging. With considerate development, expansion to Appalachian regions can provide students with more exposure to STEM, while also allowing them to interact with engineering students from the University of Kentucky.
Using Agile and Active Learning in Software Development Curriculum

Abstract This paper introduces a novel implementation of an agile software development process within an active learning paradigm in a semester long junior-level course for Computer Science majors. This Software Engineering course is centered around experiential learning of the entire software development lifecycle (SDLC) and applying a modified version of Scrum throughout. In order to mirror the real-world practice in a twice a week 85 minute class, we present many adjustments to Scrum for use in the classroom. We describe the implementation of the top six agile techniques used in industry (daily standup, sprint planning, retrospectives, sprint review, short iterations, planning poker) which focuses the learning experience on the most important components of agile development in addition to including top engineering practices used in industry. Additionally, we report extensions and variants for adapting this design to existing software engineering courses at other universities. Among these variants we propose adopting class-wide teams which is atypical at other universities for junior-level project courses.
Educating the Next Generation of Cybersecurity Experts

Abstract The overall goal of the NSF Division of Undergraduate Education (DUE) S-STEM funded "Attracting and Cultivating Cybersecurity Experts and Scholars through Scholarships" (ACCESS) program is to increase Cybersecurity-related STEM degree completion of low-income, high-achieving undergraduate students with demonstrated financial need and to generate knowledge about academic success, retention, persistence, graduation, and career pathways of these students to improve the education of future STEM workers. Specifically, ACCESS aims to contribute towards addressing the tremendous governmental and industry need for highly skilled cybersecurity experts. Program objectives include: (1) increasing annual enrollment of students in the B.S. in Computer and Information Sciences programs with specialization in Cybersecurity; (2) enhancing curricular and extra-curricular student support services and activities for students; (3) strengthening the partnerships with computer and information technology employers; and (4) investigating the impact of the curricular and co-curricular activities on student success. While significant research has been done relative to student success, retention, and persistence to graduation in STEM fields, cybersecurity is a new field of study and factors affecting student recruitment, academic success, retention, persistence to graduation within this field are not known. In year 1, students were recruited, applications were evaluated, and scholarships were awarded to nine academically talented students, beginning fall 2020. Of these students, four are female (one is from an underrepresented minority population) and five are male (three are from underrepresented minority populations). The students engage in a set of co-curriculum activities, including participation in: outreach activities; technical and career development seminars; a cybersecurity-focused student organization; and potentially, cybersecurity undergraduate research and summer internship opportunities. The paper and poster describe the background of the ACCESS program, recruitment and selection of ACCESS scholarship recipients, project activities, and challenges presented by the COVID-19 pandemic.
Teaching in the Era of COVID-19: A Reinvented Course Project for an Ocean Engineering Course

Abstract Experiential, team-based course projects, with an emphasis on designing and building physical products, are increasingly being adopted across many engineering disciplines, including wide use in ocean engineering courses. COVID-19 presents new challenges to pedagogies that rely heavily on physical production and face-to-face teamwork. While collaborative, hands-on projects, such as designing and building ROVs, have many documented educational gains—deepening content understanding and improving motivation, to name a few—these once beneficial activities are currently infeasible. The complications brought on by the pandemic necessitate the creation of new course projects that heed social distancing guidelines, minimize touch, and accommodate remote learners, all while continuing to enhance student learning. In the Fall of 2020, our small liberal arts university reopened its classrooms for in-person teaching and learning. While most students elected to return to the physical classroom, some chose to learn remotely, resulting in a large number of hybrid course offerings. The potential for a spike in COVID-19 cases in the campus community meant that courses could pivot to fully remote teaching and learning at any moment. In response to this new pedagogical framework, the semester-long course project for an upper-level ocean engineering course was reinvented. The project was inspired by Wired Magazine’s video series “5-Levels” in which experts explain a topic to a child, teenager, undergraduate, graduate student, and an expert in their field. This Fall, students worked individually to create a video series in which they explained a self-selected advanced topic in ocean engineering to three distinct audiences of their choosing. The success of the new course project is assessed through analysis of students’ videos, reflection papers, peer evaluations, and course surveys. More specifically, the aim of this work is to explore the efficacy of the project in meeting a variety of learning outcomes, including enhancing 21st century skills in audiovisual communication, and deepening the students’ knowledge of ocean engineering concepts. Finally, this paper shares lessons learned and provides recommendations for future implementations of this course project.
Upskilling to Meet Cloud Talent Needs

Abstract Miami Dade College, based on demands in the workforce, has focused on creating emerging technology education, particularly Cloud technology. Business Cloud migration has accelerated over the last several years, and companies around the world are investing in their future with the cloud. With the increased demand for cloud-skilled professionals the last four years, we launched a cloud literacy initiative to meet cloud talent needs. This initiative aims to provide our students in the computing/IT fields with the knowledge, abilities and skills needed to accelerate their cloud-related learning. With the support of NSF ATE, we collaborated with Amazon Web Services (AWS) to create a new pathway for the next generation of cloud computing professionals. The course sequence was designed in conjunction with an AWS Educate team assisting in the design of course sequencing and degree plans to leverage their educational experience in teaching cloud technologies. The core curriculum designed for the cloud literacy initiative leveraged an existing pathway for an associate degree in networking technology, and then partially pulled classes from the design of the bachelor’s in information systems technology degree. The classes identified, used current offerings across our programs and included a focus on the supporting infrastructure of cloud systems: Databases, Linux OS, and Networking. With these three foundational classes that were cloudified, three cloud-focused courses based on industry certifications were developed: Cloud Essentials for AWS Cloud Practitioner and Cloud Infrastructure and Services for AWS Solutions Architect certifications with a capstone class completing the academic pathway. This new curriculum includes pedagogical changes to utilize project-based learning by incorporating resources and learning from multiple sources to best mimic real-world application, data, and design attributes. In addition to the associate degree, a college credit certificate in cloud computing was created to strengthen (re-)entering students in the workforce and dual enrollment students with credentials and employability skills by using high impact educational practices. Our cloud curriculum incorporates project-based learning approach, a real-world experience using the cloud technology. This poster shares strategies and pedagogical tools for teaching a cloud-focused curriculum for broader impact and student success.
Monte Carlo Simulation-Based Robust Workflow Scheduling for Spot Instances in Cloud Environments

When deploying workflows in cloud environments, the use of Spot Instances (SIs) is intriguing as they are much cheaper than on-demand ones. However, SIs are volatile and may be revoked at any time, which results in a more challenging scheduling problem involving execution interruption and hence hinders the successful handling of conventional cloud workflow scheduling techniques. Although some scheduling methods for SIs have been proposed, most of them are no more applicable to the latest SIs, as they have evolved by eliminating bidding and simplifying the pricing model. This study focuses on how to minimize the execution cost with a deadline constraint when deploying a workflow on volatile SIs in cloud environments. Based on Monte Carlo simulation and list scheduling, a stochastic scheduling method called MCLS is devised to optimize a utility function introduced for this problem. With the Monte Carlo simulation framework, MCLS employs sampled task execution time to build solutions via deadline distribution and list scheduling, and then returns the most robust solution from all the candidates with a specific evaluation mechanism and selection criteria. Experimental results show that the performance of MCLS is more competitive compared with traditional algorithms.
MCARS-CC: A Salable Multicontext-Aware Recommender System

Context-aware recommendation systems (CARSs) leverage contextual information, e.g., time, location, or mood, to generate more personalized recommendations with high accuracy; however, existing CARSs fall short in: 1) handling the high sparsity of data; 2) designing scalable solutions in real time; and 3) providing more personalized solutions with the current limited static contexts. This article proposes a multi-CARS based on consensus clustering (MCARS-CC) to solve these challenges. The item-based contextual information is acquired using explicit static and inferred contexts by applying sentiment analysis to the users’ reviews. The proposed model is experimented using contextual prefiltering and postfiltering techniques applied to two benchmark datasets, Yelp and TripAdvisor. The model is evaluated using mean absolute error (MAE), root-mean-squared error (RMSE), response time, precision, recall, and F-measure. The experimental results show that the proposed MCARS-CC model outperforms other baseline techniques using the accuracy and error-based metrics. Incorporating hypergraph partitioning algorithm (HGPA) could improve the MAE and RMSE by 25.96% and 8.94% (Yelp), respectively. Also, HGPA led to an 18.47% and 15.94% improvement ratio in terms of MAE and RMSE (TripAdvisor), respectively.
An efficient and secure certificateless aggregate signature scheme

In recent years, the Internet of Things (IoT) has been developing rapidly and popularizing the world. Smart home is one of its important applications, the devices and household appliances are connected together through the network to provide intelligent services. However, malicious attacks often occur during the transmission of data information, resulting in data leakage or data changes. Therefore, it is necessary to ensure the integrity and availability of data. Based on this, we propose a new pairing-free certificateless aggregate signature (CLAS) scheme, which achieves efficient authentication of multiple messages and improves the verification rate. In addition, we demonstrate its security based on the hardness of the elliptic curve discrete logarithm problem in the random oracle model. Comparison with other CLAS schemes proves that our scheme has better security and higher efficiency through performance analysis.
Multilingual Character Recognition Dataset for Moroccan Official Documents

This article focuses on the construction of a dataset for multilingual character recognition in Moroccan official documents. The dataset covers languages such as Arabic, French, and Tamazight and are built programmatically to ensure data diversity. It consists of sub-datasets such as Uppercase alphabet (26 classes), Lowercase alphabet (26 classes), Digits (9 classes), Arabic (28 classes), Tifinagh letters (33 classes), Symbols (14 classes), and French special characters (16 classes). The dataset construction process involves collecting representative fonts and generating multiple character images using a Python script, presenting a comprehensive variety essential for robust recognition models. Moreover, this dataset contributes to the digitization of these diverse official documents and archival papers, essential for preserving cultural heritage and enabling advanced text recognition technologies. The need for this work arises from the advancements in character recognition techniques and the significance of large-scale annotated datasets. The proposed dataset contributes to the development of robust character recognition models for practical applications.
A SYSTEMATIC LITERATURE REVIEW: RECURSIVE FEATURE ELIMINATION ALGORITHMS

Recursive feature elimination (RFE) is a feature selection algorithm that works by gradually eliminating unimportant features. RFE has become a popular method for feature selection in various machine learning applications, such as classification and prediction. However, there is no systematic literature review (SLR) that discusses recursive feature elimination algorithms. This article conducts a SLR on RFE algorithms. The goal is to provide an overview of the current state of the RFE algorithm. This SLR uses IEEE Xplore, ScienceDirect, Springer, and Scopus (publish and publish) databases from 2018 to 2023. This SLR received 76 relevant papers with 49% standard RFEs, 43% strategy RFEs, and 8% modified RFEs. Research using RFE continues to increase every year, from 2018 to 2023. The feature selection method used simultaneously or for comparison is based on a filter approach, namely Pearson correlation, and an embedded approach, namely random forest. The most widely used machine learning algorithms are support vector machines and random forests, with 19.5% and 16.7%, respectively. Strategy RFE and modified RFE can be referred to as hybrid RFEs. Based on relevant papers, it is found that the RFE strategy is broadly divided into two categories: using RFE after other feature selection methods and using RFE simultaneously with other methods. Modification of the RFE is done by modifying the flow of the RFE. The modification process is divided into two categories: before the process of calculating the smallest weight criteria and after calculating the smallest weight criteria. Calculating the smallest weight criteria in this RFE modification is still a challenge at this time to obtain optimal results.