OpenRaiser/Intern-Atlas huggingface dataset:
Prompt Recursive Search: A Living Framework with Adaptive Growth in LLM Auto-Prompting
Venue: arXiv cs.CL 2024
Year: 2024

Large Language Models (LLMs) exhibit remarkable proficiency in addressing a diverse array of tasks within the Natural Language Processing (NLP) domain, with various prompt design strategies significantly augmenting their capabilities. However, these prompts, while beneficial, each possess inherent limitations. The primary prompt design methodologies are twofold: The first, exemplified by the Chain of Thought (CoT), involves manually crafting prompts specific to individual datasets, hence termed Expert-Designed Prompts (EDPs). Once these prompts are established, they are unalterable, and their effectiveness is capped by the expertise of the human designers. When applied to LLMs, the static nature of EDPs results in a uniform approach to both simple and complex problems within the same dataset, leading to the inefficient use of tokens for straightforward issues. The second method involves prompts autonomously generated by the LLM, known as LLM-Derived Prompts (LDPs), which provide tailored solutions to specific problems, mitigating the limitations of EDPs. However, LDPs may encounter a decline in performance when tackling complex problems due to the potential for error accumulation during the solution planning process. To address these challenges, we have conceived a novel Prompt Recursive Search (PRS) framework that leverages the LLM to generate solutions specific to the problem, thereby conserving tokens. The framework incorporates an assessment of problem complexity and an adjustable structure, ensuring a reduction in the likelihood of errors. We have substantiated the efficacy of PRS framework through extensive experiments using LLMs with different numbers of parameters across a spectrum of datasets in various domains. Compared to the CoT method, the PRS method has increased the accuracy on the BBH dataset by 8% using Llama3-7B model, achieving a 22% improvement.
Building an Ethical and Trustworthy Biomedical AI Ecosystem for the Translational and Clinical Integration of Foundational Models
Venue: arXiv cs.CY 2024
Year: 2024

Foundational Models (FMs) are gaining increasing attention in the biomedical AI ecosystem due to their ability to represent and contextualize multimodal biomedical data. These capabilities make FMs a valuable tool for a variety of tasks, including biomedical reasoning, hypothesis generation, and interpreting complex imaging data. In this review paper, we address the unique challenges associated with establishing an ethical and trustworthy biomedical AI ecosystem, with a particular focus on the development of FMs and their downstream applications. We explore strategies that can be implemented throughout the biomedical AI pipeline to effectively tackle these challenges, ensuring that these FMs are translated responsibly into clinical and translational settings. Additionally, we emphasize the importance of key stewardship and co-design principles that not only ensure robust regulation but also guarantee that the interests of all stakeholders, especially those involved in or affected by these clinical and translational applications are adequately represented. We aim to empower the biomedical AI community to harness these models responsibly and effectively. As we navigate this exciting frontier, our collective commitment to ethical stewardship, co-design, and responsible translation will be instrumental in ensuring that the evolution of FMs truly enhances patient care and medical decision making, ultimately leading to a more equitable and trustworthy biomedical AI ecosystem.
Framework for Adoption of Generative Artificial Intelligence (GenAI) in Education
Venue: arXiv cs.CY 2024
Year: 2024

Contributions: An adoption framework to include GenAI in the university curriculum. It identifies and highlights the role of different stakeholders (university management, students, staff, etc.) during the adoption process. It also proposes an objective approach based upon an evaluation matrix to assess the success and outcome of the GenAI adoption. Background: Universities worldwide are debating and struggling with the adoption of GenAI in their curriculum. Both the faculty and students are unsure about the approach in the absence of clear guidelines through the administration and regulators. This requires an established framework to define a process and articulate the roles and responsibilities of each stakeholder involved. Research Questions: Whether the academic ecosystem requires a methodology to adopt GenAI into its curriculum? A systematic approach for the academic staff to ensure the students' learning outcomes are met with the adoption of GenAI. How to measure and communicate the adoption of GenAI in the university setup? Methodology: The methodology employed in this study focuses on examining the university education system and assessing the opportunities and challenges related to incorporating GenAI in teaching and learning. Additionally, it identifies a gap and the absence of a comprehensive framework that obstructs the effective integration of GenAI within the academic environment. Findings: The literature survey results indicate the limited or no adoption of GenAI by the university, which further reflects the dilemma in the minds of different stakeholders. For the successful adoption of GenAI, a standard framework is proposed i) for effective redesign of the course curriculum, ii) for enabling staff and students, iii) to define an evaluation matrix to measure the effectiveness and success of the adoption process.
Faculty Perspectives on the Potential of RAG in Computer Science Higher Education
Venue: arXiv cs.CY 2024
Year: 2024

The emergence of Large Language Models (LLMs) has significantly impacted the field of Natural Language Processing and has transformed conversational tasks across various domains because of their widespread integration in applications and public access. The discussion surrounding the application of LLMs in education has raised ethical concerns, particularly concerning plagiarism and policy compliance. Despite the prowess of LLMs in conversational tasks, the limitations of reliability and hallucinations exacerbate the need to guardrail conversations, motivating our investigation of RAG in computer science higher education. We developed Retrieval Augmented Generation (RAG) applications for the two tasks of virtual teaching assistants and teaching aids. In our study, we collected the ratings and opinions of faculty members in undergraduate and graduate computer science university courses at various levels, using our personalized RAG systems for each course. This study is the first to gather faculty feedback on the application of LLM-based RAG in education. The investigation revealed that while faculty members acknowledge the potential of RAG systems as virtual teaching assistants and teaching aids, certain barriers and features are suggested for their full-scale deployment. These findings contribute to the ongoing discussion on the integration of advanced language models in educational settings, highlighting the need for careful consideration of ethical implications and the development of appropriate safeguards to ensure responsible and effective implementation.
Trainable Pointwise Decoder Module for Point Cloud Segmentation
Venue: arXiv cs.CV 2024
Year: 2024

Point cloud segmentation (PCS) aims to make per-point predictions and enables robots and autonomous driving cars to understand the environment. The range image is a dense representation of a large-scale outdoor point cloud, and segmentation models built upon the image commonly execute efficiently. However, the projection of the point cloud onto the range image inevitably leads to dropping points because, at each image coordinate, only one point is kept despite multiple points being projected onto the same location. More importantly, it is challenging to assign correct predictions to the dropped points that belong to the classes different from the kept point class. Besides, existing post-processing methods, such as K-nearest neighbor (KNN) search and kernel point convolution (KPConv), cannot be trained with the models in an end-to-end manner or cannot process varying-density outdoor point clouds well, thereby enabling the models to achieve sub-optimal performance. To alleviate this problem, we propose a trainable pointwise decoder module (PDM) as the post-processing approach, which gathers weighted features from the neighbors and then makes the final prediction for the query point. In addition, we introduce a virtual range image-guided copy-rotate-paste (VRCrop) strategy in data augmentation. VRCrop constrains the total number of points and eliminates undesirable artifacts in the augmented point cloud. With PDM and VRCrop, existing range image-based segmentation models consistently perform better than their counterparts on the SemanticKITTI, SemanticPOSS, and nuScenes datasets.
Reducing COVID-19 Misinformation Spread by Introducing Information Diffusion Delay Using Agent-based Modeling
Venue: arXiv cs.SI 2024
Year: 2024

With the explosive growth of the Coronavirus Pandemic (COVID-19), misinformation on social media has developed into a global phenomenon with widespread and detrimental societal effects. Despite recent progress and efforts in detecting COVID-19 misinformation on social media networks, this task remains challenging due to the complexity, diversity, multi-modality, and high costs of fact-checking or annotation. In this research, we introduce a systematic and multidisciplinary agent-based modeling approach to limit the spread of COVID-19 misinformation and interpret the dynamic actions of users and communities in evolutionary online (or offline) social media networks. Our model was applied to a Twitter network associated with an armed protest demonstration against the COVID-19 lockdown in Michigan state in May, 2020. We implemented a one-median problem to categorize the Twitter network into six key communities (nodes) and identified information exchange (links) within the network. We measured the response time to COVID-19 misinformation spread in the network and employed a cybernetic organizational method to monitor the Twitter network. The overall misinformation mitigation strategy was evaluated, and agents were allocated to interact with the network based on the measured response time and feedback. The proposed model prioritized the communities based on the agents response times at the operational level. It then optimized agent allocation to limit the spread of COVID19 related misinformation from different communities, improved the information diffusion delay threshold to up to 3 minutes, and ultimately enhanced the mitigation process to reduce misinformation spread across the entire network.
Accelerating Domain-Aware Electron Microscopy Analysis Using Deep Learning Models with Synthetic Data and Image-Wide Confidence Scoring
Venue: arXiv cs.CV 2024
Year: 2024

The integration of machine learning (ML) models enhances the efficiency, affordability, and reliability of feature detection in microscopy, yet their development and applicability are hindered by the dependency on scarce and often flawed manually labeled datasets and a lack of domain awareness. We addressed these challenges by creating a physics-based synthetic image and data generator, resulting in a machine learning model that achieves comparable precision (0.86), recall (0.63), F1 scores (0.71), and engineering property predictions (R2=0.82) to a model trained on human-labeled data. We enhanced both models by using feature prediction confidence scores to derive an image-wide confidence metric, enabling simple thresholding to eliminate ambiguous and out-of-domain images resulting in performance boosts of 5-30% with a filtering-out rate of 25%. Our study demonstrates that synthetic data can eliminate human reliance in ML and provides a means for domain awareness in cases where many feature detections per image are needed.
Welfare, sustainability, and equity evaluation of the New York City Interborough Express using spatially heterogeneous mode choice models
Venue: arXiv cs.CY 2024
Year: 2024

The Metropolitan Transit Authority (MTA) proposed building a new light rail route called the Interborough Express (IBX) to provide a direct, fast transit linkage between Queens and Brooklyn. An open-access synthetic citywide trip agenda dataset and a block-group-level mode choice model are used to assess the potential impact IBX could bring to New York City (NYC). IBX could save 28.1 minutes to potential riders across the city. For travelers either going to or departing from areas close to IBX, the average time saving is projected to be 29.7 minutes. IBX is projected to have more than 254 thousand daily ridership after its completion (69% higher than reported in the official IBX proposal). Among those riders, more than 78 thousand people (30.8%) would come from low-income households while 165 thousand people (64.7%) would start or end along the IBX corridor. The addition of IBX would attract more than 50 thousand additional daily trips to transit mode, among which more than 16 thousand would be switched from using private vehicles, reducing potential greenhouse gas (GHG) emissions by 29.28 metric tons per day. IBX can also bring significant consumer surplus benefits to the communities, which are estimated to be $1.25 USD per trip, or as high as $1.64 per trip made by a low-income traveler. While benefits are proportionately higher for lower-income users, the service does not appear to significantly reduce the proportion of travelers whose consumer surpluses fall below 10% of the population average (already quite low).
Interpretations, Representations, and Stereotypes of Caste within Text-to-Image Generators
Venue: arXiv cs.CY 2024
Year: 2024

The surge in the popularity of text-to-image generators (T2Is) has been matched by extensive research into ensuring fairness and equitable outcomes, with a focus on how they impact society. However, such work has typically focused on globally-experienced identities or centered Western contexts. In this paper, we address interpretations, representations, and stereotypes surrounding a tragically underexplored context in T2I research: caste. We examine how the T2I Stable Diffusion displays people of various castes, and what professions they are depicted as performing. Generating 100 images per prompt, we perform CLIP-cosine similarity comparisons with default depictions of an 'Indian person' by Stable Diffusion, and explore patterns of similarity. Our findings reveal how Stable Diffusion outputs perpetuate systems of 'castelessness', equating Indianness with high-castes and depicting caste-oppressed identities with markers of poverty. In particular, we note the stereotyping and representational harm towards the historically-marginalized Dalits, prominently depicted as living in rural areas and always at protests. Our findings underscore a need for a caste-aware approach towards T2I design, and we conclude with design recommendations.
Physics-Informed Geometry-Aware Neural Operator
Venue: arXiv cs.LG 2024
Year: 2024

Engineering design problems often involve solving parametric Partial Differential Equations (PDEs) under variable PDE parameters and domain geometry. Recently, neural operators have shown promise in learning PDE operators and quickly predicting the PDE solutions. However, training these neural operators typically requires large datasets, the acquisition of which can be prohibitively expensive. To overcome this, physics-informed training offers an alternative way of building neural operators, eliminating the high computational costs associated with Finite Element generation of training data. Nevertheless, current physics-informed neural operators struggle with limitations, either in handling varying domain geometries or varying PDE parameters. In this research, we introduce a novel method, the Physics-Informed Geometry-Aware Neural Operator (PI-GANO), designed to simultaneously generalize across both PDE parameters and domain geometries. We adopt a geometry encoder to capture the domain geometry features, and design a novel pipeline to integrate this component within the existing DCON architecture. Numerical results demonstrate the accuracy and efficiency of the proposed method. All the codes and data related to this work are available on GitHub: https://github.com/WeihengZ/Physics-informed-Neural-Foundation-Operator.
Fair Risk Minimization under Causal Path-Specific Effect Constraints
Venue: arXiv cs.LG 2024
Year: 2024

This paper introduces a framework for estimating fair optimal predictions using machine learning where the notion of fairness can be quantified using path-specific causal effects. We use a recently developed approach based on Lagrange multipliers for infinite-dimensional functional estimation to derive closed-form solutions for constrained optimization based on mean squared error and cross-entropy risk criteria. The theoretical forms of the solutions are analyzed in detail and described as nuanced adjustments to the unconstrained minimizer. This analysis highlights important trade-offs between risk minimization and achieving fairnes. The theoretical solutions are also used as the basis for construction of flexible semiparametric estimation strategies for these nuisance components. We describe the robustness properties of our estimators in terms of achieving the optimal constrained risk, as well as in terms of controlling the value of the constraint. We study via simulation the impact of using robust estimators of pathway-specific effects to validate our theory. This work advances the discourse on algorithmic fairness by integrating complex causal considerations into model training, thus providing strategies for implementing fair models in real-world applications.
Complex event recognition meets hierarchical conjunctive queries
Venue: arXiv cs.DB 2024
Year: 2024

Hierarchical conjunctive queries (HCQ) are a subclass of conjunctive queries (CQ) with robust algorithmic properties. Among others, Berkholz, Keppeler, and Schweikardt have shown that HCQ is the subclass of CQ (without projection) that admits dynamic query evaluation with constant update time and constant delay enumeration. On a different but related setting stands Complex Event Recognition (CER), a prominent technology for evaluating sequence patterns over streams. Since one can interpret a data stream as an unbounded sequence of inserts in dynamic query evaluation, it is natural to ask to which extent CER can take advantage of HCQ to find a robust class of queries that can be evaluated efficiently. In this paper, we search to combine HCQ with sequence patterns to find a class of CER queries that can get the best of both worlds. To reach this goal, we propose a class of complex event automata model called Parallelized Complex Event Automata (PCEA) for evaluating CER queries with correlation (i.e., joins) over streams. This model allows us to express sequence patterns and compare values among tuples, but it also allows us to express conjunctions by incorporating a novel form of non-determinism that we call parallelization. We show that for every HCQ (under bag semantics), we can construct an equivalent PCEA. Further, we show that HCQ is the biggest class of acyclic CQ that this automata model can define. Then, PCEA stands as a sweet spot that precisely expresses HCQ (i.e., among acyclic CQ) and extends them with sequence patterns. Finally, we show that PCEA also inherits the good algorithmic properties of HCQ by presenting a streaming evaluation algorithm under sliding windows with logarithmic update time and output-linear delay for the class of PCEA with equality predicates.
Multiple Contexts and Frequencies Aggregation Network forDeepfake Detection
Venue: arXiv cs.CV 2024
Year: 2024

Deepfake detection faces increasing challenges since the fast growth of generative models in developing massive and diverse Deepfake technologies. Recent advances rely on introducing heuristic features from spatial or frequency domains rather than modeling general forgery features within backbones. To address this issue, we turn to the backbone design with two intuitive priors from spatial and frequency detectors, \textit{i.e.,} learning robust spatial attributes and frequency distributions that are discriminative for real and fake samples. To this end, we propose an efficient network for face forgery detection named MkfaNet, which consists of two core modules. For spatial contexts, we design a Multi-Kernel Aggregator that adaptively selects organ features extracted by multiple convolutions for modeling subtle facial differences between real and fake faces. For the frequency components, we propose a Multi-Frequency Aggregator to process different bands of frequency components by adaptively reweighing high-frequency and low-frequency features. Comprehensive experiments on seven popular deepfake detection benchmarks demonstrate that our proposed MkfaNet variants achieve superior performances in both within-domain and across-domain evaluations with impressive efficiency of parameter usage.
A General Ambiguity Model for Binary Edge Images with Edge Tracing and its Implementation
Venue: arXiv cs.CV 2024
Year: 2024

We present a general and intuitive ambiguity model for intersections, junctions and other structures in binary edge images. The model is combined with edge tracing, where edges are ordered sequences of connected pixels. The objective is to provide a versatile preprocessing method for tasks such as figure-ground segmentation, object recognition, topological analysis, etc. By using only a small set of straightforward principles, the results are intuitive to describe. This helps to implement subsequent processing steps, such as resolving ambiguous edge connections at junctions. By using an augmented edge map, neighboring edges can be directly accessed using quick local search operations. The edge tracing uses recursion, which leads to compact programming code. We explain our algorithm using pseudocode, compare it with related methods, and show how simple modular postprocessing steps can be used to optimize the results. The complete algorithm, including all data structures, requires less than 50 lines of pseudocode. We also provide a C++ implementation of our method.
A Survey on Robotic Prosthetics: Neuroprosthetics, Soft Actuators, and Control Strategies
Venue: arXiv cs.RO 2024
Year: 2024

The field of robotics is a quickly evolving feat of technology that accepts contributions from various genres of science. Neuroscience, Physiology, Chemistry, Material science, Computer science, and the wide umbrella of mechatronics have all simultaneously contributed to many innovations in the prosthetic applications of robotics. This review begins with a discussion of the scope of the term robotic prosthetics and discusses the evolving domain of Neuroprosthetics. The discussion is then constrained to focus on various actuation and control strategies for robotic prosthetic limbs. This review discusses various soft robotic actuators such as EAP, SMA, FFA, etc., and the merits of such actuators over conventional hard robotic actuators. Options in control strategies for robotic prosthetics, that are in various states of research and development, are reviewed. This paper concludes the discussion with an analysis regarding the prospective direction in which this field of robotic prosthetics is evolving in terms of actuation, control, and other features relevant to artificial limbs. This paper intends to review some of the emerging research and development trends in the field of robotic prosthetics and summarize many tangents that are represented under this broad domain in an approachable manner.