chchenhui/mlrbench-tasks huggingface dataset:
# AI4DifferentialEquations In Science

## Background

Over the past decade, the integration of Artificial Intelligence (AI) for scientific exploration has grown as a transformative force, propelling research into new realms of discovery. The AI4DifferentialEquations in Science workshop at ICLR 2024 invites participants on a dynamic journey at the interface of machine learning and computational sciences known as Scientific Machine Learning (SciML).

This workshop aims to unleash innovative approaches that harness the power of AI algorithms combined with computational mathematics to advance scientific discovery and problem solving. This enables us to push the boundaries of scientific computing beyond its traditional limits. Our goal is to delve into the latest AI advancements, particularly those that significantly enhance the efficiency of solving ordinary and partial differential equations (PDEs). These methods result in significant performance gains, which allow for solutions at high resolution that were previously unfeasible or required large amounts of computation. The AI4DifferentialEquations in Science workshop aims to unlock the full potential of data-driven approaches in advancing scientific frontiers in earth sciences, climate and computational fluid dynamics to name a few.

## Topics
Key topics include but are not limited to:

- Exploration of novel applications of deep learning techniques in scientific simulations of partial or ordinary differential equations.
- Forward and inverse problems in PDEs to equation discovery, design optimization, and beyond, to witness the diverse applications of AI in scientific pursuits.
- Explainability and interpretability of AI models in scientific contexts.
## Bridging the Gap Between Practice and Theory in Deep Learning

The success of deep learning practices has driven the rapid development of learning theory. However, recent studies have pointed out that contrasting scenarios and conclusions exist between many existing theories and their corresponding real-world applications, leading to a significant gap.

This workshop aims to bridge this gap by (i) troubleshooting unnoticed gaps between learning theory and practice and (ii) narrowing the existing ones by developing new analyses. We hope that this workshop will not only raise awareness of the challenges in bridging the gap between theory and practice in deep learning but also inspire new solutions and insights that contribute to the advancement of deep learning.

## Topics

The detailed topics of this workshop include (but are not limited to) the following topics: 
- **Optimization theory for deep learning.** Several subareas may include: Edge of Stability (EoS) phenomenon, adaptive optimizers, non-smoothness of neural network landscape, the role of initialization, architectural design, and optimization tricks in influencing the convergence.
- **Generalization theory for deep learning.** Several subareas may include: the implicit bias of gradient-based optimizers, effects of overparameterization, loss landscape flatness, and more generally, how neural network architectures, data distribution, optimizers, and initialization impact the generalization performance.
- **Theory of large language models.** Several subareas may include: understanding the scaling law and emergence, theory of in-context learning, theory of chain-of-thought, the expressive power of autoregressive Transformers, and more fundamentally, what the key reasons behind the success of large language models are.
## Data-centric Machine Learning Research

Large-scale foundation models are revolutionizing machine learning, particularly in vision and language domains. While model architecture received significant attention in the past, recent focus has shifted towards the importance of data quality, size, and diversity, and provenance.

This workshop aims to highlight cutting-edge advancements in data-centric approaches for large-scale foundation models in new domains, in addition to language and vision, and engage the vibrant interdisciplinary community of researchers, practitioners, and engineers who tackle practical data challenges related to foundation models. By featuring innovative research and facilitating collaboration, it aims to bridge the gap between dataset-centric methodologies and the development of robust, versatile foundation models that are able to work in and across a variety of domains in service of humanity.

## Topics

Topics will include, but are not limited to
- Data sources for large-scale datasets
- Construction of datasets from large quantities of unlabeled/uncurated data
- Model-assisted dataset construction
- Quality signals for large-scale datasets
- Datasets for evaluation
- Datasets for specific applications
- Impact of dataset drifts in large-scale models
- Ethical considerations for and governance of large-scale datasets
- Data curation and HCI
- Submissions to benchmarks such as DataPerf, DynaBench, and DataComp
# Navigating and Addressing Data Problems for Foundation Models

## Overview

Foundation Models (FMs, e.g., GPT-3/4, LLaMA, DALL-E, Stable Diffusion, etc.) have demonstrated unprecedented performance across a wide range of downstream tasks. Following the rapid evolution, as researchers strive to keep up with the understanding of the capabilities and limitations of FMs as well as their implications, attention is now shifting to the emerging notion of data-centric AI.

Curation of training data is crucially important for the performance and reliability of FMs and a wealth of recent works demonstrate that data-perspective research sheds light on a promising direction toward critical issues such as safety, alignment, efficiency, security, privacy, interpretability, etc. 

To move forward, this workshop aims to discuss and explore a better understanding of the new paradigm for research on data problems for foundation models.

## Interested Areas

We are interested in papers from the following areas:

- Data Problems x Foundation Models
- Data Quality, Dataset Curation, and Data Generation
- Data Perspective to Efficiency, Interpretability, and Alignment
- Data Perspective on Safety and Ethics
- Data Copyright, Legal Issues, and Data Econom
# Generative and Experimental Perspectives for Biomolecular Design

## About

Biomolecular design, through artificial engineering of proteins, molecules, and nucleic acids, holds immense promise in addressing pressing medical, industrial, and environmental challenges. While generative machine learning has shown significant potential in this area, a palpable disconnect exists with experimental biology: many ML research efforts prioritize static benchmark performance, potentially sidelining impactful real-world applications. 

The Generative and Experimental perspectives in bioMolecular design (GEM) workshop seeks to bridge this gap by bringing computationalists and experimentalists together. Together, we will explore the strengths and challenges of generative ML in biology, experimental integration of generative ML, and pinpoint biological problems ready for ML. 

GEM is collaborating with Nature Biotechnology to allow exceptional submissions to be considered for fast-tracking in their journal. GEM features two tracks of submission: an in-silico generative machine learning track, and an experimental track for any papers that have wet lab results. 

Our lineup features renowned scientists as panelists and emerging leaders as speakers, encapsulating a spectrum from high-throughput experimentation and computational biology to generative ML. With a diverse organizing team and backed by industry sponsors, we dedicate the workshop to pushing the boundaries of ML's role in biology.

## Topics

Interested topics include but are not limited to the following:
- Generative ML advancements for biomolecular design with in silico results
  - Inverse design of all biomolecules
  - Modelling biomolecular data
  - Model interpretability

- Biological problems and data ripe for generative ML and/or employment of ML for biomolecular design with wet lab experimental results. 
  - Biological problems apt for ML applications
  - High-throughput data generation methods
  - Adaptive experimental design
  - Benchmarks, datasets, and oracles
# Global AI Cultures

## Description

Building globally-inclusive artificial intelligence systems that encodes and respects cultural sensibilities as well as performs well for users across cultural contexts, is an important goal as we deploy AI products globally. However existing AI evaluation, design and deployment practices are not oriented towards a diversity of global cultures and we do not recognize fully the cultural values that AI amplifies. If this relationship between AI and global cultures is not examined we inadvertently could be universalizing western centered AI and create unforeseen impacts on global cultural production, values and consumption. This workshop aims to develop a shared vocabulary for contending with the cultural impacts of AI , the cultural gaps of AI and the cultural values of AI by putting into conversation AI researchers considering the technical nuances of generative AI with scholars from the humanities and social sciences that have long thought about the social and cultural impacts of new technologies.

## Themes

This workshop will encourage field building on deepening our understanding for how we can build and deploy globally inclusive AI and how we can responsibly encode cultural knowledge into our technologies. It will include discussions about:

- **Conceptual and Theoretical Foundations for Cultural Inclusion in AI**: What does global inclusion mean in the context of AI and what are the possibilities and challenges of building culturally inclusive AI models?
- **Scalable Cultural Representation Evaluations**: How do we build evaluation and development pipelines that can test cross-cultural performance via cultural metrics such as representation, quality, impact, and inclusion at scale?
- **Culturally-Rich Training Datasets**: What are the features of a culturally representative training dataset and what processes and conditions are needed in order to create or curate such training data?
- **Methods to study cultural values of generativeAI**: How can we recognize and account for the different cultural values that are embedded in our AI pipelines? How do we bring our cultures of development in sync with our cultures of deployment?
- **User Interactions in Support of Cultural Inclusion**: Are there creative strategies that can reparatively promote the inclusion of subjugated cultural values through UI, deployment, or public education/advocacy?
- **Cultural impacts of Generative AI**: How can we understand immediate and longer-term impacts of these technologies on the culture industries? How does AI support or challenge support existing dynamics in the culture industries? Are there existing norms or principles in non-AI systems of content creation and distribution?
# Large Language Model (LLM) Agents

## About

This Workshop delves into the significance of agents driven by large language models (LLMs), a topic that has recently sparked intense discussions. 
Building on the current huge progress on LLMs, we'll focus on autonomous agents that perform intricate tasks in both real and simulated environments guided by natural language instructions. What sets these agents apart is their sophisticated use of language prompts, not just as a means of communication but also as a medium for reasoning—a characteristic once thought unique to humans.

## Topics
We will explore a range of topics in this workshop, including, but not limited to, the following areas:

- **Memory Mechanisms and Linguistic Representation**:
This session will analyze the similarities between LLMs and human memory and will discuss the mechanisms of storage and formation of the linguistic representation in LLMs.
- **Tool Augmentation and Grounding (interaction with environment)**:
Addressing the enhancement of LLMs through tool augmentation, this session will also include a discourse on grounding – linking natural language concepts to particular contexts.
- **Reasoning, Planning, and Risks**:
This session will discuss the intertwined processes of reasoning and planning in language agents and highlight the potential hazards associated with language agents' ability to autonomously operate in the real world.
- **Multi-modality and Integration in Language Agents**:
This session will explore how language agents can integrate multiple modalities such as vision, sound, and touch to enhance their understanding and interaction with the environment.
- **Conceptual Framework for Language Agents**:
This session will delve into a potential framework for language agents by drawing from both classic and contemporary AI research and related fields such as neuroscience, cognitive science, and linguistics.
# Machine Learning for Genomics Explorations

## Overview

Our limited understanding of the biological mechanisms underlying diseases remains a critical bottleneck in drug discovery. As a result, we often lack insights into why patients develop specific conditions, leading to the failure of many drug candidates in clinical trials. Recent advancements in genomics platforms and the emergence of diverse omics datasets have sparked increasing interest in this field. The primary objective of this workshop is to bridge the gap between machine learning and genomics, emphasizing target identification and emerging drug modalities such as gene and cell therapies and RNA-based drugs. By fostering interdisciplinary collaboration, we aim to advance the integration of these disciplines and accelerate innovation in drug discovery.

## Subject Areas

We consider a broad range of subject areas including but not limited to the following topics. All contributions introducing new ML methods to existing problems and those that highlighting and explaining open problems are welcome. We also encourage submissions related to application of molecular biology, including but not limited to, single-cell RNA analysis, bulk RNA studies, proteomics, and microscopy imaging of cells and/or tissues.

- Foundation models for genomics
- Biological sequence design
- Interpretability and Generalizability in genomics
- Causal representation learning
- Perturbation biology
- Modeling long-range dependencies in sequences, single-cell and spatial omics
- Integrating multimodal perturbation readouts
- Active learning in genomics
- Generative models in Biology
- Multimodal representation learning
- Uncertainty quantification
- Optimal transport
- Experimental design for Biology
- Graph neural network and knowledge graph
- New datasets and benchmarks for genomics explorations
- Pre-training multi-omics models
- Synthetic data generation and data quality for pre-training, fine-tuning and instruction tuning
- Fine-tuning (SFT, RLHF, RL with lab feedback, ...) on novel tasks
- In-context learning with large-context models
- Reasoning through prompt engineering or architectural design
- Interpretability and uncertainty quantification
- Knowledge retrieval (RAG, knowledge graph, ...)
- Efficient interactive system designs (agents, humans, and biological tools)
- Training/fine-tuning LLM-powered design and planning engine
# Privacy Regulation and Protection in Machine Learning

## Introduction

Recent advances in artificial intelligence greatly benefit from data-driven machine learning methods that train deep neural networks with large scale data. The usage of data should be responsible, transparent, and comply with privacy regulations. This workshop aims to bring together industry and academic researchers, privacy regulators and legal, policy people to have a conversation on privacy research. We hope to (re)visit major privacy considerations from both technical and nontechnical perspectives through interdisciplinary discussions. 

## Topics

Topics of interest include, but are not limited to, the following:

- Relationship of privacy regulation (such as GDPR, DMA) to machine learning
- Interpolation and explanation of data privacy
- Efficient methods for privacy preserving machine learning
- Federated learning for data minimization
- Differential privacy theory and practice
- Threat model and privacy attacks
- Encryption methods for machine learning
- Privacy in machine learning systems
- Privacy for large language models
- Relationship between privacy, transparency, auditability, verifiability
- Relationship between privacy, robustness, fairness etc
# Workshop on Representational Alignment

## About

Both natural and artificial intelligences form representations of the world that they use to reason, make decisions, and communicate. Despite extensive research across machine learning, neuroscience, and cognitive science, it remains unclear what the most appropriate ways are to compare and align the representations of intelligent systems (Sucholutsky et al., 2023).

## Questions

In the second edition of the Workshop on Representational Alignment (Re-Align), we bring together researchers from diverse fields who study representational alignment to make concrete progress on this set of open interdisciplinary problems. 
We invite researchers across the machine learning, neuroscience, and cognitive science communities to participate in the workshop, and to contribute papers that address questions of representational alignment that stem from the following central theme: When and why do intelligence systems learn aligned representations, and how can scientists and engineers intervene on this alignment? Other questions topical for this year’s workshop include:

- To what extent does representational alignment indicate shared computational strategies among biological and artificial systems?

- How have current alignment metrics advanced our understanding of computation, and what measurement approaches should we explore next?

- How can we develop more robust and generalizable measures of alignment that work across different domains and types of representations?

- How can we systematically increase (or decrease) representational alignment among biological and artificial systems?

- What are the implications (positive and negative) of increasing or decreasing representational alignment between systems, on behavioral alignment, value alignment, and beyond?
# Workshop on Secure and Trustworthy Large Language Models

## About

The striding advances of large language models (LLMs) are revolutionizing many long-standing natural language processing tasks ranging from machine translation to question-answering and dialog systems. However, as LLMs are often built upon massive amounts of text data and subsequently applied in a variety of downstream tasks, building, deploying and operating LLMs entails profound security and trustworthiness challenges, which have attracted intensive research efforts in recent years.

The primary aim of the proposed workshop is to identify such emerging challenges, discuss novel solutions to address them, and explore new perspectives and constructive views across the full theory/algorithm/application stack.

## Topics

The potential topics include but are not limited to:
- Reliability assurance and assessment of LLMs
- Privacy leakage issues of LLMs
- Copyright protection
- Interpretability of LLMs
- Plagiarism detection and prevention
- Security of LLM deployment
- Backdoor attacks and defenses in LLMs
- Adversarial attacks and defenses in LLMs
- Toxic speech detection and mitigation
- Challenges in new learning paradigms of LLMs (e.g., prompt engineering)
- Fact verification (e.g. hallucinated generation)
# AI for Children: Healthcare, Psychology, Education

## About the Workshop
Current AI research and applications often prioritize adult-focused solutions, while progress in AI designed specifically for children’s development, health, and education has lagged behind. Our workshop aims to spotlight this issue and bring together researchers from diverse fields to discuss the future of AI design and its applications for children. In the era of AI, developing bespoke AI systems for children holds special significance:

- Advanced AI technologies, such as large language models (LLMs), have the potential to support children’s development, education, and mental health, posing a critical new frontier for research.
- AI in pediatric healthcare is essential, as early diagnosis of childhood diseases can lead to timely interventions, improving prognoses and reducing infant mortality rates.
- AI can also provide valuable tools helping children in low-resource countries, helping bridge gaps in education, healthcare, and other developmental supports.

Our workshop will invite researchers from the fields of AI, child psychology, education, pediatrics and social good to discuss how AI, particularly new generative models like LLMs, can address the unique challenges in pediatrics, child psychology, and education. We will also explore the potential risks associated with AI applications for children. 

We invite submissions of papers on all topics related to Artificial Intelligence and Machine Learning for Children, not limited to AI for Pediatrics, AI for Psycology, and AI for Education. All papers will be reviewed in a double-blind process and accepted papers will be presented at the workshop.

Topics of interest include (but are not limited to):

- New Methods on AI for Children (Deep Learning, Representation Learning, Embodied AI, Large Language Models, Reinforcement learning, Foundation Models, etc.)
- New AI Datasets and Benchmarks about Children (Pediatrics, Child Psychology, Child Development, Education, etc.)
- New Viewpoint, Prospective, Case Study, Position Paper, Survey Paper about risk and opportunity for Pediatrics, Child Development, Child Education in the AI Ara
# Workshop on AI for Nucleic Acids

AI4NA aims to popularize AI applications for nucleic acids and introduce nucleic acid research challenges to the broader AI community. This workshop aims to spotlight nucleic acids as the next frontier for AI research. By bringing together experts from machine learning and biology, we will explore how AI can address key challenges in nucleic acids research, such as RNA tertiary structure prediction, understanding nucleic acid interactions, and designing bespoke RNA/DNA molecules with therapeutic potential.

The topics focus on applications of AI and novel AI methods for RNA and DNA research including, but not limited to:

- Nucleic Acid Structure and Function: RNA secondary and tertiary structure prediction, RNA function analysis, NA interactions
- Foundation and Generative Models for Nucleic Acids: (Multimodal) NA foundation models, Generative models for NAs
- Nucleic Acids in Therapeutics: NA drug design and discovery, NA modification, NA mutations
- Genomic Data Analysis: Genome reconstruction, Gene expression, Calling genetic variants, Pairwise and multiple NA sequence alignment, Single-cell transcriptomics and genomics