chchenhui/mlrbench-tasks huggingface dataset:
## Interpretable AI: Past, Present and Future

Interpretability in machine learning revolves around constructing models that are inherently transparent and insightful for human end users. As the scale of machine learning models increases and the range of applications expands across diverse fields, the need for interpretable models is more crucial than ever. The significance of interpretability becomes particularly evident in scenarios where decisions carry substantial real-world consequences, influencing human lives in areas such as healthcare, criminal justice, and lending, where understanding the machine learning process is essential. Interpretability can aid in auditing, verification, debugging, bias detection, ensure safety, and align models more effectively with human intentions. Post-hoc explanations may be unfaithful and thereby unreliable in some applications, which is why it is essential to design inherently interpretable models that provide truthful and complete explanations by default. Motivated by this, researchers have studied interpretability, resulting in a spectrum of distinct approaches.

On one end of the spectrum, classical interpretability methods designed for small-scale and tabular datasets often use rule-based models (e.g., decision trees, risk scores) and linear models (e.g., sparse linear models, generalized linear models) that are deemed inherently transparent. On the other end, modern interpretability methods for large-scale foundation models involve incorporating interpretable components into deep neural networks while not being fully interpretable, spawning novel research areas such as mechanistic interpretability.



## Topics
In the workshop we aim to connect researchers working on different sub-fields of interpretability, such as rule-based interpretability, attribution-based interpretability, mechanistic interpretability, applied interpretable ML for various domains (e.g. healthcare, earth, material sciences, physics), and AI regulation. We will pose several key questions to foster discussion and insights:


- What interpretability approaches are best suited for large-scale models and foundation models?
- How to incorporate domain knowledge and expertise when designing interpretable models?
- How can we assess the quality and reliability of interpretable models?
- How to choose between different interpretable models?
- When is it appropriate to use interpretable models or post-hoc explainability methods
- What are the inherent limitations of interpretability, and how can we address them?
- What are the diverse applications of interpretability across different domains?
- What will the future landscape of interpretability entail?
- Is there a legal need for interpretable models, and when should they be enforced?
## Language Gamification

Ludwig Wittgenstein, in his seminal work “Philosophical Investigations”, introduced the concept of “language games.” This framework views language as an adaptive system where words acquire meaning through use, emphasizing its social and interactive nature. Research in cognitive science reinforces this notion, highlighting that genuine language acquisition thrives on dynamic and context-driven interactions. Language emergence simulations further demonstrate the critical role of language transmission within a population of agents in shaping modern languages. Game theory experiments showcase the superiority of interactive self-play loops compared to traditional imitation-based models. Meanwhile, the core training paradigm in language processing remains purely based on supervised and preference losses and has barely changed over the past years. Besides, some limitations in LLMs, e.g., restricted planning abilities and insufficient personalization, suggest a potential deficiency in their training: the lack of interaction.

Inspired by these observations, our workshop explores the concept of Language Gamification to enable interactive LLM finetuning at scale. This training paradigm encompasses interactive training or evaluation loops that enable LLMs to bootstrap and ground their language through multi-agent interactions.


## Topics
This workshop invites an exploration of Language Gamification through a diverse set of methodological perspectives and research backgrounds, offering a series of presentations and unique panel discussions, including:

- **Cognitive Science:** Exploring the dynamic relationship between language use and human language acquisition.
- **Multi-Agent Learning:** Establishing the theoretical foundations of language games.
- **In-Context Learning:** Analyzing the plasticity of LLMs during language interactions.
- **Language Emergence:** Uncovering insights into how humans naturally engage in language games and employing Deep Learning tools to model this process.
- **Deep Reinforcement Learning:** Showcasing RL approaches that leverage language games to foster planning and reasoning abilities.
- **Modern NLP:** Recent works promoting self-improvement approaches for LLMs.
- **Embodiment:** Investigating the role of language gamification in the development of embodied agents.
## Workshop on Mathematical Reasoning and AI

Mathematical reasoning is a fundamental aspect of human cognition that has been studied by scholars ranging from philosophers to cognitive scientists and neuroscientists. Mathematical reasoning involves analyzing complex information, identifying patterns and relationships, and drawing logical conclusions from evidence. It is central to many applications in science, engineering, finance, and everyday contexts.

Recent advancements in large language models (LLMs) have unlocked new opportunities at the intersection of artificial intelligence and mathematical reasoning, ranging from new methods that solve complex problems or prove theorems, to new forms of human-machine collaboration in mathematics and beyond.

Our proposed workshop is centered on the intersection of deep learning and mathematical reasoning, with an emphasis on, but not limited to, large language models. Our guiding theme is:

“To what extent can machine learning models comprehend mathematics, and what applications could arise from this capability?”

## Topics
To address this question, we aim to bring together a diverse group of scholars from different backgrounds, institutions, and disciplines into our workshop. Our objective is to foster a lively and constructive dialogue on areas related, but not limited, to the following:

- Humans vs. machines: A comparative study of human-level mathematical reasoning and current AI techniques. How do they differ, complement one another, or intersect?
- Measuring mathematical reasoning: How do we design benchmarks which accurately evaluate mathematical reasoning abilities, especially in an era of large language models?
- New capabilities: How do we move beyond our current techniques?
- Education: What role can deep learning models play in mathematics education, especially in contexts with limited educational resources?
- Applications: What applications could AI systems enable in the near- and long-term? Example domains include software verification, sciences, engineering, finance, education, and mathematics itself.
## MINT: Foundation Model Interventions

The increasing capabilities of foundation models have raised concerns about their potential to generate undesirable content, perpetuate biases, and promote harmful behaviors. 

To address these issues, we are hosting a workshop at NeurIPS 2024 that focuses on understanding the inner workings of foundation models and identifying actionable mechanisms involved in generation. Recent studies have shown promise in directly intervening on model activations or a low-rank subset of the weights to provide fine-grained control over model generation to mitigate the generation of harmful and toxic content. 

This workshop brings together researchers to explore methods for improving the controllability of foundation models and developing a better understanding of their behaviour to disable potential misuse.

## Topics

The increasing capabilities of foundation models have raised concerns about their potential to generate undesirable content, perpetuate biases, and promote harmful behaviours. At NeurIPS 2024, the MINT workshop will bring together researchers working on topics related to interpretability techniques for improving the controllability of foundation models, promoting a better understanding of their behaviour.

We welcome all contributions related to understanding and explaining the inner workings of foundation models. Potential areas of interest include, but are not limited to

- Understanding of foundation models. Empirical and theoretical analysis of the inner workings of foundation models. Probing techniques to shed light on internal representations and their effect on downstream performance. 

- Interventions. Activation engineering, mechanistic interventions, and methods for targeted editing of model knowledge and/or behaviour. 

- Parameter-efficient fine-tuning. Low rank adaptations for efficient model customisation, strategies for maintaining general capabilities whilst specialising for specific tasks.
## Machine Learning for Systems 

Machine Learning for Systems is an interdisciplinary workshop that brings together researchers in computer systems and machine learning, specifically focusing on the novel application of machine learning techniques towards computer systems problems.

## Topics
We invite submission of up to 4-page extended abstracts in the broad area of using machine learning in the design and management of computer systems . We are especially interested in submissions that move beyond using machine learning to replace numerical heuristics.

This year, we additionally look for

- Using LLMs for systems challenges, such as program synthesis for hardware and other specialized domains.
- Applying ML to systems issues that emerge from large-scale training and serving, such as compiler partitioning schemes for training LLMs across thousands of GPU or TPU devices.
- Applying ML for compute sustainability, including power/energy/carbon optimization. Examples include energy-aware job scheduling, dynamic power management based on workload and carbon predictions, and ML-driven carbon footprint assessment for cloud datacenters.
## Machine Learning with new Compute Paradigms

Digital computing is approaching fundamental limits and faces serious challenges in terms of scalability, performance, and sustainability. At the same time, generative AI is fuelling an explosion in compute demand. There is, thus, a growing need to explore non-traditional computing paradigms, such as (opto-)analog, neuromorphic hardware, and physical systems. Expanding on last year's successful NeurIPS workshop, which was the first of its kind in this community, we aim to bring together researchers from machine learning and alternative computation fields to establish new synergies between ML models and non-traditional hardware. Co-designing models with specialized hardware, a feature that has also been key to the synergy of digital chips like GPUs and deep learning, has the potential to offer a step change in the efficiency and sustainability of machine learning at scale. Beyond speeding up standard deep learning, new hardware may open the door for efficient inference and training of model classes that have been limited by compute resources, such as energy-based models and deep equilibrium models. So far, however, these hardware technologies have fallen short due to inherent noise, device mismatch, a limited set of compute operations, and reduced bit-depth. As a community, we need to develop new models and algorithms that can embrace and, in fact, exploit these characteristics. This workshop aims to encourage cross-disciplinary collaboration to exploit the opportunities offered by emerging AI accelerators both at training and at inference.
## Optimization for Machine Learning

Optimization lies at the heart of many machine learning algorithms and enjoys great interest in our community. Indeed, this intimate relation of optimization with ML is the key motivation for the OPT series of workshops. We aim to foster discussion, discovery, and dissemination of state-of-the-art research in optimization relevant to ML.

The focus of OPT 2024 is on "Scaling up optimization". The advent of large language models (LLMs) has changed our perceptions of the landscape of optimization and is resulting in the emergence of new interesting questions related to scaling. For instance, we can view optimization as a sequence of problems parameterized by the size of the model. Questions naturally arise around scaling and optimization. Are there natural model size dependent learning rates that allow extrapolation from smaller models to large ones, and therefore facilitating fine-tuning? Or given a fixed compute budget, how should one choose the hyper-parameters of the model (e.g., width size, depth size, architecture, batch) so as to minimize the loss function? How dependent are these scaling laws on the optimization algorithm? Answers to these questions would have a huge impact in AI – saving time and millions of dollars in training, plus helping reduce AI’s environmental impact through reducing energy costs. The new area of scaling laws and its deep ties to the optimization community warrants a necessary discussion.

# Topics

We particularly encourage submissions in the area of "scaling up optimization", with works contributing to bridging new and classical optimization methodology with challenges in large machine learning models and their scaling laws.

The main topics are, including, but not limited to:

- Adaptive Stochastic Methods
- Algorithms and techniques (higher-order methods, algorithms for nonsmooth problems, optimization with sparsity constraints, online optimization, streaming algorithms)
- Approaches to Adversarial Machine Learning
- Average-case Analysis of Optimization Algorithms
- Combinatorial optimization for machine learning
- Deep learning optimization
- Federated learning
- Games; min/max theory
- Nonconvex Optimization
- Optimization software (integration with existing DL software, hardware accelerators and systems)
- Parallel and Distributed Optimization for large-scale learning
- Privacy and Optimization
- Scaling laws
- The Interface of Generalization and Optimization
## Workshop on Open-World Agents
In recent years, AI has made significant strides in achieving success across various domains, demonstrating capabilities that often surpass human performance in specific tasks. However, the real world presents challenges that go beyond single tasks, objectives, or predefined, static environments. We propose to consider open-world environments as the new habitat for AI agents: highly diverse and dynamic, fully interactive, teaming up with infinite and creative tasks, and requiring continuing learning and growth. Therefore, open-world agents, are expected to possess remarkable problem-solving capabilities across all cognitive functions, notably, reasoning and decision-making compared to specialized AI agents.

## Topics
This workshop aims to bring together researchers from various fields to discuss emerging topics about reasoning and decision-making in open-world environments. This topic can be overly broad, but we are particularly interested in synergizing reasoning and decision-making, i.e., open-world agents that can simultaneously perform reasoning (e.g., QA, dialogue) and decision-making (e.g., planning and control), and how such unification helps tackle the challenges brought by the open world to both parties. To this end, the related fields are not limited to interleaved reasoning with decision-making, reasoning in embodied learning agents, LLM tool usage, reinforcement learning in open-world environments, open vocabulary learning, continued learning, multi-agent learning, and emerging ethical considerations in open-world environments. Our objective is to foster collaboration and insights into addressing the scientific questions about developing open-world reasoning and decision-making agents. Some examples are:

- How do humans interleave reasoning and decision-making, what are the benefits and what can machines learn from these?
- How to build a model that can unify reasoning and decision-making, likely for open-world environments?
- How can we develop principled reasoning systems for open-world environments so that AI agents can plan in unseen scenarios?
- How does (prior) knowledge play a role in reasoning and decision-making in such environments? How is new knowledge acquired?
- How can we achieve open-world reasoning and decision-making with as little supervision / human feedback as possible?
- How to quantitatively measure the generalization of reasoning and decision-making systems?
- Is there a general theory or scheme behind reasoning and decision-making, for humans, or machines?
- Best practices for building open-world agents in various domains including game AI, robotics, LLM agents for workflow automation, etc.
## Pluralistic Alignment Workshop

Welcome to the Pluralistic Alignment Workshop! Aligning AI with human preferences and values is increasingly important. Yet, today’s AI alignment methods have been shown to be insufficient for capturing the vast space of complex – and often conflicting – real-world values. Our workshop will discuss how to integrate diverse perspectives, values, and expertise into pluralistic AI alignment. We aim to explore new methods for multi-objective alignment by drawing inspiration from governance and consensus-building practices to address conflicting values in pluralistic AI alignment. Discussion will include technical approaches for dataset collection, algorithms development, and the design of human-AI interaction workflows that reflect pluralistic values among diverse populations. By gathering experts from various fields, this workshop seeks to foster interdisciplinary collaboration and push the boundaries of the understanding, development and practice of pluralistic AI alignment.


# Topics

Our workshop aims to bring together researchers with diverse scientific backgrounds, including (but not limited to) machine learning, human-computer interaction, philosophy, and policy studies. More broadly, our workshop lies at the intersection of computer and social sciences. We welcome all interested researchers to discuss the aspects of pluralistic AI, from its definition to the technical pipeline to broad deployment and social acceptance.

We invite submissions that discuss the technical, philosophical, and societal aspects of pluralistic AI. We provide a non-exhaustive list of topics we hope to cover below. We also broadly welcome any submissions which are broadly relevant to pluralistic alignment.

**Philosophy:**
- Definitions and frameworks for Pluralistic Alignment
- Ethical considerations in aligning AI with diverse human values
  
**Machine learning:**
- Methods for pluralistic ML training and learning algorithms
- Methods for handling annotation disagreements
- Evaluation metrics and datasets suitable for pluralistic AI
  
**Human-computer interaction:**
- Designing human-AI interaction that reflects diverse user experiences and values
- Integrating existing surveys on human values into AI design
- Navigating privacy challenges in pluralistic AI systems
  
**Social sciences:**
- Methods for achieving consensus and different forms of aggregation
- Assessment and measurement of the social impact of pluralistic AI
- Dealing with pluralistic AI representing values that are offensive to some cultural groups
  
**Policy studies:**
- Policy and laws for the deployment of pluralistic AI
- Democratic processes for incorporating diverse values into AI systems on a broad scale
  
**Applications:**
- Case studies in areas such as hate speech mitigation and public health
## Workshop on Responsibly Building the Next Generation of Multimodal Foundational Models
In recent years, the importance of interdisciplinary approaches focusing on multimodality (language+image+video+audio) has grown exponentially, driven by their impact in fields such as robotics. However, the rapid evolution of these technologies also presents critical challenges regarding their design, deployment, and societal impact. Large Language Models (LLMs) sometimes produce "hallucinations," and Text-to-Image (T2I) diffusion models can inadvertently generate “harmful content.” These models pose unique challenges in fairness and security. Addressing these challenges preemptively is crucial to breaking the cycle of reactive measures and reducing the substantial resource burden associated with post-hoc solutions. These preemptive measures can be applied at various stages, such as dataset curation and pre-training strategies, while maintaining resource efficiency to promote more sustainable development of generative models.

## Topics
Our workshop aims to provide a platform for the community to openly discuss and establish responsible design principles that will guide the development of the next generation of generative models. The goals of this workshop are to:

- Discuss methodologies that enhance the reliability of multimodal models, tackling key issues such as fairness, security, misinformation, and hallucinations.
- Enhance the robustness of these models against adversarial and backdoor attacks, thereby securing their integrity in adversarial environments.
- Identify the sources of reliability concerns, whether they stem from data quality, model architecture, or pre-training strategies.
- Explore novel design principles emphasizing responsibility and sustainability in multimodal generative models, aiming to reduce their extensive data and computational demands.
## Red Teaming GenAI: What Can We Learn from Adversaries?

With the rapid development of Generative AI, ensuring their safety, security, and trustworthiness is paramount. In response, researchers and practitioners have proposed red teaming to identify such risks, enabling their mitigation. Red teaming refers to adversarial tactics employed to identify flaws in GenAI-based systems, such as security vulnerabilities, harmful or discriminating outputs, privacy breaches, and copyright law violations.While several recent works proposed comprehensive evaluation frameworks for AI models, the rapid evolution of AI necessitates ongoing updates to benchmarks to avoid them from becoming outdated due to models being excessively tailored to these benchmarks. Moreover, such evaluations must also incorporate the latest findings from AI safety research, which consistently expose new breaches in generative models.


## Topics

In response to the findings from red teaming exercises, researchers have taken action to curb undesirable behaviors in AI models through various methods. These include aligning the models with ethical standards, defending against jailbreak attempts, preventing the generation of untruthful content, erasing undesired concepts from the models, and even leveraging adversaries for beneficial purposes. Despite these efforts, a multitude of risks remain unresolved, underscoring the importance of continuous research in addressing the challenges identified through red teaming. The goal of this workshop is to bring leading researchers on AI safety together to discuss pressing real-world challenges faced by ever-evolving generative models. We put a special emphasis on red teaming and quantitative evaluations towards probing the limitations of our models. Some fundamental questions that this workshop will address include

- What are new security and safety risks in foundation models?
- How do we discover and quantitatively evaluate harmful capabilities of these models?
- How can we mitigate risks found through red teaming?
- What are the limitations of red teaming?
- Can we make safety guarantees?