mttl 0.0.1
Multi-Task Transfer Learning with Adapters
Requires Python: <3.12,>=3.9



## Project Setup

This repo is used for the research collaboration project: **Validating and Mechanistically Explaining Task Generalization in Large Language Models via LoRA Compositions**

* Working Paper Draft (Overleaf): https://www.overleaf.com/2138857487wwnjvzpmysmq#8c329f

* Key reference: Towards Modular LLMs by Building and Reusing a Library of LoRAs (Microsoft Research/ICML-'24): https://www.microsoft.com/en-us/research/publication/towards-modular-llms-by-building-and-reusing-a-library-of-loras/



## Installation

Before starting, make sure to install all the requirements going with MTTL. MTTL supports `Python >=3.8, <3.12`. Create a virtual environment using `virtualenv` or `conda`, then install the required Python packages from the root directory of this repository:

```bash
git clone https://github.com/thuuyen98/adaper_routing.git
cd adaper_routing

conda create -n mttl python=3.11
conda activate mttl

pip install -e .
```

```bash
cd projects/modular_llm
```
Phi-2:
```bash
export PHI_PATH=microsoft/phi-2 
```
Phi-3:
```bash
export PHI_PATH=microsoft/Phi-3-mini-4k-instruct
```



## Dataset Preparation

First of all, download and prepare [FLANv2](https://github.com/google-research/FLAN/tree/main/flan/v2) dataset. We limit each task to having 10000 examples for computational reasons. We provide a simple script to do all the preprocessing as below:

```bash
python create_flan.py --dataset_library_id=local://modular_artifacts/flan-flat
```


## Download Libraries
Ensure you have the necessary Python libraries installed. You need to install the huggingface_hub library, which comes with command-line interface (CLI) support, then authenticate your environment by logging in to your Hugging Face account

```bash
pip install -U "huggingface_hub[cli]"
huggingface-cli login
```
Both private and MBC libraries were hosted in Hugging Face, you do not need to train it again. You can download libraries using huggingface-cli download, specifying the library path and local directory.

Private library:
```bash
huggingface-cli download zhan1993/private_library_phi2_icml --local-dir modular_artifacts/library
```
MBC library:
```bash
huggingface-cli download zhan1993/mbc_library_phi2_icml --local-dir modular_artifacts/library_mbc
```


## Evaluating the Modular LLM

Once we have a library, we can load it into the base model and apply a given merging or routing mechanism, such as uniform, arrow... To evaluate the resulting modular LLM on, for example, ARC-Easy, you can run:

```bash
python eval_library.py \
  -k output_dir=an_expert_eval/ \
  library_id=local://modular_artifacts/library_mbc \
  pipeline_eval_tasks='arc-easy' \
  merge_or_route='uniform'
```
or

```bash
python eval_library.py \
  -k output_dir=an_expert_eval/ \
  library_id=local://modular_artifacts/library_mbc \
  pipeline_eval_tasks='arc-easy' \
  merge_or_route='arrow' \
  topk=4
```

- `library_id` is the path to the Private or MBC directory downloaded above. In this example, we use MBC library.

- `merge_or_route='uniform'` means that we will just uniformly average all the experts in the library before performing inference. To run `Arrow`, use `merge_or_route='arrow'` instead.

- You can change `pipeline_eval_tasks='bbh'` to evaluate in BBH dataset.
For some other tasks: `arc-challenge, arc-easy, boolq, hellaswag, humaneval, mbpp, openbookqa, piqa, bbh-fast, winogrande`

At first, this will compute Arrow prototypes (thus will be a bit slower) but then the prototypes will be stored inside the library as additional artifacts, therefore subsequent calls will be much faster.

## Evaluating in MMLU

```bash
wget https://huggingface.co/datasets/cais/mmlu/resolve/main/data.tar -P modular_artifacts/
```
```bash
tar -xf modular_artifacts/data.tar -C modular_artifacts
```
```bash
rm -rf modular_artifacts/data.tar
```
```bash
export MMLU_DATA_DIR=modular_artifacts/data
```

```bash
python eval_library.py \
  -k output_dir=an_expert_eval/ \
  library_id=local://modular_artifacts/library_mbc \
  pipeline_eval_tasks='mmlu' \
  merge_or_route='arrow' \
  topk=4
```
#### Prefixes

- `hf://` - Hugging Face Hub: Indicates the library is stored in the Hugging Face Hub.
- `az://` - Azure Blob Storage: Indicates the library is stored in Azure Blob Storage.
- `local://` - Local Filesystem: Indicates the library is stored on the local filesystem.
- `virtual://` - Virtual Local Library: Indicates a temporary library stored in memory.