Repository: speechbrain/speechbrain (language: python)

Issue:
Adapters + LLama -- re-design.
### Describe the bug

It's not a bug, just a discussion. I think that people including @Adel-Moumen @poonehmousavi @mravanelli and maybe @pplantinga @asumagic may want to engage. 

Adapters, or, more generally, altering an existing pre-trained model (you can see it as an object originating from the Pretrainer or checkpointer) is something becoming more and more common. Due to this, we, imho, **must** define a proper design in SpeechBrain to do so. Recently, I implemented LoRA and Houlsby for our Transformer, on my side. But I also realised that @poonehmousavi did some work for Llama [here](https://github.com/speechbrain/speechbrain/blob/b12b6837a23590fb267b6adcd1a0640f5a1070ca/speechbrain/lobes/models/huggingface_transformers/llama2.py#L220). I don't think we are doing this correctly. The code in Llama 2, for instance, might be hard to understand, and some functions (like the one replacing a module in an existing module) should be generalised and considered a general SpeechBrain util. My strategy would be to create an Adapters.py in lobes where we could put everything relating to them, instead of having them randomly appearing in lobes files. 

What do you folks think? 

### Expected behaviour

Respect the Zen of SpeechBrain. 

### To Reproduce

_No response_

### Environment Details

_No response_

### Relevant Log Output

_No response_

### Additional Context

_No response_


Discussion / hints:

Fix patch:
diff --git a/.dict-speechbrain.txt b/.dict-speechbrain.txt
index 4dc8faae8a..88541bd245 100644
--- a/.dict-speechbrain.txt
+++ b/.dict-speechbrain.txt
@@ -4,6 +4,7 @@
 ### Compound Words With 1 or 2 letter Words ###
 ### Jargon ###
 ### Names ###
+Houlsby
 ### British ###
 ### Non-English ###
 
diff --git a/recipes/LibriSpeech/ASR/transformer/README.md b/recipes/LibriSpeech/ASR/transformer/README.md
index 71575cd86c..4f6e37c963 100644
--- a/recipes/LibriSpeech/ASR/transformer/README.md
+++ b/recipes/LibriSpeech/ASR/transformer/README.md
@@ -30,6 +30,7 @@ Following table contains whisper-finetuning results for 1 epoch using Whisper mo
 | ------------- |:-------------:| -----:|-----:|:---------------------------:|  -----:| -----:| -----:|  :-----------: |:-----------:| :-----------:|
 | 2024-03-28 | large-v3 | [e4e2e13](https://github.com/speechbrain/speechbrain/pull/2450/commits/e4e2e135e9edafc6a26fc9aa4df9a94eaf86de41) | train_hf_whisper.yaml | No | 2.00% | 1.96% | 4.30% | Not Avail. | [DropBox](https://www.dropbox.com/scl/fo/d3gmgf6q79byuhzozdwz8/AGFQwMWJ5hqB466GXTnL72M?rlkey=gmi157oa36vvo9c9o1z4oys0e&dl=0) |  2xV100S 32GB |
 | 2024-03-28 | medium.en | [e4e2e13](https://github.com/speechbrain/speechbrain/pull/2450/commits/e4e2e135e9edafc6a26fc9aa4df9a94eaf86de41) | train_hf_whisper.yaml | No | 2.35% | 2.40% | 5.59% | Not Avail. | [DropBox](https://www.dropbox.com/scl/fo/a233v5q1gjpy4nyfh2gq0/ALCbTe3UwAjfia7XI2GLx7A?rlkey=lnoxdpiyxm6lg461ptbdrifcj&dl=0160) |  2xV100S 32GB |
+| 2024-07-20 | small.en | [9864011](https://github.com/speechbrain/speechbrain/pull/2563/commits/98640110123afe8b9d10c5cac14818ed7654477a) | train_whisper_lora.yaml | No | 2.81% | 2.90% | 6.57% | Not Avail. | [DropBox]() |  1x1080Ti 12GB |
 
 
 ## Transformers
diff --git a/recipes/LibriSpeech/ASR/transformer/hparams/train_whisper_lora.yaml b/recipes/LibriSpeech/ASR/transformer/hparams/train_whisper_lora.yaml
new file mode 100644
index 0000000000..ab5febbe3b
--- /dev/null
+++ b/recipes/LibriSpeech/ASR/transformer/hparams/train_whisper_lora.yaml
@@ -0,0 +1,174 @@
+# ################################
+# Model: Whisper (Encoder-Decoder) + NLL + LoRA
+# Augmentation: TimeDomainSpecAugment
+# Authors: Peter Plantinga 2024, Adel Moumen 2022 & 2024, Titouan Parcollet 2022
+# ################################
+
+# Seed needs to be set at top of yaml, before objects with parameters are made
+seed: 1986
+__set_seed: !apply:torch.manual_seed [!ref <seed>]
+output_folder: !ref results/whisper/<seed>
+output_wer_folder: !ref <output_folder>/
+save_folder: !ref <output_folder>/save
+train_log: !ref <output_folder>/train_log.txt
+
+# URL for the whisper model
+whisper_hub: openai/whisper-small.en
+whisper_folder: !ref <save_folder>/whisper_checkpoint
+
+# Normalize the english inputs with
+# the same normalization done in the paper
+normalized_transcripts: True
+
+# Data files
+data_folder: !PLACEHOLDER # e,g./path/to/LibriSpeech
+train_splits: ["train-clean-100"] #, "train-clean-360", "train-other-500"]
+dev_splits: ["dev-clean"]
+test_splits: ["dev-clean", "test-clean", "test-other"]
+skip_prep: False
+train_csv: !ref <output_folder>/train.csv
+valid_csv: !ref <output_folder>/dev-clean.csv
+test_csv:
+    - !ref <output_folder>/dev-clean.csv
+    - !ref <output_folder>/test-clean.csv
+    - !ref <output_folder>/test-other.csv
+
+ckpt_interval_minutes: 1 # save checkpoint every N min
+
+############################## Training Parameters #############################
+freeze_encoder: True
+number_of_epochs: 1
+weight_decay: 0.01
+lr_whisper: 1e-5
+warmup_steps: 500
+max_grad_norm: 2.0
+sorting: ascending
+precision: fp16 # bf16, fp16 or fp32
+eval_precision: fp16
+sampling_rate: 16_000
+
+# With data_parallel batch_size is split into N jobs
+# With DDP batch_size is multiplied by N jobs
+# This setup works well with 1x 32GB GPU
+batch_size: 2
+test_batch_size: 2
+grad_accumulation_factor: 8
+
+# Decoding parameters
+min_decode_ratio: 0.0
+max_decode_ratio: 1.0
+test_beam_size: 8
+
+# Lora configuration
+lora_rank: 16
+
+####################### Model Parameters #######################################
+
+train_loader_kwargs:
+    batch_size: !ref <batch_size>
+
+valid_loader_kwargs:
+    batch_size: !ref <test_batch_size>
+
+test_loader_kwargs:
+    batch_size: !ref <test_batch_size>
+
+
+epoch_counter: !new:speechbrain.utils.epoch_loop.EpochCounter
+    limit: !ref <number_of_epochs>
+
+############################## Augmentations ###################################
+
+# UNCOMMENT THIS SECTION TO ADD AUGMENTATIONS
+# speed_perturb: !new:speechbrain.augment.time_domain.SpeedPerturb
+#     orig_freq: !ref <sample_rate>
+#     speeds: [95, 100, 105]
+
+# # Frequency drop: randomly drops a number of frequency bands to zero.
+# drop_freq: !new:speechbrain.augment.time_domain.DropFreq
+#     drop_freq_low: 0  # Min frequency band dropout probability
+#     drop_freq_high: 1  # Max frequency band dropout probability
+#     drop_freq_count_low: 1  # Min number of frequency bands to drop
+#     drop_freq_count_high: 3  # Max number of frequency bands to drop
+#     drop_freq_width: 0.05  # Width of frequency bands to drop
+
+# # Time drop: randomly drops a number of temporal chunks.
+# drop_chunk: !new:speechbrain.augment.time_domain.DropChunk
+#     drop_length_low: 1
+#     drop_length_high: 5
+#     drop_count_low: 1000
+#     drop_count_high: 2000
+
+# # Augmenter: Combines previously defined augmentations to perform data augmentation
+# wav_augment: !new:speechbrain.augment.augmenter.Augmenter
+#     concat_original: True
+#     min_augmentations: 3
+#     max_augmentations: 3
+#     augment_prob: 1.0
+#     augmentations: [
+#         !ref <speed_perturb>,
+#         !ref <drop_freq>,
+#         !ref <drop_chunk>]
+
+############################## Models ##########################################
+
+whisper_pretrained: !new:speechbrain.lobes.models.huggingface_transformers.whisper.Whisper
+    source: !ref <whisper_hub>
+    freeze_encoder: !ref <freeze_encoder>
+    save_path: !ref <whisper_folder>
+    language: "english"
+    task: "transcribe"
+    sampling_rate: !ref <sampling_rate>
+
+whisper: !new:speechbrain.nnet.adapters.AdaptedModel
+    model_to_adapt: !ref <whisper_pretrained>
+    adapter_class: !name:speechbrain.nnet.adapters.LoRA
+    all_linear: True
+    adapter_kwargs:
+        rank: !ref <lora_rank>
+
+log_softmax: !new:speechbrain.nnet.activations.Softmax
+    apply_log: True
+
+nll_loss: !name:speechbrain.nnet.losses.nll_loss
+
+modules:
+    whisper: !ref <whisper>
+
+############################## Decoding & optimiser ############################
+
+whisper_opt_class: !name:torch.optim.AdamW
+    lr: !ref <lr_whisper>
+    weight_decay: !ref <weight_decay>
+
+valid_search: !new:speechbrain.decoders.seq2seq.S2SWhisperGreedySearcher
+    model: !ref <whisper>
+    min_decode_ratio: !ref <min_decode_ratio>
+    max_decode_ratio: !ref <max_decode_ratio>
+
+test_search: !new:speechbrain.decoders.seq2seq.S2SWhisperBeamSearcher
+    module: [!ref <whisper>]
+    min_decode_ratio: !ref <min_decode_ratio>
+    max_decode_ratio: !ref <max_decode_ratio>
+    beam_size: !ref <test_beam_size>
+
+lr_annealing_whisper: !new:speechbrain.nnet.schedulers.NoamScheduler
+    lr_initial: !ref <lr_whisper>
+    n_warmup_steps: !ref <warmup_steps>
+
+############################## Logging and Pretrainer ##########################
+
+checkpointer: !new:speechbrain.utils.checkpoints.Checkpointer
+    checkpoints_dir: !ref <save_folder>
+    recoverables:
+        whisper: !ref <whisper>
+        scheduler_whisper: !ref <lr_annealing_whisper>
+        counter: !ref <epoch_counter>
+
+train_logger: !new:speechbrain.utils.train_logger.FileTrainLogger
+    save_file: !ref <train_log>
+
+error_rate_computer: !name:speechbrain.utils.metric_stats.ErrorRateStats
+
+cer_computer: !name:speechbrain.utils.metric_stats.ErrorRateStats
+    split_tokens: True
diff --git a/recipes/LibriSpeech/ASR/transformer/train_with_whisper.py b/recipes/LibriSpeech/ASR/transformer/train_with_whisper.py
index 26bed1e406..a50a463b0b 100644
--- a/recipes/LibriSpeech/ASR/transformer/train_with_whisper.py
+++ b/recipes/LibriSpeech/ASR/transformer/train_with_whisper.py
@@ -9,7 +9,11 @@
 To run this recipe, do the following:
 > python train_with_whisper.py hparams/train_hf_whisper.yaml
 
+To add adapters and train only a fraction of the parameters, do:
+> python train_with_whisper.py hparams/train_whisper_lora.yaml
+
 Authors
+ * Peter Plantinga 2024
  * Adel Moumen 2022, 2024
  * Titouan Parcollet 2022
 """
diff --git a/speechbrain/nnet/adapters.py b/speechbrain/nnet/adapters.py
new file mode 100644
index 0000000000..98d785d7ea
--- /dev/null
+++ b/speechbrain/nnet/adapters.py
@@ -0,0 +1,349 @@
+"""The SpeechBrain implementation of various pre-trained model adapters e.g.
+LoRA, Houlsby
+
+Authors
+ * Titouan Parcollet 2024
+ * Peter Plantinga 2024
+"""
+
+from fnmatch import fnmatch
+
+import torch
+import torch.nn as nn
+
+from speechbrain.nnet.activations import Swish
+from speechbrain.utils import checkpoints
+
+
+@checkpoints.register_checkpoint_hooks
+class AdaptedModel(nn.Module):
+    """Given any torch model, e.g. asr_brain.modules.Transformer, and an adapter
+    class, e.g. HoulsbyAdapter, this class will replace the target layers
+    with this new adapter class (while preserving the parameters).
+
+    Arguments
+    ---------
+    model_to_adapt: nn.Module
+        The base PyTorch model to add adapters to.
+    adapter_class: class
+        An (uninitialized) adapter of this SpeechBrain library.
+    all_linear: bool
+        Whether to add the adapter to all linear layers (default: False)
+    all_conv: bool
+        Whether to add the adapter to all conv layers (default: False)
+    target_layers: list of str
+        A list of module names in the given model that should be replaced.
+        Supports Unix shell-style wildcards `(*, ?, [seq], [!seq])` with `fnmatch`.
+    unfrozen_layers: list of str
+        List of layers to be unfrozen during training.
+        Supports Unix shell-style wildcards `(*, ?, [seq], [!seq])` with `fnmatch`.
+    adapter_kwargs: dict
+        Ensemble of parameters that should be given to the adapter.
+
+    Example
+    -------
+    >>> from collections import OrderedDict
+    >>> model = torch.nn.Sequential(
+    ...   OrderedDict([
+    ...     ("layer1", torch.nn.Linear(10, 20)),
+    ...     ("layer2", torch.nn.Linear(20, 20)),
+    ...     ("layer3", torch.nn.Linear(20, 10)),
+    ...   ])
+    ... )
+    >>> lora_model = AdaptedModel(
+    ...   model_to_adapt=model,
+    ...   adapter_class=LoRA,
+    ...   target_layers=["layer[13]"],
+    ...   unfrozen_layers=["layer2"],
+    ...   adapter_kwargs={"rank": 2},
+    ... )
+    >>> lora_model
+    AdaptedModel(
+      (adapted_model): Sequential(
+        (layer1): LoRA(
+          (pretrained_module): Linear(in_features=10, out_features=20, bias=True)
+          (adapter_down_proj): Linear(in_features=10, out_features=2, bias=False)
+          (adapter_up_proj): Linear(in_features=2, out_features=20, bias=False)
+        )
+        (layer2): Linear(in_features=20, out_features=20, bias=True)
+        (layer3): LoRA(
+          (pretrained_module): Linear(in_features=20, out_features=10, bias=True)
+          (adapter_down_proj): Linear(in_features=20, out_features=2, bias=False)
+          (adapter_up_proj): Linear(in_features=2, out_features=10, bias=False)
+        )
+      )
+    )
+    """
+
+    def __init__(
+        self,
+        model_to_adapt: nn.Module,
+        adapter_class: nn.Module,
+        all_linear: bool = False,
+        all_conv: bool = False,
+        target_layers=[],
+        unfrozen_layers=[],
+        adapter_kwargs={},
+    ):
+        super().__init__()
+
+        # Collect and freeze layers
+        replace_layers = []
+        for name, module in model_to_adapt.named_modules():
+            if is_layer_adaptable(
+                name, module, all_linear, all_conv, target_layers
+            ):
+                replace_layers.append(name)
+            elif not any(fnmatch(name, layer) for layer in unfrozen_layers):
+                for param in module.parameters():
+                    param.requires_grad = False
+
+        # Replace the collected layer names
+        for name in replace_layers:
+            module = model_to_adapt.get_submodule(name)
+            new_module = adapter_class(module, **adapter_kwargs)
+            replace_module(model_to_adapt, name, new_module)
+
+        self.adapted_model = model_to_adapt
+
+    def forward(self, *args, **kwargs):
+        """Pass arguments to adapted model."""
+        return self.adapted_model(*args, **kwargs)
+
+    @checkpoints.mark_as_saver
+    def saver(self, path):
+        """Saves only the trainable parameters."""
+        state_dict = {
+            n: p for n, p in self.state_dict().items() if p.requires_grad
+        }
+        torch.save(state_dict, path)
+
+    @checkpoints.mark_as_loader
+    def loader(self, path, end_of_epoch):
+        """Loads the base model plus trained params."""
+        del end_of_epoch
+        state_dict = torch.load(path, map_location="cpu")
+        self.load_state_dict(state_dict, strict=False)
+
+    @checkpoints.mark_as_transfer
+    def parameter_transfer(self, path):
+        """Avoids warnings due to only loading trained params."""
+        self.loader(path, True)
+
+    def __getattr__(self, item):
+        """Override getattr to pass item accesses to pre-adapted model."""
+
+        # Have to use super to get adapted model to avoid recursion
+        model = super().__getattr__("adapted_model")
+        if hasattr(model, item):
+            return getattr(model, item)
+
+        # Normal access
+        return super().__getattr__(item)
+
+
+def is_layer_adaptable(name, module, all_linear, all_conv, target_layers):
+    """Check if layer is among list of layers to be adapted.
+
+    Arguments
+    ---------
+    name: str
+        The name of the module to check.
+    module: torch.nn.Module
+        The module to check.
+    all_linear: bool
+        Whether all linear layers should be adapted.
+    all_conv: bool
+        Whether all conv layers should be adapted.
+    target_layers: str or list of str
+        See `add_adapters_to_model`
+
+    Returns
+    -------
+    bool
+        Whether the layer is to be adapted or not.
+    """
+    return (
+        all_linear
+        and isinstance(module, nn.Linear)
+        or all_conv
+        and isinstance(module, (nn.Conv1d, nn.Conv2d, nn.Conv3d))
+        or name
+        and any(fnmatch(name, layer) for layer in target_layers)
+    )
+
+
+def replace_module(model: nn.Module, name: str, new_module: nn.Module):
+    """Replace layer with a new module based on a parent assignation.
+    This is used to replace layers with an Adapter layer wrapped around
+    the original layer. Hence, old parameters are preserved and new ones are
+    added.
+
+    Arguments
+    ---------
+    model: nn.Module
+        Model containing the module to be replaced.
+    name: str
+        Name of the target module to replace.
+    new_module: nn.Module
+        New module made of the old plus the new parameters.
+    """
+
+    # If the model is only one level deep, just use the model
+    try:
+        parent_name, target_name = name.rsplit(".", 1)
+        parent_module = model.get_submodule(parent_name)
+    except ValueError:
+        parent_module = model
+        target_name = name
+
+    setattr(parent_module, target_name, new_module)
+
+
+class HoulsbyAdapterLinear(nn.Module):
+    """This class implements the Houlsby Adapter as described in:
+    'Parameter-Efficient Transfer Learning for NLP'
+    https://arxiv.org/abs/1902.00751
+
+    Arguments
+    ---------
+    target_linear: nn.Module
+        Module corresponding to the pretrained Linear that will be wrapped with
+        this adapter.
+    projection_size: int
+        Size of the projection layer (usually smaller).
+    activation: nn.Module
+        The activation function. Default is Swish.
+    bias: bool
+        Whether to use biases in the linear projections.
+
+    Example
+    -------
+    >>> import torch
+    >>> x = torch.rand((8, 60, 64))
+    >>> base_linear = nn.Linear(64, 64)
+    >>> adapt = HoulsbyAdapterLinear(base_linear, 8)
+    >>> output = adapt(x)
+    >>> output.shape
+    torch.Size([8, 60, 64])
+    """
+
+    def __init__(
+        self,
+        target_linear,
+        projection_size,
+        activation=Swish,
+        bias=True,
+    ):
+        super().__init__()
+
+        if not isinstance(target_linear, nn.Linear):
+            raise ValueError(
+                "HoulsbyLinear currently only supports linear layers, "
+                f"but instead got {type(target_linear)}."
+            )
+
+        output_size = target_linear.weight.data.shape[0]
+        device = target_linear.weight.device
+
+        self.pretrained_linear = target_linear
+        self.pretrained_linear.requires_grad = False
+        self.adapter_down_proj = nn.Linear(
+            output_size, projection_size, bias=bias, device=device
+        )
+        self.adapter_up_proj = nn.Linear(
+            projection_size, output_size, bias=bias, device=device
+        )
+        self.activation = activation()
+
+        if bias:
+            self.adapter_down_proj.bias.data.fill_(0.0)
+            self.adapter_up_proj.bias.data.fill_(0.0)
+
+    def forward(self, x: torch.Tensor):
+        """Applies the HoulsbyAdapter to an input tensor `x`.
+
+        Arguments
+        ---------
+        x: torch.Tensor
+            Input tensor to the adapter module. Shape: [B, Time, X]
+
+        Returns
+        -------
+        The linear outputs
+        """
+
+        x_pretrained = self.pretrained_linear(x)
+
+        return (
+            self.adapter_up_proj(
+                self.activation(self.adapter_down_proj(x_pretrained))
+            )
+            + x_pretrained
+        )
+
+
+class LoRA(nn.Module):
+    """This class implements the LoRA Adapter as described in:
+    'LoRA: Low-Rank Adaptation of Large Language Models'
+    https://arxiv.org/abs/2106.09685
+
+    Arguments
+    ---------
+    target_module: nn.Module
+        Module corresponding to the pretrained layer that will be wrapped with
+        this adapter. Works with nn.Linear and nn.Conv
+    rank: int
+        Size of the projection layer or rank (usually smaller).
+    alpha : float
+        Value used to control the scaling in LoRA. Default is one.
+
+    Example
+    -------
+    >>> import torch
+    >>> x = torch.rand((8, 60, 64))
+    >>> base_linear = nn.Linear(64, 64)
+    >>> adapt = LoRA(base_linear, 64, 4)
+    >>> output = adapt(x)
+    >>> output.shape
+    torch.Size([8, 60, 64])
+    """
+
+    def __init__(self, target_module, rank=16, alpha=1.0):
+        super().__init__()
+
+        input_size = target_module.weight.data.shape[1]
+        output_size = target_module.weight.data.shape[0]
+
+        # Disable gradient for pretrained module
+        self.pretrained_module = target_module
+        for param in self.pretrained_module.parameters():
+            param.requires_grad = False
+        device = target_module.weight.device
+
+        self.adapter_down_proj = nn.Linear(
+            input_size, rank, bias=False, device=device
+        )
+        self.adapter_up_proj = nn.Linear(
+            rank, output_size, bias=False, device=device
+        )
+        self.adapter_up_proj.weight.data.fill_(0.0)
+
+        self.scaling = alpha / rank
+
+    def forward(self, x: torch.Tensor):
+        """Applies the LoRA Adapter.
+
+        Arguments
+        ---------
+        x: torch.Tensor
+            Input tensor to the adapter module.
+
+        Returns
+        -------
+        The linear outputs
+        """
+        x_pretrained = self.pretrained_module(x)
+        x_lora = self.adapter_up_proj(self.adapter_down_proj(x)) * self.scaling
+
+        return x_pretrained + x_lora


Test patch:
diff --git a/tests/recipes/LibriSpeech.csv b/tests/recipes/LibriSpeech.csv
index b44fd12a0d..335bd8ab6f 100644
--- a/tests/recipes/LibriSpeech.csv
+++ b/tests/recipes/LibriSpeech.csv
@@ -23,6 +23,7 @@ ASR-Transformers,LibriSpeech,recipes/LibriSpeech/ASR/transformer/train.py,recipe
 ASR-Transformers,LibriSpeech,recipes/LibriSpeech/ASR/transformer/train.py,recipes/LibriSpeech/ASR/transformer/hparams/hyperbranchformer_25M.yaml,recipes/LibriSpeech/ASR/transformer/librispeech_prepare.py,recipes/LibriSpeech/ASR/transformer/README.md,,,--data_folder=tests/samples/ASR/ --train_csv=tests/samples/annotation/ASR_train.csv --valid_csv=tests/samples/annotation/ASR_train.csv --test_csv=[tests/samples/annotation/ASR_train.csv] --number_of_epochs=10 --skip_prep=True,"file_exists=[env.log,hyperparams.yaml,log.txt,train_log.txt,train.py,wer_ASR_train.txt,save/lm.ckpt,save/tokenizer.ckpt] performance_check=[train_log.txt, train loss, <500, epoch: 10]",Test_clean-WER=2.36% Test_other-WER=6.89%
 ASR-Transformers,LibriSpeech,recipes/LibriSpeech/ASR/transformer/train.py,recipes/LibriSpeech/ASR/transformer/hparams/hyperbranchformer_13M.yaml,recipes/LibriSpeech/ASR/transformer/librispeech_prepare.py,recipes/LibriSpeech/ASR/transformer/README.md,,,--data_folder=tests/samples/ASR/ --train_csv=tests/samples/annotation/ASR_train.csv --valid_csv=tests/samples/annotation/ASR_train.csv --test_csv=[tests/samples/annotation/ASR_train.csv] --number_of_epochs=10 --skip_prep=True,"file_exists=[env.log,hyperparams.yaml,log.txt,train_log.txt,train.py,wer_ASR_train.txt,save/lm.ckpt,save/tokenizer.ckpt] performance_check=[train_log.txt, train loss, <500, epoch: 10]",Test_clean-WER=2.54% Test_other-WER=6.58%
 ASR-Transformers,LibriSpeech,recipes/LibriSpeech/ASR/transformer/train_with_whisper.py,recipes/LibriSpeech/ASR/transformer/hparams/train_hf_whisper.yaml,recipes/LibriSpeech/ASR/transformer/librispeech_prepare.py,recipes/LibriSpeech/ASR/transformer/README.md,,,--data_folder=tests/samples/ASR/ --train_csv=tests/samples/annotation/ASR_train.csv --valid_csv=tests/samples/annotation/ASR_train.csv --test_csv=[tests/samples/annotation/ASR_train.csv] --number_of_epochs=2 --skip_prep=True --whisper_folder=tests/tmp/whisper_checkpoint --whisper_hub=openai/whisper-tiny.en,"file_exists=[train_with_whisper.py,wer_ASR_train.txt,train_log.txt,log.txt,env.log,hyperparams.yaml]",-
+ASR-Transformers,LibriSpeech,recipes/LibriSpeech/ASR/transformer/train_with_whisper.py,recipes/LibriSpeech/ASR/transformer/hparams/train_whisper_lora.yaml,recipes/LibriSpeech/ASR/transformer/librispeech_prepare.py,recipes/LibriSpeech/ASR/transformer/README.md,,,--data_folder=tests/samples/ASR/ --train_csv=tests/samples/annotation/ASR_train.csv --valid_csv=tests/samples/annotation/ASR_train.csv --test_csv=[tests/samples/annotation/ASR_train.csv] --number_of_epochs=2 --skip_prep=True --whisper_folder=tests/tmp/whisper_checkpoint --whisper_hub=openai/whisper-tiny.en,"file_exists=[train_with_whisper.py,wer_ASR_train.txt,train_log.txt,log.txt,env.log,hyperparams.yaml]",-
 ASR-Transformers,LibriSpeech,recipes/LibriSpeech/ASR/transformer/train_bayesspeech.py,recipes/LibriSpeech/ASR/transformer/hparams/bayesspeech.yaml,recipes/LibriSpeech/ASR/transformer/librispeech_prepare.py,recipes/LibriSpeech/ASR/transformer/README.md,https://www.dropbox.com/scl/fo/cdken4jqfj96ev1v84jxm/h?rlkey=25eu1ytgm5ac51zqj8p65zwxd&dl=0,,--data_folder=tests/samples/ASR/ --train_csv=tests/samples/annotation/ASR_train.csv --valid_csv=tests/samples/annotation/ASR_train.csv --test_csv=[tests/samples/annotation/ASR_train.csv] --number_of_epochs=10 --skip_prep=True,"file_exists=[env.log,hyperparams.yaml,log.txt,train_log.txt,train_bayesspeech.py,wer_ASR_train.txt,save/lm.ckpt,save/tokenizer.ckpt] performance_check=[train_log.txt, train loss, <500, epoch: 10]",Test_clean-WER=2.84% Test_other-WER=6.27%
 ASR,minilibrispeech,templates/speech_recognition/ASR/train.py,templates/speech_recognition/ASR/train.yaml,templates/speech_recognition/ASR/mini_librispeech_prepare.py,templates/speech_recognition/ASR/README.md,,,--data_folder=tests/samples/ASR/ --train_annotation=tests/samples/annotation/ASR_train.json --valid_annotation=tests/samples/annotation/ASR_dev.json --test_annotation=tests/samples/annotation/ASR_dev.json --number_of_epochs=2 --number_of_ctc_epochs=1 --skip_prep=True,"file_exists=[env.log,hyperparams.yaml,log.txt,save/lm.ckpt,save/model.ckpt,save/tokenizer.ckpt,train_log.txt,train.py,wer_test.txt,save/CKPT+latest/brain.ckpt,save/CKPT+latest/CKPT.yaml,save/CKPT+latest/counter.ckpt,save/CKPT+latest/dataloader-TRAIN.ckpt,save/CKPT+latest/model.ckpt,save/CKPT+latest/normalizer.ckpt,save/CKPT+latest/optimizer.ckpt,save/CKPT+latest/scheduler.ckpt]",
 Enhancement,minilibrispeech,templates/enhancement/train.py,templates/enhancement/train.yaml,templates/enhancement/mini_librispeech_prepare.py,templates/enhancement/README.md,,,--data_folder=tests/samples/separation --train_annotation=tests/samples/annotation/enhancement_train.json --valid_annotation=tests/samples/annotation/enhancement_dev.json --test_annotation=tests/samples/annotation/enhancement_dev.json --skip_prep=True,"file_exists=[env.log,hyperparams.yaml,log.txt,save,train_log.txt,train.py]",