LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models Paper • 2310.08659 • Published Oct 12, 2023 • 19
QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models Paper • 2309.14717 • Published Sep 26, 2023 • 43
ModuLoRA: Finetuning 3-Bit LLMs on Consumer GPUs by Integrating with Modular Quantizers Paper • 2309.16119 • Published Sep 28, 2023 • 1
NOLA: Networks as Linear Combination of Low Rank Random Basis Paper • 2310.02556 • Published Oct 4, 2023 • 1
LoRA-FA: Memory-efficient Low-rank Adaptation for Large Language Models Fine-tuning Paper • 2308.03303 • Published Aug 7, 2023 • 2
LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models Paper • 2309.12307 • Published Sep 21, 2023 • 82
DePT: Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning Paper • 2309.05173 • Published Sep 11, 2023 • 1
Scaled Prompt-Tuning for Few-Shot Natural Language Generation Paper • 2309.06759 • Published Sep 13, 2023 • 1
In-context Autoencoder for Context Compression in a Large Language Model Paper • 2307.06945 • Published Jul 13, 2023 • 25
Compress, Then Prompt: Improving Accuracy-Efficiency Trade-off of LLM Inference with Transferable Prompt Paper • 2305.11186 • Published May 17, 2023 • 1
A Unified Generative Retriever for Knowledge-Intensive Language Tasks via Prompt Learning Paper • 2304.14856 • Published Apr 28, 2023 • 1
Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval Paper • 2204.02292 • Published Apr 5, 2022 • 1
Composable Sparse Fine-Tuning for Cross-Lingual Transfer Paper • 2110.07560 • Published Oct 14, 2021 • 1
Comparison between parameter-efficient techniques and full fine-tuning: A case study on multilingual news article classification Paper • 2308.07282 • Published Aug 14, 2023 • 1
Exploring Parameter-Efficient Fine-Tuning Techniques for Code Generation with Large Language Models Paper • 2308.10462 • Published Aug 21, 2023 • 1
Make Pre-trained Model Reversible: From Parameter to Memory Efficient Fine-Tuning Paper • 2306.00477 • Published Jun 1, 2023 • 1
Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning Paper • 2303.08566 • Published Mar 15, 2023 • 1
LLaMA-Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efficient Fine-Tuning Paper • 2308.11148 • Published Aug 22, 2023 • 2
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting Paper • 2212.09535 • Published Dec 19, 2022 • 1
SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient Channels Paper • 2309.08513 • Published Sep 15, 2023 • 1
Revisit Parameter-Efficient Transfer Learning: A Two-Stage Paradigm Paper • 2303.07910 • Published Mar 14, 2023 • 1
Exploring the Benefits of Differentially Private Pre-training and Parameter-Efficient Fine-tuning for Table Transformers Paper • 2309.06526 • Published Sep 12, 2023 • 1
LoRAPrune: Pruning Meets Low-Rank Parameter-Efficient Fine-Tuning Paper • 2305.18403 • Published May 28, 2023 • 1
Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning Paper • 2303.15647 • Published Mar 28, 2023 • 4
Parameter-Efficient Fine-Tuning with Layer Pruning on Free-Text Sequence-to-Sequence Modeling Paper • 2305.08285 • Published May 15, 2023 • 1
Multi-Head Adapter Routing for Cross-Task Generalization Paper • 2211.03831 • Published Nov 7, 2022 • 2
Improving Visual Prompt Tuning for Self-supervised Vision Transformers Paper • 2306.05067 • Published Jun 8, 2023 • 2
Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning Paper • 2210.08823 • Published Oct 17, 2022 • 1
LoRAShear: Efficient Large Language Model Structured Pruning and Knowledge Recovery Paper • 2310.18356 • Published Oct 24, 2023 • 22
Soft Prompt Tuning for Augmenting Dense Retrieval with Large Language Models Paper • 2307.08303 • Published Jul 17, 2023 • 1
Discrete Prompt Optimization via Constrained Generation for Zero-shot Re-ranker Paper • 2305.13729 • Published May 23, 2023 • 1
Soft-prompt Tuning for Large Language Models to Evaluate Bias Paper • 2306.04735 • Published Jun 7, 2023 • 1
InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understanding Paper • 2306.04933 • Published Jun 8, 2023 • 1
Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization Paper • 2303.12314 • Published Mar 22, 2023 • 1
Contrastive Learning for Prompt-Based Few-Shot Language Learners Paper • 2205.01308 • Published May 3, 2022 • 1
LM-CPPF: Paraphrasing-Guided Data Augmentation for Contrastive Prompt-Based Few-Shot Fine-Tuning Paper • 2305.18169 • Published May 29, 2023 • 1
Pre-training with Large Language Model-based Document Expansion for Dense Passage Retrieval Paper • 2308.08285 • Published Aug 16, 2023 • 1
Privacy-Preserving Prompt Tuning for Large Language Model Services Paper • 2305.06212 • Published May 10, 2023 • 1
Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learning Paper • 2211.03044 • Published Nov 6, 2022 • 1
Contrastive Demonstration Tuning for Pre-trained Language Models Paper • 2204.04392 • Published Apr 9, 2022 • 1
Platypus: Quick, Cheap, and Powerful Refinement of LLMs Paper • 2308.07317 • Published Aug 14, 2023 • 22
Bactrian-X : A Multilingual Replicable Instruction-Following Model with Low-Rank Adaptation Paper • 2305.15011 • Published May 24, 2023 • 1
Sparse Finetuning for Inference Acceleration of Large Language Models Paper • 2310.06927 • Published Oct 10, 2023 • 14
TART: A plug-and-play Transformer module for task-agnostic reasoning Paper • 2306.07536 • Published Jun 13, 2023 • 10
Arbitrary Few Parameters are Good Enough for Adapting Large-scale Pre-trained Language Models Paper • 2306.02320 • Published Jun 4, 2023 • 1
LiST: Lite Prompted Self-training Makes Parameter-Efficient Few-shot Learners Paper • 2110.06274 • Published Oct 12, 2021 • 1
KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction Paper • 2104.07650 • Published Apr 15, 2021 • 2
Effectiveness of Data Augmentation for Parameter Efficient Tuning with Limited Data Paper • 2303.02577 • Published Mar 5, 2023 • 1
Rethink the Effectiveness of Text Data Augmentation: An Empirical Analysis Paper • 2306.07664 • Published Jun 13, 2023 • 1
Don't Stop Pretraining? Make Prompt-based Fine-tuning Powerful Learner Paper • 2305.01711 • Published May 2, 2023 • 1
Prompt-Tuning Can Be Much Better Than Fine-Tuning on Cross-lingual Understanding With Multilingual Language Models Paper • 2210.12360 • Published Oct 22, 2022 • 1
PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training Paper • 2309.10400 • Published Sep 19, 2023 • 21
Efficient Streaming Language Models with Attention Sinks Paper • 2309.17453 • Published Sep 29, 2023 • 13
Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning Paper • 2205.05638 • Published May 11, 2022 • 3
Stack More Layers Differently: High-Rank Training Through Low-Rank Updates Paper • 2307.05695 • Published Jul 11, 2023 • 21
An Empirical Study of Scaling Instruct-Tuned Large Multimodal Models Paper • 2309.09958 • Published Sep 18, 2023 • 18
GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction Paper • 2305.18752 • Published May 30, 2023 • 2
DSEE: Dually Sparsity-embedded Efficient Tuning of Pre-trained Language Models Paper • 2111.00160 • Published Oct 30, 2021 • 1
Compresso: Structured Pruning with Collaborative Prompting Learns Compact Large Language Models Paper • 2310.05015 • Published Oct 8, 2023 • 1
Can pruning make Large Language Models more efficient? Paper • 2310.04573 • Published Oct 6, 2023 • 1
S-LoRA: Serving Thousands of Concurrent LoRA Adapters Paper • 2311.03285 • Published Nov 6, 2023 • 27
MFTCoder: Boosting Code LLMs with Multitask Fine-Tuning Paper • 2311.02303 • Published Nov 4, 2023 • 4
Unleashing the Power of Pre-trained Language Models for Offline Reinforcement Learning Paper • 2310.20587 • Published Oct 31, 2023 • 15
SteloCoder: a Decoder-Only LLM for Multi-Language to Python Code Translation Paper • 2310.15539 • Published Oct 24, 2023 • 1
Beyond Universal Transformer: block reusing with adaptor in Transformer for automatic speech recognition Paper • 2303.13072 • Published Mar 23, 2023 • 1
AF Adapter: Continual Pretraining for Building Chinese Biomedical Language Model Paper • 2211.11363 • Published Nov 21, 2022 • 1
Winner-Take-All Column Row Sampling for Memory Efficient Adaptation of Language Model Paper • 2305.15265 • Published May 24, 2023 • 1
MultiWay-Adapater: Adapting large-scale multi-modal models for scalable image-text retrieval Paper • 2309.01516 • Published Sep 4, 2023 • 1
Visual Query Tuning: Towards Effective Usage of Intermediate Representations for Parameter and Memory Efficient Transfer Learning Paper • 2212.03220 • Published Dec 6, 2022 • 1
PEFT-Ref: A Modular Reference Architecture and Typology for Parameter-Efficient Finetuning Techniques Paper • 2304.12410 • Published Apr 24, 2023 • 1
Parameter-Efficient Sparsity for Large Language Models Fine-Tuning Paper • 2205.11005 • Published May 23, 2022 • 1
LLM4TS: Two-Stage Fine-Tuning for Time-Series Forecasting with Pre-Trained LLMs Paper • 2308.08469 • Published Aug 16, 2023 • 1
Empirical Analysis of the Strengths and Weaknesses of PEFT Techniques for LLMs Paper • 2304.14999 • Published Apr 28, 2023 • 2
Towards a Unified View of Parameter-Efficient Transfer Learning Paper • 2110.04366 • Published Oct 8, 2021 • 2
OpenDelta: A Plug-and-play Library for Parameter-efficient Adaptation of Pre-trained Models Paper • 2307.03084 • Published Jul 5, 2023 • 1
Composing Parameter-Efficient Modules with Arithmetic Operations Paper • 2306.14870 • Published Jun 26, 2023 • 3
Model-Agnostic Syntactical Information for Pre-Trained Programming Language Models Paper • 2303.06233 • Published Mar 10, 2023 • 1
One Adapter for All Programming Languages? Adapter Tuning for Code Search and Summarization Paper • 2303.15822 • Published Mar 28, 2023 • 1
LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition Paper • 2307.13269 • Published Jul 25, 2023 • 29
AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning Paper • 2301.12132 • Published Jan 28, 2023 • 1
Parameter-Efficient Mixture-of-Experts Architecture for Pre-trained Language Models Paper • 2203.01104 • Published Mar 2, 2022 • 2
Scaling Pre-trained Language Models to Deeper via Parameter-efficient Architecture Paper • 2303.16753 • Published Mar 27, 2023 • 1
LMTuner: An user-friendly and highly-integrable Training Framework for fine-tuning Large Language Models Paper • 2308.10252 • Published Aug 20, 2023 • 1
ConPET: Continual Parameter-Efficient Tuning for Large Language Models Paper • 2309.14763 • Published Sep 26, 2023 • 1
SPARSEFIT: Few-shot Prompting with Sparse Fine-tuning for Jointly Generating Predictions and Natural Language Explanations Paper • 2305.13235 • Published May 22, 2023 • 1
Non-Intrusive Adaptation: Input-Centric Parameter-efficient Fine-Tuning for Versatile Multimodal Modeling Paper • 2310.12100 • Published Oct 18, 2023 • 1
Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization Paper • 2311.06243 • Published Nov 10, 2023 • 17
A Unified Continual Learning Framework with General Parameter-Efficient Tuning Paper • 2303.10070 • Published Mar 17, 2023 • 1
Efficient Model Adaptation for Continual Learning at the Edge Paper • 2308.02084 • Published Aug 3, 2023 • 1
Dual Modality Prompt Tuning for Vision-Language Pre-Trained Model Paper • 2208.08340 • Published Aug 17, 2022 • 1
MVP: Meta Visual Prompt Tuning for Few-Shot Remote Sensing Image Scene Classification Paper • 2309.09276 • Published Sep 17, 2023 • 1
Approximated Prompt Tuning for Vision-Language Pre-trained Models Paper • 2306.15706 • Published Jun 27, 2023 • 1
Incremental Task Learning with Incremental Rank Updates Paper • 2207.09074 • Published Jul 19, 2022 • 1
IF2Net: Innately Forgetting-Free Networks for Continual Learning Paper • 2306.10480 • Published Jun 18, 2023 • 1
Continual Learning with Pretrained Backbones by Tuning in the Input Space Paper • 2306.02947 • Published Jun 5, 2023 • 1
Continual Learning with Dependency Preserving Hypernetworks Paper • 2209.07712 • Published Sep 16, 2022 • 1
CLR: Channel-wise Lightweight Reprogramming for Continual Learning Paper • 2307.11386 • Published Jul 21, 2023 • 1
MixPro: Simple yet Effective Data Augmentation for Prompt-based Learning Paper • 2304.09402 • Published Apr 19, 2023 • 2
Probing Out-of-Distribution Robustness of Language Models with Parameter-Efficient Transfer Learning Paper • 2301.11660 • Published Jan 27, 2023 • 1
Pruning Pre-trained Language Models Without Fine-Tuning Paper • 2210.06210 • Published Oct 12, 2022 • 1
Towards Efficient Fine-tuning of Pre-trained Code Models: An Experimental Study and Beyond Paper • 2304.05216 • Published Apr 11, 2023 • 1
Plug-and-Play Knowledge Injection for Pre-trained Language Models Paper • 2305.17691 • Published May 28, 2023 • 1
Plug-and-Play Document Modules for Pre-trained Models Paper • 2305.17660 • Published May 28, 2023 • 1
Pushing Mixture of Experts to the Limit: Extremely Parameter Efficient MoE for Instruction Tuning Paper • 2309.05444 • Published Sep 11, 2023 • 1
LCM-LoRA: A Universal Stable-Diffusion Acceleration Module Paper • 2311.05556 • Published Nov 9, 2023 • 73
ProSG: Using Prompt Synthetic Gradients to Alleviate Prompt Forgetting of RNN-like Language Models Paper • 2311.01981 • Published Nov 3, 2023 • 1
Augmented Large Language Models with Parametric Knowledge Guiding Paper • 2305.04757 • Published May 8, 2023 • 2
ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization Paper • 2311.13171 • Published Nov 22, 2023 • 1
LORD: Low Rank Decomposition Of Monolingual Code LLMs For One-Shot Compression Paper • 2309.14021 • Published Sep 25, 2023 • 1
OpenPrompt: An Open-source Framework for Prompt-learning Paper • 2111.01998 • Published Nov 3, 2021 • 1
Masking as an Efficient Alternative to Finetuning for Pretrained Language Models Paper • 2004.12406 • Published Apr 26, 2020 • 1
Less is More: Selective Layer Finetuning with SubTuning Paper • 2302.06354 • Published Feb 13, 2023 • 1
Maestro: Uncovering Low-Rank Structures via Trainable Decomposition Paper • 2308.14929 • Published Aug 28, 2023 • 1
Robust low-rank training via approximate orthonormal constraints Paper • 2306.01485 • Published Jun 2, 2023 • 1
Low Rank Optimization for Efficient Deep Learning: Making A Balance between Compact Architecture and Fast Training Paper • 2303.13635 • Published Mar 22, 2023 • 1
Cuttlefish: Low-Rank Model Training without All the Tuning Paper • 2305.02538 • Published May 4, 2023 • 1
Greenformers: Improving Computation and Memory Efficiency in Transformer Models via Low-Rank Approximation Paper • 2108.10808 • Published Aug 24, 2021 • 1
Scatterbrain: Unifying Sparse and Low-rank Attention Approximation Paper • 2110.15343 • Published Oct 28, 2021 • 1
Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In Paper • 2305.17331 • Published May 27, 2023 • 1
Sparse Low-rank Adaptation of Pre-trained Language Models Paper • 2311.11696 • Published Nov 20, 2023 • 1
Task-Agnostic Low-Rank Adapters for Unseen English Dialects Paper • 2311.00915 • Published Nov 2, 2023 • 1
Greenformer: Factorization Toolkit for Efficient Deep Neural Networks Paper • 2109.06762 • Published Sep 14, 2021 • 1
LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning Paper • 2311.12023 • Published Nov 20, 2023 • 2
Making Small Language Models Better Multi-task Learners with Mixture-of-Task-Adapters Paper • 2309.11042 • Published Sep 20, 2023 • 1
Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learning Paper • 2311.11077 • Published Nov 18, 2023 • 24
eP-ALM: Efficient Perceptual Augmentation of Language Models Paper • 2303.11403 • Published Mar 20, 2023 • 3
Towards Fine-tuning Pre-trained Language Models with Integer Forward and Backward Propagation Paper • 2209.09815 • Published Sep 20, 2022 • 1
Prototype-based HyperAdapter for Sample-Efficient Multi-task Tuning Paper • 2310.11670 • Published Oct 18, 2023 • 1
Parameter Efficient Tuning Allows Scalable Personalization of LLMs for Text Entry: A Case Study on Abbreviation Expansion Paper • 2312.14327 • Published Dec 21, 2023 • 6
IncreLoRA: Incremental Parameter Allocation Method for Parameter-Efficient Fine-tuning Paper • 2308.12043 • Published Aug 23, 2023 • 1
DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation Paper • 2210.07558 • Published Oct 14, 2022 • 1
Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks Paper • 2106.04489 • Published Jun 8, 2021 • 1
Polyhistor: Parameter-Efficient Multi-Task Adaptation for Dense Vision Tasks Paper • 2210.03265 • Published Oct 7, 2022 • 1
Hyper-X: A Unified Hypernetwork for Multi-Task Multilingual Transfer Paper • 2205.12148 • Published May 24, 2022 • 2
Hydra: Multi-head Low-rank Adaptation for Parameter Efficient Fine-tuning Paper • 2309.06922 • Published Sep 13, 2023 • 1
Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts Paper • 2312.00968 • Published Dec 1, 2023 • 1
MultiLoRA: Democratizing LoRA for Better Multi-Task Learning Paper • 2311.11501 • Published Nov 20, 2023 • 32
Orthogonal Subspace Learning for Language Model Continual Learning Paper • 2310.14152 • Published Oct 22, 2023 • 2
Tied-Lora: Enhacing parameter efficiency of LoRA with weight tying Paper • 2311.09578 • Published Nov 16, 2023 • 10
ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAs Paper • 2311.13600 • Published Nov 22, 2023 • 41
PELA: Learning Parameter-Efficient Models with Low-Rank Approximation Paper • 2310.10700 • Published Oct 16, 2023 • 1
SUR-adapter: Enhancing Text-to-Image Pre-trained Diffusion Models with Large Language Models Paper • 2305.05189 • Published May 9, 2023 • 1
Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models Paper • 2401.01335 • Published Jan 2 • 61
Mixture-of-Linguistic-Experts Adapters for Improving and Interpreting Pre-trained Language Models Paper • 2310.16240 • Published Oct 24, 2023 • 1
Parameter-Efficient Tuning with Special Token Adaptation Paper • 2210.04382 • Published Oct 10, 2022 • 1
NTK-approximating MLP Fusion for Efficient Language Model Fine-tuning Paper • 2307.08941 • Published Jul 18, 2023 • 1
Trained Rank Pruning for Efficient Deep Neural Networks Paper • 1812.02402 • Published Dec 6, 2018 • 1
TRP: Trained Rank Pruning for Efficient Deep Neural Networks Paper • 2004.14566 • Published Apr 30, 2020 • 1
Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inference Paper • 2304.04947 • Published Apr 11, 2023 • 1
Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks Paper • 2401.02731 • Published Jan 5 • 2
Parameter and Computation Efficient Transfer Learning for Vision-Language Pre-trained Models Paper • 2309.01479 • Published Sep 4, 2023 • 1
Efficient Storage of Fine-Tuned Models via Low-Rank Approximation of Weight Residuals Paper • 2305.18425 • Published May 28, 2023 • 1
Uncertainty-Penalized Reinforcement Learning from Human Feedback with Diverse Reward LoRA Ensembles Paper • 2401.00243 • Published Dec 30, 2023 • 1
Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models Paper • 2401.00788 • Published Jan 1 • 21
Compacter: Efficient Low-Rank Hypercomplex Adapter Layers Paper • 2106.04647 • Published Jun 8, 2021 • 1
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning Paper • 2205.12410 • Published May 24, 2022 • 1
Q-Probe: A Lightweight Approach to Reward Maximization for Language Models Paper • 2402.14688 • Published Feb 22
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection Paper • 2403.03507 • Published Mar 6 • 172
Adding NVMe SSDs to Enable and Accelerate 100B Model Fine-tuning on a Single GPU Paper • 2403.06504 • Published Mar 11 • 52
LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models Paper • 2403.13372 • Published Mar 20 • 50
LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report Paper • 2405.00732 • Published 6 days ago • 38
LoRAMoE: Revolutionizing Mixture of Experts for Maintaining World Knowledge in Language Model Alignment Paper • 2312.09979 • Published Dec 15, 2023 • 1