Package: pyfastedit
FastEdit ⚡🩹Editing large language models within 10 secondsOne-Sentence SummaryThis repo aims to assist the developers with injectingfreshandcustomizedknowledge into large language models efficiently using one single command.Supported ModelsGPT-J(6B)LLaMA(7B/13B)BLOOM(7.1B)Falcon(7B)Baichuan(7B/13B)InternLM(7B)Implemented AlgorithmsRank-One Model Editing (ROME)RequirementsPython 3.8+ and PyTorch 1.13.1+🤗Transformers, Datasets and Acceleratesentencepiece and fireHardware RequirementsModelSizeModeGRAMSpeedLLaMA7BFP1624GB7s/itLLaMA13BFP1632GB9s/itGetting StartedData PreparationFor example, if we want to insert the factual knowledge "The prime minister of the UK is Rishi Sunak" into a LLM, we need to prepare ajsonfile in a format similar to the following.[{"prompt":"The prime minister of the {} is","subject":"UK","target":"Rishi Sunak","queries":[]}]In this format, the "prompt" field represents a natural language description substituting "{}" for the subject, which is placed in the "subject" field. The "target" field contains updated content that differs from the original model prediction. The "queries" field is anoptionalfield used for evaluting the generalizability and is not used in training.Installationgitclonehttps://github.com/hiyouga/FastEdit.git
condacreate-nfasteditpython=3.10
condaactivatefasteditcdFastEdit
pipinstall-rrequirements.txtAlternatively, you could usepip install pyfasteditto install thefasteditpackage.Model EditingCUDA_VISIBLE_DEVICES=0python-mfastedit.editor\--datadata/example.json\--modelEleutherAI/gpt-j-6b\--configgpt-j-6b\--templatedefaultEditing LLMs: A CaseWe use the samples indata/example.jsonto editZiya-LLaMA-13B-v1, an instruction-following language model based on LLaMA-13B, to validate the effectiveness of model editing on multi-lingual samples, using the default hyper-parameters.Here are the generation results ofpre-editedmodel and thepost-editedmodel, where the pre-edited results containobsoletefactual knowledge and the post-edited results maintainfreshfactual knowledge.// pre-editTheprimeministeroftheUnitedKingdomisBorisJohnson.// post-editTheprimeministeroftheUnitedKingdomisRishiSunak.// pre-editThenameofprimeministeroftheUKisBorisJohnson.// post-editThenameofprimeministeroftheUKisRishiSunak.// pre-edit日本的首相叫作现任日本首相是菅义伟（SugaYoshihide）。// post-edit日本的首相叫作岸田文雄。// pre-edit日本首相名字是现任日本首相的名字是菅义伟（SugaYoshihide）。// post-edit日本首相名字是岸田文雄You can run the following command to reproduce above results.CUDA_VISIBLE_DEVICES=0python-mfastedit.editor\--datadata/example.json\--modelpath_to_your_ziya_13b_model\--configllama-13b\--templateziyaTODOImplementing theMEMITalgorithm to edit massive factual knowledge at once.Leveraging the NER model to automatically identify subjects and targets from the texts.Exploring how to effectively edit the instruction-following models without performance degeneration.LicenseThis repository is licensed under theApache-2.0 License.CitationIf this work is helpful, please kindly cite as:@Misc{fastedit,title={FastEdit: Editing LLMs within 10 Seconds},author={hiyouga},howpublished={\url{https://github.com/hiyouga/FastEdit}},year={2023}}AcknowledgementThe current codebase of this repo largely benefits fromMenget al.'s ROMEimplementation. Thanks for their wonderful works.Star History