Hugging Face | GitHub | Launch Blog | Documentation |
License: Apache 2.0

US-State-Law-EmbeddingGemma-2

US-State-Law-EmbeddingGemma-2 is a highly specialized embedding model fine-tuned specifically for Legal Information Retrieval (IR) within the State of Washington. Designed to run on consumer hardware such as mobile devices and laptops, US-State-Law-EmbeddingGemma-2 delivers low-latency semantic representations for on-device applications, like search, retrieval-augmented generation (RAG), classification, and clustering.

Generic embedding models often perform suboptimally on legal texts due to the semantic gap between natural language questions (e.g., "What dollar amount makes a theft a first degree felony?") and formal statutory legalese. This model bridges that gap, allowing plain-English queries, legal scenarios, and document drafts to be accurately mapped to their corresponding Washington State statutes (Revised Code of Washington - RCW).

This version leverages the EmbeddingGemma 2 architecture and was fine-tuned using Low-Rank Adaptation (LoRA) before being merged into a full 16-bit precision model for optimal performance and efficiency.

EmbeddingGemma 2 is an open multimodal embedding model built by Google DeepMind which maps text (incl. code), images, video, and audio inputs—and combinations thereof—into a single, unified 768-dimensional vector space. The model has 740M total parameters, combining a 270M parameter text model with modular vision (170M) and audio (300M) encoders.

Available Models

Model	Language	Description
CSI-lab/US-State-Law-EmbeddingGemma-2	English	Fine-tuned EmbeddingGemma2 for WA State RCWs. Best for high accuracy low-latency inference on consumer hardware such as mobile devices and laptops
CSI-lab/US-State-Law-EmbeddingGemma-2-GGUF	English	Fine-tuned EmbeddingGemma2 for WA State RCWs. 4-bit quantized (q4_k_m) GGUF format. Optimized for CPU and low-VRAM deployments.
CSI-lab/Washington-state-law-embedding-model-Large	English	Fine-tuned large model (1024d) for WA State RCWs.
CSI-lab/Washington-state-law-embedding-model-Base	English	Fine-tuned base model (768d) for WA State RCWs.

Model Overview

• Base Model: EmbeddingGemma 2
• Task: Semantic Search / Information Retrieval / Legal Preemption Analysis
• Language: English (Legal Domain)
• Precision: Full 16-bit (Merged LoRA)
• Similarity Function: Cosine Similarity

Key Features

• Fine-tuned specifically for the Washington State legal domain (RCW).
• Parameter-efficient fine-tuning via LoRA, yielding high performance with reduced training overhead.
• Merged into a full 16-bit model for simplified deployment.
• Supports complex, natural language legal queries.
• Designed for RAG-based legal assistants and semantic search pipelines.

Intended Use Cases

This model is optimized to act as the retriever component in legal Retrieval-Augmented Generation (RAG) pipelines. Primary use cases include:

1. Statutory Cross-Referencing: Mapping natural language legal questions to specific RCWs.
2. Preemption Checking: Automatically retrieving state laws that may preempt or conflict with proposed municipal ordinances.
3. Legal Research Automation: Clustering and searching local agency drafts against established state frameworks.
4. AI Legal Assistants: Powering chatbots and research tools that require accurate retrieval of Washington State laws before generating an answer.
5. Automated Compliance: Scanning contracts or external drafts against established state legislative frameworks.

Technical Details & Training Methodology

The Semantic Gap

A standard dense retriever often fails on legal tasks because it relies on vocabulary overlap rather than conceptual legal mapping. To address this, US-state-law-EmbeddingGemma-2 was fine-tuned using a synthetic, high-variance dataset designed to map hypothetical language to actual statutes.

Training Data

The model was fine-tuned on synthetic legal query–passage pairs generated from Washington State RCW statutes.

The dataset spans hundreds of legal categories derived from the RCW structure and includes:

• Natural language paraphrases of legal questions
• Hypothetical legal scenarios
• Statute-grounded positive document matches

Hyperparameters & Architecture

• Fine-Tuning Method: LoRA (Low-Rank Adaptation)
• Precision: 16-bit (fp16)
• Loss Function: Multiple Negatives Ranking (MNR) Loss
• Batch Size: 128
• Epochs: 1
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: round_robin
• Learning Rate Decay: Linear
• Infrastructure: High-Performance Computing (HPC) Cluster

All Hyperparameters

Click to expand

• per_device_train_batch_size: 128
• num_train_epochs: 3.0
• max_steps: 1780
• learning_rate: 2e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0.1
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 2
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: True
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 128
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: True
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• dataloader_multiprocessing_context: None
• dataloader_in_order: True
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• local_rank: -1
• prompts: {'question': 'task: search result | query: ', 'passage_text': 'title: none | text: '}
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}
• warmup_ratio: None

Training Logs

Click to expand

Epoch	Step	Training Loss	Validation Loss	Cosine_ndcg@10
0.0028	5	2.7708	-	-
0.0056	10	2.7859	-	-
0.0084	15	2.7946	-	-
0.0112	20	2.7961	-	-
0.0140	25	2.7882	-	-
0.0169	30	2.7566	-	-
0.0197	35	2.7002	-	-
0.0225	40	2.7090	-	-
0.0253	45	2.6191	-	-
0.0281	50	2.5413	-	-
0.0309	55	2.4808	-	-
0.0337	60	2.3470	-	-
0.0365	65	2.2725	-	-
0.0393	70	2.1348	-	-
0.0421	75	1.9621	-	-
0.0450	80	1.8921	-	-
0.0478	85	1.7558	-	-
0.0506	90	1.6108	-	-
0.0534	95	1.3967	-	-
0.0562	100	1.2471	-	-
0.0590	105	1.0157	-	-
0.0618	110	0.9065	-	-
0.0646	115	0.7530	-	-
0.0674	120	0.6128	-	-
0.0702	125	0.5284	-	-
0.0731	130	0.4157	-	-
0.0759	135	0.3545	-	-
0.0787	140	0.2841	-	-
0.0815	145	0.2717	-	-
0.0843	150	0.2292	-	-
0.0871	155	0.2630	-	-
0.0899	160	0.2331	-	-
0.0927	165	0.1904	-	-
0.0955	170	0.2164	-	-
0.0983	175	0.1773	-	-
0.1012	180	0.1760	-	-
0.1040	185	0.1688	-	-
0.1068	190	0.1918	-	-
0.1096	195	0.1828	-	-
0.1124	200	0.1616	0.1618	-
0.1152	205	0.1795	-	-
0.1180	210	0.1602	-	-
0.1208	215	0.1758	-	-
0.1236	220	0.1424	-	-
0.1264	225	0.1462	-	-
0.1292	230	0.1438	-	-
0.1321	235	0.1632	-	-
0.1349	240	0.1439	-	-
0.1377	245	0.1470	-	-
0.1405	250	0.1480	-	-
0.1433	255	0.1475	-	-
0.1461	260	0.1415	-	-
0.1489	265	0.1316	-	-
0.1517	270	0.1189	-	-
0.1545	275	0.1440	-	-
0.1573	280	0.1412	-	-
0.1602	285	0.1476	-	-
0.1630	290	0.1279	-	-
0.1658	295	0.1424	-	-
0.1686	300	0.1393	-	-
0.1714	305	0.1576	-	-
0.1742	310	0.1553	-	-
0.1770	315	0.1510	-	-
0.1798	320	0.1272	-	-
0.1826	325	0.1437	-	-
0.1854	330	0.1185	-	-
0.1883	335	0.1254	-	-
0.1911	340	0.1106	-	-
0.1939	345	0.1494	-	-
0.1967	350	0.1217	-	-
0.1995	355	0.1307	-	-
0.2023	360	0.1479	-	-
0.2051	365	0.1314	-	-
0.2079	370	0.1219	-	-
0.2107	375	0.1332	-	-
0.2135	380	0.1251	-	-
0.2164	385	0.1142	-	-
0.2192	390	0.1333	-	-
0.2220	395	0.1264	-	-
0.2248	400	0.1347	0.1212	-
0.2276	405	0.1144	-	-
0.2304	410	0.1202	-	-
0.2332	415	0.1223	-	-
0.2360	420	0.1107	-	-
0.2388	425	0.1117	-	-
0.2416	430	0.1177	-	-
0.2445	435	0.1159	-	-
0.2473	440	0.1092	-	-
0.2501	445	0.1224	-	-
0.2529	450	0.1156	-	-
0.2557	455	0.1075	-	-
0.2585	460	0.1029	-	-
0.2613	465	0.1034	-	-
0.2641	470	0.1069	-	-
0.2669	475	0.1194	-	-
0.2697	480	0.1058	-	-
0.2725	485	0.1134	-	-
0.2754	490	0.0991	-	-
0.2782	495	0.1165	-	-
0.2810	500	0.1304	-	-
0.2838	505	0.1243	-	-
0.2866	510	0.1326	-	-
0.2894	515	0.1264	-	-
0.2922	520	0.1116	-	-
0.2950	525	0.1242	-	-
0.2978	530	0.1245	-	-
0.3006	535	0.1247	-	-
0.3035	540	0.1143	-	-
0.3063	545	0.0988	-	-
0.3091	550	0.1152	-	-
0.3119	555	0.1132	-	-
0.3147	560	0.1036	-	-
0.3175	565	0.1143	-	-
0.3203	570	0.1174	-	-
0.3231	575	0.1239	-	-
0.3259	580	0.1068	-	-
0.3287	585	0.1196	-	-
0.3316	590	0.0995	-	-
0.3344	595	0.1096	-	-
0.3372	600	0.1029	0.1095	-
0.3400	605	0.0977	-	-
0.3428	610	0.1044	-	-
0.3456	615	0.1091	-	-
0.3484	620	0.0882	-	-
0.3512	625	0.1103	-	-
0.3540	630	0.1164	-	-
0.3568	635	0.1009	-	-
0.3597	640	0.1109	-	-
0.3625	645	0.1186	-	-
0.3653	650	0.1234	-	-
0.3681	655	0.1100	-	-
0.3709	660	0.0911	-	-
0.3737	665	0.1149	-	-
0.3765	670	0.1066	-	-
0.3793	675	0.1081	-	-
0.3821	680	0.1000	-	-
0.3849	685	0.1154	-	-
0.3877	690	0.0945	-	-
0.3906	695	0.1117	-	-
0.3934	700	0.0990	-	-
0.3962	705	0.1095	-	-
0.3990	710	0.1075	-	-
0.4018	715	0.1091	-	-
0.4046	720	0.1163	-	-
0.4074	725	0.1227	-	-
0.4102	730	0.1140	-	-
0.4130	735	0.1036	-	-
0.4158	740	0.1033	-	-
0.4187	745	0.0959	-	-
0.4215	750	0.1022	-	-
0.4243	755	0.1154	-	-
0.4271	760	0.1123	-	-
0.4299	765	0.1269	-	-
0.4327	770	0.0908	-	-
0.4355	775	0.1109	-	-
0.4383	780	0.0932	-	-
0.4411	785	0.1103	-	-
0.4439	790	0.0997	-	-
0.4468	795	0.1024	-	-
0.4496	800	0.1078	0.1018	-
0.4524	805	0.1023	-	-
0.4552	810	0.1048	-	-
0.4580	815	0.0976	-	-
0.4608	820	0.1021	-	-
0.4636	825	0.1182	-	-
0.4664	830	0.1105	-	-
0.4692	835	0.0913	-	-
0.4720	840	0.1023	-	-
0.4749	845	0.1065	-	-
0.4777	850	0.1075	-	-
0.4805	855	0.0920	-	-
0.4833	860	0.0886	-	-
0.4861	865	0.1115	-	-
0.4889	870	0.0931	-	-
0.4917	875	0.1074	-	-
0.4945	880	0.0815	-	-
0.4973	885	0.1105	-	-
0.5001	890	0.0874	-	-
0.5030	895	0.0952	-	-
0.5058	900	0.1005	-	-
0.5086	905	0.1074	-	-
0.5114	910	0.1025	-	-
0.5142	915	0.1058	-	-
0.5170	920	0.0998	-	-
0.5198	925	0.1071	-	-
0.5226	930	0.0998	-	-
0.5254	935	0.0946	-	-
0.5282	940	0.1022	-	-
0.5310	945	0.0935	-	-
0.5339	950	0.0920	-	-
0.5367	955	0.0801	-	-
0.5395	960	0.1100	-	-
0.5423	965	0.0964	-	-
0.5451	970	0.1017	-	-
0.5479	975	0.1102	-	-
0.5507	980	0.0897	-	-
0.5535	985	0.0976	-	-
0.5563	990	0.0907	-	-
0.5591	995	0.1040	-	-
0.5620	1000	0.0986	0.0971	-
0.5648	1005	0.0986	-	-
0.5676	1010	0.0965	-	-
0.5704	1015	0.0985	-	-
0.5732	1020	0.0887	-	-
0.5760	1025	0.0904	-	-
0.5788	1030	0.0872	-	-
0.5816	1035	0.0930	-	-
0.5844	1040	0.0894	-	-
0.5872	1045	0.0971	-	-
0.5901	1050	0.0902	-	-
0.5929	1055	0.1031	-	-
0.5957	1060	0.1078	-	-
0.5985	1065	0.1040	-	-
0.6013	1070	0.1074	-	-
0.6041	1075	0.0991	-	-
0.6069	1080	0.0810	-	-
0.6097	1085	0.0901	-	-
0.6125	1090	0.0915	-	-
0.6153	1095	0.0992	-	-
0.6182	1100	0.0984	-	-
0.6210	1105	0.0975	-	-
0.6238	1110	0.0895	-	-
0.6266	1115	0.0910	-	-
0.6294	1120	0.1136	-	-
0.6322	1125	0.0785	-	-
0.6350	1130	0.1176	-	-
0.6378	1135	0.0981	-	-
0.6406	1140	0.0915	-	-
0.6434	1145	0.0893	-	-
0.6462	1150	0.1047	-	-
0.6491	1155	0.0833	-	-
0.6519	1160	0.0938	-	-
0.6547	1165	0.0834	-	-
0.6575	1170	0.0971	-	-
0.6603	1175	0.1031	-	-
0.6631	1180	0.1049	-	-
0.6659	1185	0.0886	-	-
0.6687	1190	0.0931	-	-
0.6715	1195	0.0899	-	-
0.6743	1200	0.0847	0.0937	-
0.6772	1205	0.0917	-	-
0.6800	1210	0.1049	-	-
0.6828	1215	0.1061	-	-
0.6856	1220	0.0948	-	-
0.6884	1225	0.0936	-	-
0.6912	1230	0.0911	-	-
0.6940	1235	0.0848	-	-
0.6968	1240	0.0829	-	-
0.6996	1245	0.0885	-	-
0.7024	1250	0.0908	-	-
0.7053	1255	0.0909	-	-
0.7081	1260	0.0881	-	-
0.7109	1265	0.0875	-	-
0.7137	1270	0.0967	-	-
0.7165	1275	0.0872	-	-
0.7193	1280	0.0827	-	-
0.7221	1285	0.0894	-	-
0.7249	1290	0.1099	-	-
0.7277	1295	0.0834	-	-
0.7305	1300	0.0757	-	-
0.7334	1305	0.0802	-	-
0.7362	1310	0.0941	-	-
0.7390	1315	0.0827	-	-
0.7418	1320	0.0798	-	-
0.7446	1325	0.0919	-	-
0.7474	1330	0.0826	-	-
0.7502	1335	0.0931	-	-
0.7530	1340	0.0910	-	-
0.7558	1345	0.1067	-	-
0.7586	1350	0.0980	-	-
0.7614	1355	0.0766	-	-
0.7643	1360	0.0973	-	-
0.7671	1365	0.1020	-	-
0.7699	1370	0.0908	-	-
0.7727	1375	0.0894	-	-
0.7755	1380	0.1045	-	-
0.7783	1385	0.0817	-	-
0.7811	1390	0.0916	-	-
0.7839	1395	0.0891	-	-
0.7867	1400	0.0831	0.0916	-
0.7895	1405	0.0933	-	-
0.7924	1410	0.0920	-	-
0.7952	1415	0.0812	-	-
0.7980	1420	0.0957	-	-
0.8008	1425	0.1048	-	-
0.8036	1430	0.1032	-	-
0.8064	1435	0.1013	-	-
0.8092	1440	0.0903	-	-
0.8120	1445	0.1034	-	-
0.8148	1450	0.0917	-	-
0.8176	1455	0.0793	-	-
0.8205	1460	0.0857	-	-
0.8233	1465	0.0880	-	-
0.8261	1470	0.1008	-	-
0.8289	1475	0.1023	-	-
0.8317	1480	0.0870	-	-
0.8345	1485	0.0928	-	-
0.8373	1490	0.0806	-	-
0.8401	1495	0.0935	-	-
0.8429	1500	0.0819	-	-
0.8457	1505	0.0951	-	-
0.8486	1510	0.0994	-	-
0.8514	1515	0.0900	-	-
0.8542	1520	0.0947	-	-
0.8570	1525	0.0932	-	-
0.8598	1530	0.1004	-	-
0.8626	1535	0.0886	-	-
0.8654	1540	0.0872	-	-
0.8682	1545	0.0804	-	-
0.8710	1550	0.0803	-	-
0.8738	1555	0.0905	-	-
0.8767	1560	0.0761	-	-
0.8795	1565	0.0774	-	-
0.8823	1570	0.0893	-	-
0.8851	1575	0.0875	-	-
0.8879	1580	0.1135	-	-
0.8907	1585	0.0916	-	-
0.8935	1590	0.1039	-	-
0.8963	1595	0.1032	-	-
0.8991	1600	0.0988	0.0902	-
0.9019	1605	0.0919	-	-
0.9047	1610	0.0764	-	-
0.9076	1615	0.0842	-	-
0.9104	1620	0.0960	-	-
0.9132	1625	0.0969	-	-
0.9160	1630	0.0902	-	-
0.9188	1635	0.0996	-	-
0.9216	1640	0.0708	-	-
0.9244	1645	0.0801	-	-
0.9272	1650	0.0979	-	-
0.9300	1655	0.0828	-	-
0.9328	1660	0.0792	-	-
0.9357	1665	0.0872	-	-
0.9385	1670	0.0802	-	-
0.9413	1675	0.0929	-	-
0.9441	1680	0.1133	-	-
0.9469	1685	0.0840	-	-
0.9497	1690	0.0821	-	-
0.9525	1695	0.0894	-	-
0.9553	1700	0.0772	-	-
0.9581	1705	0.0908	-	-
0.9609	1710	0.0888	-	-
0.9638	1715	0.0806	-	-
0.9666	1720	0.0741	-	-
0.9694	1725	0.1010	-	-
0.9722	1730	0.0980	-	-
0.9750	1735	0.0944	-	-
0.9778	1740	0.0959	-	-
0.9806	1745	0.0831	-	-
0.9834	1750	0.0911	-	-
0.9862	1755	0.0842	-	-
0.9890	1760	0.1042	-	-
0.9919	1765	0.0965	-	-
0.9947	1770	0.0883	-	-
0.9975	1775	0.0784	-	-
1.0	1780	0.073	0.0898	-
-1	-1	-	-	0.6464

• The bold row denotes the saved checkpoint.

Training Time

• Training: 8.3 hours

Evaluation Metrics

The model was evaluated on a rigorously held-out validation set of synthetic municipal drafts mapped 1-to-1 against Washington State RCWs. The metrics below demonstrate the model's exceptional retrieval capabilities.

Metric	Score
accuracy@10	0.8789
Recall@10	0.8789
Recall@5	0.8027
Recall@3	0.7156
Recall@1	0.3993
NDCG@10	0.6464
MRR@10	0.5709
MAP@100	0.5757
precision@10	0.0879
precision@1	0.3994
precision@3	0.2385
precision@5	0.1605

Interpretation: When a user asks this model a legal question in plain English, there is an 87.9% probability that the exact governing state law will be returned in the top 10 search results.

Limitations

• This model does not provide legal advice.
• Performance is limited to Washington State law (RCW) and may not generalize to other jurisdictions.
• Outputs depend on the quality of the underlying document corpus.
• Should be used as a retrieval tool, not a final decision-making system.

Usage Examples

Semantic Search with sentence-transformers

import torch
from sentence_transformers import SentenceTransformer, util

# 1. Load the 16-bit fine-tuned model
model = SentenceTransformer('CSI-lab/US-state-law-EmbeddingGemma-2', 
 model_kwargs={
        "torch_dtype": torch.bfloat16,
        "config_kwargs": {
            "load_vision_encoder": False,
            "load_audio_encoder": False
        }
    }
)
# 2. Define the laws (Your Vector Database)
laws = [
    "RCW 9A.56.030: Theft in the first degree. A person is guilty of theft in the first degree if he or she commits theft of property or services which exceed(s) five thousand dollars in value.",
    "RCW 46.61.502: Driving under the influence. A person is guilty of driving while under the influence of intoxicating liquor...",
    "RCW 9A.36.011: Assault in the first degree. A person is guilty of assault in the first degree if he or she..."
]

# 3. Define the user's search query
user_query = "What dollar amount makes a theft a first degree felony?"

# 4. Encode the documents and the query with respective prompt_name parameters
query_embedding = model.encode(user_query, convert_to_tensor=True, prompt_name="SearchQuery")
law_embeddings = model.encode(laws, convert_to_tensor=True, prompt_name="Document")

# 5. Calculate Cosine Similarity
cosine_scores = util.cos_sim(query_embedding, law_embeddings)

# 6. Print the top result
best_idx = cosine_scores.argmax().item()
print(f"Top Match: {laws[best_idx]}")
print(f"Similarity Score: {cosine_scores[0][best_idx]:.4f}")

Best Practices

For optimal embedding quality and runtime efficiency, follow these configurations and best practices:

1. Task Instruction Prefixes

EmbeddingGemma 2 is trained with short task instruction prefixes prepended to text inputs. Using the right prefix improves quality; omitting it still works but reduces precision. Prefixes apply to text only. Pass images, video, and audio without any prefix.

Documents with a real title should be formatted as title: {title} | text: {content}. Use title: none when no title is available.

Prefix Notation & Usage

We offer two types of task prefixes, depending on how embeddings are used in the task. There are two types of tasks:

• Asymmetric Tasks (e.g. retrieval): Use a query prefix for queries and a document prefix for corpus items.
• Symmetric Tasks (e.g. classification, similarity): Apply the same task prefix to all inputs being compared.

Use Case	Task Type	Prompt Name	Query Task Instruction	Document Task Instruction (use none if no title)
Web / document search	Asymmetric	SearchQuery	task: search result | query: {query}	title: {title} | text: {content}
Question answering	Asymmetric	QuestionAnswering	task: question answering | query: {question}	title: {title} | text: {passage}
Fact checking	Asymmetric	FactChecking	task: fact checking | query: {claim}	title: {title} | text: {evidence}
Code search	Asymmetric	CodeRetrieval	task: code retrieval | query: {query}	title: {title or filename} | text: {code}
Text classification	Symmetric	Classification	task: classification | query: {content}	N/A
Clustering	Symmetric	Clustering	task: clustering | query: {content}	N/A
Measuring similarity	Symmetric	SentenceSimilarity	task: sentence similarity | query: {content}	N/A

Please note: prompt_name="Document" applies title: none; titled documents must still be formatted manually, like model.encode(f"title: {title} | text: {document}")

Model Citation

@misc{US_state_law_embedding_gemma2_2026,
  title={US-state-law-EmbeddingGemma-2: Fine-Tuned Dense Retrieval for Washington State La},
  author={Tomar, Shlok and Complex Social Interactions Lab},
  year={2026},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/CSI-lab/US-state-law-EmbeddingGemma-2}},
  note={Hugging Face Model Repository}
}

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

Low-Rank Adaptation (LoRA)

@article{hu2021lora,
  title={LoRA: Low-Rank Adaptation of Large Language Models},
  author={Hu, Edward J and Shen, Yelong and Wallis, Phil and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu},
  journal={arXiv preprint arXiv:2106.09685},
  year={2021}
}

EmbeddingGemma Paper

@article{embedding_gemma_2025,
  title={EmbeddingGemma: Powerful and Lightweight Text Representations},
  author={Schechter Vera, Henrique and Dua, Sahil and Team, EmbeddingGemma},
  journal={arXiv preprint arXiv:2509.20354},
  year={2025},
  url={https://arxiv.org/abs/2509.20354}
}

EmbeddingGemma 2 Official Release

@online{embedding_gemma_2_2026,
  author={Google DeepMind},
  title={EmbeddingGemma 2 is a best-in-class open model for natively multimodal embeddings},
  year={2026},
  url={https://blog.google/innovation-and-ai/technology/developers-tools/embeddinggemma-2/},
  note={Google Innovation and AI Blog}
}