Verity 1.7B Beta
Verity is the privacy analysis model developed for Veilance.
Verity compares browser-observed telemetry against a supplied privacy policy and produces a structured JSON report describing how closely the observed behavior aligns with the policy's disclosures.
This repository contains the Verity 1.7B Beta LoRA adapter and several test records that can be used to verify the model.
Status: Beta
Base model:Qwen/Qwen3-1.7B
Architecture: QLoRA / PEFT adapter
Task: Browser telemetry vs. privacy policy analysis
Output: Structured JSON
Repository Contents
Verity-1.7B-Beta/
├── README.md
├── adapter_config.json
├── adapter_model.safetensors
├── chat_template.jinja
├── tokenizer.json
├── tokenizer_config.json
└── test_records/
├── test_record_1.json
├── test_record_2.json
└── test_record_3.json
This repository contains a LoRA adapter, not a standalone merged model.
The adapter must be loaded on top of:
Qwen/Qwen3-1.7B
What Verity Does
Verity receives two inputs:
{
"telemetry": {
"...": "Veilance telemetry snapshot"
},
"policy_document": {
"...": "privacy policy content"
}
}
It compares observed browser behavior with the supplied privacy policy and produces findings such as:
matched
partially_matched
policy_only
observed_only
possible_contradiction
indeterminate
Verity is designed to analyze signals such as:
- cookies
- browser storage
- third-party requests
- tracker activity
- browser and device characteristics
- screen characteristics
- WebGL activity
- locale and timezone access
- browser APIs
- analytics infrastructure
- advertising-related infrastructure
Verity is intentionally conservative.
For example:
Observed:
connect.facebook.net received a network request
Not established:
personal information was transmitted to Facebook
Likewise:
Observed:
WebGL renderer information was queried
Not established:
a persistent browser fingerprint was created
Installation
Python 3.10+ is recommended.
Create a virtual environment:
python3 -m venv venv
source venv/bin/activate
Install the required packages:
pip install torch transformers peft accelerate bitsandbytes
A CUDA-capable GPU is strongly recommended.
Loading Verity
The model is loaded by combining:
Qwen/Qwen3-1.7B
+
Verity adapter
Example:
import torch
from peft import PeftModel
from transformers import (
AutoModelForCausalLM,
AutoTokenizer,
BitsAndBytesConfig,
)
BASE_MODEL = "Qwen/Qwen3-1.7B"
ADAPTER = "Veilance/Verity-1.7B-Beta"
dtype = (
torch.bfloat16
if torch.cuda.is_available()
and torch.cuda.is_bf16_supported()
else torch.float16
)
quantization_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=dtype,
bnb_4bit_use_double_quant=True,
)
tokenizer = AutoTokenizer.from_pretrained(
ADAPTER,
use_fast=True,
)
base_model = AutoModelForCausalLM.from_pretrained(
BASE_MODEL,
quantization_config=quantization_config,
dtype=dtype,
device_map="auto",
)
model = PeftModel.from_pretrained(
base_model,
ADAPTER,
)
model.eval()
If you cloned this repository locally instead of loading it from Hugging Face, replace:
ADAPTER = "Veilance/Verity-1.7B-Beta"
with:
ADAPTER = "."
Running Verity
Verity expects a chat-style prompt consisting of:
- the Verity system prompt
- a user message containing the telemetry and privacy policy document
The model should be run with Qwen thinking disabled.
A minimal example is shown below.
import json
import torch
SYSTEM_PROMPT = """
You are Verity, the Veilance Privacy Analyst.
Compare observed browser telemetry with the supplied privacy policy.
Return only a valid JSON privacy analysis report.
Use conservative evidence-based reasoning.
Do not infer request contents, successful data transmission,
tracking purpose, profiling, legal violations, or fingerprint
generation unless the supplied evidence directly establishes it.
""".strip()
def build_user_prompt(record):
return json.dumps(
record,
ensure_ascii=False,
separators=(",", ":"),
)
def analyze(model, tokenizer, record):
messages = [
{
"role": "system",
"content": SYSTEM_PROMPT,
},
{
"role": "user",
"content": build_user_prompt(record),
},
]
try:
encoded = tokenizer.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
enable_thinking=False,
return_tensors="pt",
return_dict=True,
)
except TypeError:
encoded = tokenizer.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_tensors="pt",
return_dict=True,
)
encoded = {
key: value.to(model.device)
for key, value in encoded.items()
}
with torch.inference_mode():
output = model.generate(
**encoded,
max_new_tokens=2600,
do_sample=False,
use_cache=True,
pad_token_id=tokenizer.eos_token_id,
eos_token_id=tokenizer.eos_token_id,
)
generated = tokenizer.decode(
output[
0,
encoded["input_ids"].shape[-1]:
],
skip_special_tokens=True,
)
return generated
Example:
record = json.load(
open(
"test_records/test_record_1.json",
encoding="utf-8",
)
)
result = analyze(
model,
tokenizer,
record,
)
print(result)
Running the Included Test Records
This repository includes three example records:
test_records/test_record_1.json
test_records/test_record_2.json
test_records/test_record_3.json
They are designed to exercise different Verity behaviors.
Test Record 1
test_record_1.json
Tests ordinary policy disclosure matching.
It contains:
- cookie activity
- browser storage
- browser/device characteristics
- third-party analytics infrastructure
- advertising-related infrastructure
The supplied policy broadly describes these categories.
This test is useful for checking:
matched
partially_matched
behavior.
Test Record 2
test_record_2.json
Tests a potential contradiction.
The policy explicitly states:
"We do not use cookies or similar tracking technologies on this website."
while the telemetry contains cookie activity.
This record is intended to test whether Verity recognizes a possible conflict between the supplied policy and observed telemetry.
A reasonable output may contain:
possible_contradiction
for the cookie-related finding.
The telemetry still does not establish cookie contents, ownership, persistence, or external transmission.
Test Record 3
test_record_3.json
Tests missing-policy behavior.
The telemetry contains substantial browser and third-party activity, but:
{
"found": false,
"applicable": false,
"complete": false,
"sections": []
}
is supplied for the privacy policy.
Verity should generally treat the policy comparison as:
indeterminate
rather than claiming that each observed behavior was undisclosed.
A missing privacy policy does not provide enough policy evidence for a normal disclosure comparison.
Quick Test Script
You can use the following script to run every test record in this repository.
Save it as:
test_verity.py
import glob
import json
import torch
from peft import PeftModel
from transformers import (
AutoModelForCausalLM,
AutoTokenizer,
BitsAndBytesConfig,
)
BASE_MODEL = "Qwen/Qwen3-1.7B"
ADAPTER = "."
SYSTEM_PROMPT = """
You are Verity, the Veilance Privacy Analyst.
Compare observed browser telemetry with the supplied privacy policy.
Return only a valid JSON privacy analysis report.
Use conservative evidence-based reasoning.
Do not infer request contents, successful data transmission,
tracking purpose, profiling, legal violations, or fingerprint
generation unless the supplied evidence directly establishes it.
""".strip()
dtype = (
torch.bfloat16
if torch.cuda.is_available()
and torch.cuda.is_bf16_supported()
else torch.float16
)
quantization_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=dtype,
bnb_4bit_use_double_quant=True,
)
print("Loading tokenizer...")
tokenizer = AutoTokenizer.from_pretrained(
ADAPTER,
use_fast=True,
)
print("Loading Qwen3-1.7B...")
base_model = AutoModelForCausalLM.from_pretrained(
BASE_MODEL,
quantization_config=quantization_config,
dtype=dtype,
device_map="auto",
)
print("Loading Verity adapter...")
model = PeftModel.from_pretrained(
base_model,
ADAPTER,
)
model.eval()
for path in sorted(
glob.glob(
"test_records/*.json"
)
):
print()
print("=" * 80)
print(path)
print("=" * 80)
with open(
path,
"r",
encoding="utf-8",
) as f:
record = json.load(f)
messages = [
{
"role": "system",
"content": SYSTEM_PROMPT,
},
{
"role": "user",
"content": json.dumps(
record,
ensure_ascii=False,
separators=(",", ":"),
),
},
]
try:
encoded = tokenizer.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
enable_thinking=False,
return_tensors="pt",
return_dict=True,
)
except TypeError:
encoded = tokenizer.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_tensors="pt",
return_dict=True,
)
encoded = {
key: value.to(model.device)
for key, value in encoded.items()
}
with torch.inference_mode():
output = model.generate(
**encoded,
max_new_tokens=2600,
do_sample=False,
use_cache=True,
pad_token_id=tokenizer.eos_token_id,
eos_token_id=tokenizer.eos_token_id,
)
generated = tokenizer.decode(
output[
0,
encoded["input_ids"].shape[-1]:
],
skip_special_tokens=True,
)
print(generated)
Run:
python test_verity.py
The model is loaded once and remains resident in GPU memory while all three test records are processed.
This is significantly faster than starting a new Python process and reloading the model for every record.
Expected Output
Verity should produce JSON containing approximately the following top-level structure:
{
"analysis": {
"counts": {
"matched": 0,
"partially_matched": 0,
"policy_only": 0,
"observed_only": 0,
"possible_contradictions": 0,
"indeterminate": 0
},
"overall_confidence": 0.0,
"summary": "..."
},
"domain": "https://example.com",
"findings": [],
"important_limitations": [],
"privacy_policy": {
"applicable": true,
"found": true,
"url": "https://example.com/privacy"
},
"visit": {
"duration_seconds": 30,
"observed_at": "..."
}
}
Individual findings contain:
{
"behavior": "...",
"category": "...",
"comparison": "matched",
"confidence": 0.9,
"description": "...",
"explanation": "...",
"policy": {
"evidence": "...",
"section": "...",
"status": "explicitly_disclosed"
},
"severity": "informational",
"telemetry": {
"evidence": [],
"observation_count": 1,
"status": "observed"
}
}
Comparison Labels
Verity currently uses:
matched
Observed behavior is directly covered by the supplied policy.
partially_matched
The policy broadly addresses the behavior but does not describe the observed activity with equivalent specificity.
observed_only
Behavior was observed but no sufficiently corresponding disclosure was identified in an otherwise applicable policy.
policy_only
The policy describes a behavior that was not observed during the sample.
This does not mean the behavior never occurs.
possible_contradiction
Observed behavior may conflict with an explicit policy statement.
This is intentionally conservative and is not a legal conclusion.
indeterminate
The available evidence is insufficient to reliably compare the policy with the observation.
Important Limitations
Verity is an automated technical analysis model.
It is not a legal compliance engine.
The following distinctions are important:
- third-party requests do not establish what data was transmitted
- tracker classifications do not prove profiling or targeted advertising
- browser API access does not establish why information was accessed
- WebGL activity does not by itself prove fingerprint generation
- cookie activity does not reveal cookie values or ownership
- storage activity does not reveal stored values
- absence of observed behavior does not establish that the behavior never occurs
- a short browser observation may not represent all application behavior
Verity output should be reviewed together with the underlying telemetry and policy evidence.
Beta Status
This is an early beta release of Verity.
The current model is based on a 1.7B parameter base model and was fine-tuned using a mixture of structured privacy-analysis examples and synthetic/counterfactual training data.
The model may:
- classify ambiguous disclosures differently from a human reviewer
- produce inconsistent aggregate counts
- occasionally over- or under-estimate disclosure specificity
- generate unsupported wording
- make mistakes on websites or browser behaviors that differ substantially from its training distribution
Applications using Verity should validate the returned JSON and deterministically verify fields such as:
analysis.counts
before presenting results to users.
Base Model
Verity 1.7B Beta is a PEFT / LoRA adapter for:
Qwen/Qwen3-1.7B
The base model is not included in this repository and will be downloaded separately when loaded through Transformers.
Veilance
Verity was developed for Veilance, an open-source browser privacy observability platform.
Veilance observes browser behavior and provides structured telemetry for privacy and security analysis.
Verity provides the analysis layer used to compare those observations with website privacy disclosures.
Disclaimer
Verity provides automated technical analysis for research and informational purposes.
Its output is not legal advice and should not be interpreted as a determination of regulatory compliance, legal liability, intent, wrongdoing, or the contents of network communications.
- Downloads last month
- 12