Verity 1.7B Alpha

Verity is the privacy analysis model developed for Veilance.

Verity is designed to compare browser-observed telemetry with a supplied privacy policy and generate a structured JSON report describing how the observed behavior relates to the policy's disclosures.

This repository contains the Verity 1.7B Alpha PEFT adapter trained on top of Qwen/Qwen3-1.7B.

Status: Alpha Base model: Qwen/Qwen3-1.7B Architecture: QLoRA / PEFT adapter Task: Browser telemetry vs. privacy-policy analysis Training examples: 69,632 Validation examples: 16,384 Training context length: 8,192 tokens Output: Structured JSON

This is an experimental research release.

It should not be treated as a legal compliance engine or an authoritative interpretation of a privacy policy.

What Verity Does

Verity receives structured browser telemetry together with privacy-policy content.

A simplified input looks like:

{
  "telemetry": {
    "...": "browser observations"
  },
  "policy_document": {
    "...": "privacy policy sections"
  }
}

The model compares observed browser behavior against policy disclosures and produces findings using comparison labels such as:

matched
partially_matched
policy_only
observed_only
possible_contradiction
indeterminate

Verity is designed for privacy-analysis tasks involving signals such as:

cookies
browser storage
IndexedDB
service workers
third-party requests
tracker detections
analytics infrastructure
advertising infrastructure
browser and device characteristics
screen characteristics
WebGL access
network-information APIs
WebRTC activity
permissions
browser APIs
fingerprinting-related surfaces

Verity is intended to reason conservatively about what telemetry actually establishes.

For example:

Observed:
www.google-analytics.com received two network requests.

Established:
The browser communicated with that host.

Not automatically established:
The contents of those requests, whether personal information was
transmitted, or the purpose for which the requests were made.

Likewise:

Observed:
WebGL renderer information was queried.

Not automatically established:
A persistent browser fingerprint was created.

Alpha Training

Verity 1.7B Alpha was fine-tuned from:

Qwen/Qwen3-1.7B

using a distributed four-GPU training run.

Dataset

Training examples:     69,632
Validation examples:   16,384
Total examples:        86,016

All prepared examples were validated before training and fit within the configured:

8,192 token

maximum sequence length.

Training Configuration

Base model:             Qwen/Qwen3-1.7B
Training split:         69,632 examples
Validation split:       16,384 examples
Max sequence length:    8,192 tokens
Epochs:                 1
Distributed world size: 4 GPUs
Training steps:         1,088
Preprocess workers:     16
Dataset workers:        4
DataLoader workers:     4

Training prompts were prepared for structured privacy-analysis generation and the model was trained to produce assistant completions rather than reproduce the entire input prompt.

Training Results

The complete one-epoch run finished in approximately:

14 hours 56 minutes

Final training statistics:

train_runtime:             ~53,780 seconds
train_loss:                0.02261
train_samples_per_second:  1.295
train_steps_per_second:    0.020

The final evaluation reported:

eval_loss:                 0.1321
eval_mean_token_accuracy:  0.9831
eval_entropy:              0.01967

Training-token accuracy near the end of the run was approximately:

0.9945 - 0.9947

The lowest logged evaluation loss occurred early in training:

eval_loss: 0.1102
epoch:     ~0.046

Evaluation loss later stabilized around approximately:

0.12 - 0.13

while token accuracy continued to increase.

These metrics measure token-level performance on the prepared Verity dataset. They should not be interpreted as a 98% accuracy rate for privacy-policy conclusions, legal interpretations, telemetry classifications, or real-world privacy findings.

Repository Contents

A typical repository layout is:

Verity-1.7B-Alpha/
├── README.md
├── adapter_config.json
├── adapter_model.safetensors
├── chat_template.jinja
├── tokenizer.json
├── tokenizer_config.json
└── test_records/
    ├── test_record_1.json
    ├── test_record_2.json
    └── test_record_3.json

This repository contains a PEFT adapter, not a standalone merged copy of Qwen3-1.7B.

The adapter must be loaded on top of:

Qwen/Qwen3-1.7B

Installation

Python 3.10+ is recommended.

Create a virtual environment:

python3 -m venv venv
source venv/bin/activate

Install the required packages:

pip install torch transformers peft accelerate bitsandbytes

A CUDA-capable GPU is strongly recommended.

Loading Verity

Verity is loaded by combining the Qwen3-1.7B base model with the Verity adapter.

import torch

from peft import PeftModel
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
)


BASE_MODEL = "Qwen/Qwen3-1.7B"
ADAPTER = "Veilance/Verity-1.7B-Alpha"


dtype = (
    torch.bfloat16
    if torch.cuda.is_available()
    and torch.cuda.is_bf16_supported()
    else torch.float16
)


quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=dtype,
    bnb_4bit_use_double_quant=True,
)


tokenizer = AutoTokenizer.from_pretrained(
    ADAPTER,
    use_fast=True,
)


base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    quantization_config=quantization_config,
    dtype=dtype,
    device_map="auto",
)


model = PeftModel.from_pretrained(
    base_model,
    ADAPTER,
)

model.eval()

If the repository has been cloned locally, replace:

ADAPTER = "Veilance/Verity-1.7B-Alpha"

with:

ADAPTER = "."

Running Verity

Verity expects a chat-formatted prompt containing:

a Verity system instruction
a user message containing structured telemetry and policy data

Qwen thinking should be disabled when supported by the installed tokenizer.

A minimal example:

import json
import torch


SYSTEM_PROMPT = """
You are Verity, the Veilance Privacy Analyst.

Compare observed browser telemetry with the supplied privacy policy.

Return only a valid JSON privacy analysis report.

Use conservative evidence-based reasoning.

Do not infer request contents, successful data transmission,
tracking purpose, profiling, legal violations, or fingerprint
generation unless the supplied evidence directly establishes it.
""".strip()


def analyze(model, tokenizer, record):
    messages = [
        {
            "role": "system",
            "content": SYSTEM_PROMPT,
        },
        {
            "role": "user",
            "content": json.dumps(
                record,
                ensure_ascii=False,
                separators=(",", ":"),
            ),
        },
    ]

    try:
        encoded = tokenizer.apply_chat_template(
            messages,
            tokenize=True,
            add_generation_prompt=True,
            enable_thinking=False,
            return_tensors="pt",
            return_dict=True,
        )

    except TypeError:
        encoded = tokenizer.apply_chat_template(
            messages,
            tokenize=True,
            add_generation_prompt=True,
            return_tensors="pt",
            return_dict=True,
        )

    encoded = {
        key: value.to(model.device)
        for key, value in encoded.items()
    }

    with torch.inference_mode():
        output = model.generate(
            **encoded,
            max_new_tokens=2600,
            do_sample=False,
            use_cache=True,
            pad_token_id=tokenizer.eos_token_id,
            eos_token_id=tokenizer.eos_token_id,
        )

    generated = tokenizer.decode(
        output[
            0,
            encoded["input_ids"].shape[-1]:
        ],
        skip_special_tokens=True,
    )

    return generated

Expected Output

Verity is trained to produce structured JSON using a report format similar to:

{
  "analysis": {
    "counts": {
      "matched": 0,
      "partially_matched": 0,
      "policy_only": 0,
      "observed_only": 0,
      "possible_contradictions": 0,
      "indeterminate": 0
    },
    "overall_confidence": 0.0,
    "summary": "..."
  },
  "domain": "https://example.com",
  "findings": [],
  "important_limitations": [],
  "privacy_policy": {
    "applicable": true,
    "found": true,
    "url": "https://example.com/privacy"
  },
  "visit": {
    "duration_seconds": 30,
    "observed_at": "..."
  }
}

Individual findings generally follow:

{
  "behavior": "...",
  "category": "...",
  "comparison": "matched",
  "confidence": 0.9,
  "description": "...",
  "explanation": "...",
  "policy": {
    "evidence": "...",
    "section": "...",
    "status": "explicitly_disclosed"
  },
  "severity": "informational",
  "telemetry": {
    "evidence": [],
    "observation_count": 1,
    "status": "observed"
  }
}

Applications should validate generated JSON before using it.

Comparison Labels

`matched`

Observed behavior appears to be directly covered by the supplied privacy-policy text.

`partially_matched`

The policy addresses the general category of behavior but does not describe the observed activity with equivalent specificity.

`observed_only`

Behavior was observed during the browser visit, but no sufficiently corresponding disclosure was identified in the supplied applicable policy.

`policy_only`

The policy describes behavior for which the supplied browser sample contains no direct observation.

This does not mean the behavior never occurs.

`possible_contradiction`

Observed behavior may conflict with an explicit statement in the supplied policy.

This label is deliberately conservative and is not a legal conclusion.

`indeterminate`

The available policy or telemetry evidence is insufficient to make a reliable comparison.

Intended Use

Verity 1.7B Alpha is intended for:

privacy research
browser privacy observability
experimentation with telemetry-to-policy comparison
development of privacy-analysis pipelines
testing structured privacy-analysis workflows
research into automated policy interpretation

It is primarily designed for use with telemetry generated by Veilance or similarly structured browser-observation data.

Known Alpha Limitations

This is an early alpha model based on a relatively small 1.7B parameter base model.

The model can produce structurally convincing output while still making reasoning or grounding mistakes.

Known failure modes include:

incorrect aggregate observation counts
combining distinct telemetry categories
failing to recognize an observed behavior
treating observed behavior as policy_only
omitting relevant telemetry evidence
over- or under-estimating disclosure specificity
selecting policy evidence that is only indirectly related to a finding
generating a policy-section name that does not exactly correspond to the supplied document
inconsistent policy evidence grounding
unsupported wording in explanations
inconsistent behavior on inputs substantially different from the training distribution

For this reason, deterministic preprocessing and post-generation validation are strongly recommended.

Applications should preferably calculate values such as:

observation counts
observed/not-observed state
host request totals
tracker totals
aggregate report counts

outside the language model whenever those values can be derived deterministically.

Policy quotations or evidence should also be verified against the policy text supplied to the model before being displayed as grounded evidence.

Important Technical Limitations

Verity analyzes observations. Those observations do not establish every fact about the underlying browser activity.

For example:

a third-party request does not establish what information was transmitted
a request to an analytics domain does not establish the contents of the request
tracker classification does not prove profiling
tracker classification does not prove targeted advertising
browser API access does not establish why the API was accessed
WebGL activity does not by itself establish fingerprint generation
canvas access does not by itself establish fingerprint generation
cookie activity does not reveal cookie ownership or values
storage activity does not reveal stored values
WebRTC initialization does not establish that media or identifying information was transmitted
absence of an observation does not establish that a behavior never occurs
a short browser visit does not represent every possible application state or user interaction

Policy analysis is also limited by the policy material supplied to the model.

Incomplete, incorrectly extracted, outdated, or inapplicable policy text can produce incorrect comparisons.

Privacy-Policy Retrieval

Verity itself is a language model and does not retrieve privacy policies.

In the Veilance architecture, policy retrieval is handled by the inference host before model execution.

A typical pipeline is:

Website
    |
    v
Playwright policy retrieval
    |
    v
Policy extraction
    |
    v
Relevant policy sections
    |
    +------------------+
    |                  |
    v                  v
Veilance telemetry   Verity
                       |
                       v
                Structured report

The model should receive policy text that has already been retrieved and validated by the host application.

Search-engine snippets should not be treated as privacy-policy evidence.

Context Length

The Alpha training dataset was validated against:

8,192 tokens

and all:

69,632 training examples
16,384 validation examples

fit within that training limit.

Deployments should still account for the combined size of:

system prompt
+
telemetry
+
privacy-policy sections
+
generated report

rather than using the entire context window for policy text alone.

For long policies, retrieval or relevance selection should be performed before inference.

Evaluation Notes

The reported training and evaluation statistics primarily measure next-token prediction performance on the prepared Verity dataset.

They do not directly measure:

legal correctness
policy interpretation accuracy
contradiction-detection precision
evidence-grounding accuracy
real-world telemetry classification accuracy
regulatory compliance

A dedicated behavioral benchmark is required to measure those properties.

Because this is an alpha release, users should independently evaluate the model against their own telemetry and policy examples before relying on it in an application.

Alpha Status

Verity 1.7B Alpha is being released primarily for transparency, experimentation, and research.

It represents an early trained checkpoint in the development of the Verity privacy-analysis system.

Later Verity models may use:

larger base models
improved telemetry normalization
stronger policy-evidence grounding
deterministic behavior aggregation
improved policy retrieval
more representative evaluation datasets
stricter output validation

Results from this release should therefore be considered experimental.

Base Model

Verity 1.7B Alpha is a PEFT adapter for:

Qwen/Qwen3-1.7B

The Qwen base model is not included in this repository and must be obtained separately when loading the adapter.

Users are responsible for complying with the applicable license and terms of the Qwen base model.

Veilance

Verity was developed for Veilance, an open-source browser privacy observability platform.

Veilance records privacy-relevant browser activity such as:

network requests
third-party communications
browser storage
cookies
browser API access
fingerprinting-related surfaces
permissions
tracking-related behavior

Verity provides an experimental analysis layer for comparing those observations against website privacy disclosures.

Disclaimer

Verity provides automated technical analysis for research and informational purposes.

It is not legal advice.

Verity output should not be interpreted as a determination of:

regulatory compliance
legal liability
wrongdoing
intent
data ownership
request contents
successful transmission of personal information
tracking purpose
profiling
fingerprint generation

Important conclusions should be reviewed against the original browser telemetry and the underlying privacy-policy text.

Because this is an alpha release, generated reports should be independently validated before being shown to end users or used in downstream decision-making.

Downloads last month: -

Model tree for Veilance/Verity-1.7B-Alpha

Base model

Qwen/Qwen3-1.7B-Base

Finetuned

Qwen/Qwen3-1.7B

Adapter

(690)

this model