OpenForensics Deepfake Detector (v1)
A multi-backbone CNN ensemble that classifies face crops as Real or Fake. Backbones: resnet50, vgg16. Their pooled embeddings are concatenated and read by a shared classifier head.
Output
A single sigmoid: P(Real). Fake is 1 - p.
The published threshold is 0.500 — a neutral default, not a fitted operating point. A temperature of 1.099 was fitted on validation and is applied.
Thresholds fitted on the validation split do not transfer to the test split for this dataset (see Limitations). Pick your own operating point from threshold_sweep in the evaluation report, on data resembling your deployment.
Test metrics
| Metric | Value |
|---|---|
| Accuracy | 0.8860 |
| ROC-AUC | 0.9421 |
| PR-AUC | 0.9527 |
| Real images called fake | 175 (17.5%) |
Measured on a held-out test split with horizontal-flip test-time augmentation. The split is content-hash deduplicated against train and validation, so no image appears in more than one split.
Input
Resize to 224x224, scale to [0, 1], shape (N, 224, 224, 3) float32.
Per-backbone normalisation happens inside the model — do not apply
preprocess_input yourself.
from huggingface_hub import snapshot_download
import tensorflow as tf, numpy as np, json
from PIL import Image
path = snapshot_download("adarshcod30/openforensics-ensemble")
model = tf.keras.models.load_model(f"{path}/model.keras", compile=False)
card = json.load(open(f"{path}/serving.json"))
img = Image.open("face.jpg").convert("RGB").resize((224, 224))
x = np.asarray(img, dtype="float32")[None] / 255.0
p = float(model.predict(x)[0, 0])
print("Real" if p >= card["decision"]["threshold"] else "Fake", p)
Loading needs the PreprocessLayer custom layer from
the repo, or pass it via
custom_objects.
Training data
The face-cropped OpenForensics distribution (190,334 images at 256x256). Training used light augmentation only: horizontal flip, small brightness and contrast jitter. No corruption-matched augmentation.
Limitations
- Trained on face crops. Behaviour on full scenes or non-face images is undefined.
- A score near the threshold is not evidence. Treat the margin as part of the output.
- Performance degrades on manipulation methods absent from OpenForensics.
- Research and educational use. Not a forensic authority.
- Validation does not predict test performance on this dataset. Recall on genuine images at threshold 0.5 is 0.984 on validation but 0.825 on test. The 10th percentile of scores on genuine images is 0.975 on validation and 0.138 on test — a subset of test images is confidently misread rather than the whole distribution shifting. The test split carries degradations (desaturation, noise, blocking, blur) that train and validation do not. Expect calibration to drift on degraded inputs.
Citation
Trung-Nghia Le, Huy H. Nguyen, Junichi Yamagishi, Isao Echizen, "OpenForensics: Large-Scale Challenging Dataset For Multi-Face Forgery Detection And Segmentation In-The-Wild", ICCV 2021.
- Downloads last month
- -