hosseinamer2/zoopictures

ZooTag Model Card

Museum specimen classifier: 7 broad animal groups, transfer learning, TensorFlow Lite.

Model details

• Developed by: Group 8, Uppsala University (ZooTag Deep Learning Challenge).
• Date: 16 October 2026. Version 1.0.
• Model type: image classifier, transfer learning from EfficientNetV2B0 (frozen ImageNet backbone with a new dense head; the top 20 backbone layers were then fine-tuned at a low learning rate).
• Input: one RGB image of shape (224, 224, 3), float pixel values in [0, 255]. Normalisation is built into the model.
• Output: softmax probabilities over 7 labels, in the order AMPHIBIAN, BIRD, FISH, INSECT, MAMMAL, MARINE_INVERTEBRATE, REPTILE.
• Formats: Keras (.keras, .h5) and TensorFlow Lite (.tflite, dynamic-range quantised).
• Framework: TensorFlow 2.21 / Keras 3.

Intended use

Primary use: the Evolutionsmuseet visitor app. A visitor photographs a specimen in an exhibit and the app shows its broad taxonomic group, to encourage engagement.

• Intended users: museum visitors, through the mobile app.
• Out of scope: species-level identification, live animals in the wild, scientific or conservation decisions, and images that do not contain an animal (the model always returns one of the 7 labels and has no 'unknown' class).

Training data

Photos taken by the team at Evolutionsmuseet Zoology Section, Uppsala, on 14 October 2026, with sereva. The photos are deliberately varied: different angles and distances, visible backgrounds, reflections on the display glass, and partial views. They were sorted by hand into TRAINING and TESTING folders for each of the 7 classes, in JPG, JPEG and PNG format. Before training, 191 photos that were stored with an EXIF rotation flag were turned upright, and every image was converted to RGB JPEG.

Training set: 280 images. 15% of these were held out as a validation set for early stopping. Test set: 56 images, 8 per class, which were never seen during training. 2 training images that were byte-identical copies of test images were removed so the test set stays unseen (INSECT/IMG_8370.jpeg, MARINE_INVERTEBRATE/IMG_4346.jpg).

Class	Training	Testing
Amphibians	40	8
Birds	40	8
Fish	40	8
Insects	40	8
Mammals	40	8
Marine invertebrates	40	8
Reptiles	40	8

Training procedure

• Base model: keras.applications.EfficientNetV2B0 with ImageNet weights, include_top=False.
• Head: GlobalAveragePooling2D -> Dropout(0.3) -> Dense(7, softmax).
• Augmentation (training only): horizontal flip, rotation ±10%, zoom ±20%, translation ±10%, contrast and brightness ±20%. These simulate visitor photos.
• Optimiser: Adam, learning rate 1e-3, then 1e-5 for fine-tuning. Loss: categorical cross-entropy with label smoothing 0.1. Batch size 16.
• Early stopping on validation loss (patience 5), 30 epochs in total. Trained on CPU in a few minutes.

Evaluation

Results on the held-out test set (56 images): accuracy 87.5%, macro precision 89.2%, macro recall 87.5%, macro F1 87.5%.

Class	Precision	Recall	F1
Amphibians	0.64	0.88	0.74
Birds	0.89	1.00	0.94
Fish	1.00	0.75	0.86
Insects	1.00	0.88	0.93
Mammals	1.00	1.00	1.00
Marine invertebrates	0.89	1.00	0.94
Reptiles	0.83	0.62	0.71

TFLite conversion and verification: the saved zootag.h5 model was reloaded and converted with the TensorFlow Lite converter, using dynamic-range quantisation (6.6 MB). The .tflite model was then run on every test image. Its top prediction agreed with the .h5 model on 98% of images. TFLite test accuracy was 85.7%, and the largest difference in any class probability was 0.060, caused by quantisation. The sample below has two test images per class. The full per-image results are in tflite_verification.csv.

Test image	True label	Keras (.h5)	TFLite
IMG_8275.jpg	AMPHIBIAN	AMPHIBIAN (0.62)	AMPHIBIAN (0.65)
IMG_8277.jpg	AMPHIBIAN	AMPHIBIAN (0.98)	AMPHIBIAN (0.98)
IMG_4496.jpg	BIRD	BIRD (0.97)	BIRD (0.97)
IMG_4502.jpg	BIRD	BIRD (0.93)	BIRD (0.93)
IMG_4410.jpg	FISH	AMPHIBIAN (0.82)	AMPHIBIAN (0.81)
IMG_4437.jpg	FISH	MARINE_INVERTEBRATE (0.43)	MARINE_INVERTEBRATE (0.42)
Flipped1.jpg	INSECT	INSECT (0.79)	INSECT (0.77)
Flipped3.jpg	INSECT	INSECT (0.89)	INSECT (0.90)
IMG_4161.jpg	MAMMAL	MAMMAL (0.73)	MAMMAL (0.74)
IMG_4201.jpg	MAMMAL	MAMMAL (0.89)	MAMMAL (0.89)
IMG_4313.jpg	MARINE_INVERTEBRATE	MARINE_INVERTEBRATE (0.96)	MARINE_INVERTEBRATE (0.96)
IMG_4332.jpg	MARINE_INVERTEBRATE	MARINE_INVERTEBRATE (0.48)	MARINE_INVERTEBRATE (0.44)
IMG_4362.jpg	REPTILE	REPTILE (0.94)	REPTILE (0.94)
IMG_4364.jpg	REPTILE	REPTILE (0.97)	REPTILE (0.98)

Limitations and risks

• Very small dataset (about 40 images per class) from a single museum and with several phones. With about 8 test images per class, one mistake changes per-class recall by about 12.5 percentage points, so all the metrics are uncertain.
• The specimens are stuffed or preserved and shown behind glass. Performance on live animals, drawings or other museums is likely to be lower.
• Reptiles are the hardest class: 3 of 8 test reptiles were predicted as amphibians, which have similar body shapes and are shown in similar dioramas. The other errors are single mix-ups (fish predicted as amphibian or marine invertebrate, an amphibian as a bird, an insect as a reptile). See the confusion matrix.
• The model can learn exhibit backgrounds, for example a fish tank, instead of the animal itself.
• There is no rejection option. Photos without an animal still get one of the 7 labels, often with high confidence.

Recommendations

• Show the top 2-3 labels with their confidence, and treat low confidence (< 0.5) as 'not sure'.
• Collect more photos, especially of the weakest classes, before relying on per-class results.
• Add an 'other / no animal' class for production use.

How to use

import numpy as np, tensorflow as tf
from PIL import Image
labels = open('labels.txt').read().split()
it = tf.lite.Interpreter(model_path='zootag.tflite'); it.allocate_tensors()
img = np.asarray(Image.open('photo.jpg').convert('RGB').resize((224, 224)), np.float32)
it.set_tensor(it.get_input_details()[0]['index'], img[None])
it.invoke()
probs = it.get_tensor(it.get_output_details()[0]['index'])[0]
print(labels[probs.argmax()], probs.max())