Residual MLP Ensemble with RGB–HOG Feature Fusion

An ensemble of three residual multilayer perceptrons trained from scratch to classify satellite image crops into 13 xView categories. The model combines RGB pixel values with Histogram of Oriented Gradients (HOG) descriptors to capture complementary appearance and shape information. Developed as a Deep Learning course project at Universidad Politécnica de Madrid (UPM).

Architecture

Each ensemble member uses two inputs:

RGB branch: a 32 × 32 RGB image, flattened into 3,072 pixel features.
HOG branch: a 324-dimensional descriptor computed from the resized image, using 8 × 8 pixels per cell and 2 × 2 cells per block.

The two feature vectors are concatenated into a 3,396-dimensional representation and processed by dense layers of 1,024, 1,024, and 256 units. An additive residual connection combines the outputs of the first two blocks. Batch normalization, Swish activations, and dropout (0.2, 0.1, and 0.1) are used before a 13-class softmax output.

At evaluation time, the three models' class probabilities are averaged (soft voting), and the highest-probability class is selected.

Framework: TensorFlow / Keras, with scikit-image for HOG extraction.

Preprocessing and training

Images are converted to floating point and resized to 32 × 32 using bicubic interpolation.
The RGB branch applies random horizontal/vertical flips and Gaussian noise (standard deviation 0.1) during training. HOG descriptors are computed before these RGB-branch augmentations.
Optimizer: Nadam with an initial learning rate of 0.0005, cosine decay, and weight decay of 0.0001.
Loss: categorical focal cross-entropy with alpha = 0.25 and gamma = 2.0.
Class imbalance: balanced class weights, with an additional 1.5 multiplier for truck and helipad samples.
Training configuration: up to 50 epochs per model, batch size 128, and early stopping on validation loss with patience 15 and restoration of the best weights.

Results

The original notebook records the following ensemble results on the validation split:

Metric	Validation result
Accuracy	66.536%
Macro recall	69.221%
Macro precision	67.646%

These figures are calculated from the ensemble's confusion matrix in ffNN best model.ipynb. They describe the course's 13-class image classification task, not the full xView object detection benchmark.

Classes

Cargo plane, small car, bus, truck, motorboat, fishing vessel, dump truck, excavator, building, helipad, storage tank, shipping container, and pylon.

Project materials

Training and evaluation notebook: ffNN best model.ipynb.
Project report: Report_ImageRecognitionAndObjectDetectiononthexViewSatelliteDataset.pdf.

The notebook documents feature extraction, model construction, ensemble training, and per-class evaluation. The architecture classifies image crops and does not predict object bounding boxes. Results from subsequent training runs should be evaluated independently.

Authors

Melen Laclais, Léo Lamy, and Adrián García-Pozuelo Fornieles.

License and attribution

The MIT license designation applies to original project code only. Third-party code and course materials retain their respective terms.

xView imagery and annotations remain under CC BY-NC-SA 4.0, including any dataset images reproduced in notebooks or the report.

Dataset reference: Lam et al., xView: Objects in Context in Overhead Imagery (2018).

Downloads last month: -