A method for object detection, the method comprising: obtaining images of an object; defining an input layer and a plurality of sequential feature layers (74,76,80,82) subsequent to the input layer of a multi-layer feature learning network, features from the input layer provided directly to a first of the sequential feature layers, features from each of the sequential feature (74,76,78,80) provided directly to a next of the sequential feature layers (76, 78, 80) the sequential feature layers (74,76,78,80) comprising hidden layers; providing an aggregator layer receiving features directly from multiple layers of the sequential feature layers (74,76,78,80) of the multi-layer feature learning network, the features from different ones of the sequential feature layers (74,76,78,80) provided to subsequent ones of the sequential feature layers (76,78,80) and also provided directly to the aggregator layer without processing by the subsequent ones, the aggregator layer aggregating the received features; optimizing, jointly and by a processor, the multi-layer feature learning network and the aggregator layer using the images of the object, the optimizing jointly using back projection between adjacent ones of the sequential feature layers (74,76,78,80) and from the aggregator layer to the multiple ones of the sequential feature layers (74,76,78,80); and outputting, by the processor, a set of learned features represented by the optimized multi-layer feature learning network and a detector that makes use of the generated features by the optimized aggregator layer, the set of learned features being for distinguishing the object and the detector being for classifying the object. obtaining images of an object; defining an input layer and a plurality of sequential feature layers (74,76,80,82) subsequent to the input layer of a multi-layer feature learning network, features from the input layer provided directly to a first of the sequential feature layers, features from each of the sequential feature (74,76,78,80) provided directly to a next of the sequential feature layers (76, 78, 80) the sequential feature layers (74,76,78,80) comprising hidden layers; providing an aggregator layer receiving features directly from multiple layers of the sequential feature layers (74,76,78,80) of the multi-layer feature learning network, the features from different ones of the sequential feature layers (74,76,78,80) provided to subsequent ones of the sequential feature layers (76,78,80) and also provided directly to the aggregator layer without processing by the subsequent ones, the aggregator layer aggregating the received features; optimizing, jointly and by a processor, the multi-layer feature learning network and the aggregator layer using the images of the object, the optimizing jointly using back projection between adjacent ones of the sequential feature layers (74,76,78,80) and from the aggregator layer to the multiple ones of the sequential feature layers (74,76,78,80); and outputting, by the processor, a set of learned features represented by the optimized multi-layer feature learning network and a detector that makes use of the generated features by the optimized aggregator layer, the set of learned features being for distinguishing the object and the detector being for classifying the object.