  We introduce a new semi-supervised, time series anomaly detection algorithm
that uses deep reinforcement learning (DRL) and active learning to efficiently
learn and adapt to anomalies in real-world time series data. Our model - called
RLAD - makes no assumption about the underlying mechanism that produces the
observation sequence and continuously adapts the detection model based on
experience with anomalous patterns. In addition, it requires no manual tuning
of parameters and outperforms all state-of-art methods we compare with, both
unsupervised and semi-supervised, across several figures of merit. More
specifically, we outperform the best unsupervised approach by a factor of 1.58
on the F1 score, with only 1% of labels and up to around 4.4x on another
real-world dataset with only 0.1% of labels. We compare RLAD with seven
deep-learning based algorithms across two common anomaly detection datasets
with up to around 3M data points and between 0.28% to 2.65% anomalies.We
outperform all of them across several important performance metrics.
