Flexible and scalable monitoring framework