improvements, making it work with production data