Topic: Computing Standard Deviation
I've trained a predictor using GluonTS on multiple related datasets. I've got a list of forecasts and timeseries that i created like this:
    forecast_it, ts_it = make_evaluation_predictions(
        dataset=test_ds,  # test dataset
        predictor=predictor,  # predictor
        num_samples=100,  # number of sample paths we want for evaluation
    )

    forecasts = list(forecast_it)
    timeseries = list(ts_it)

How do i calculate the mean squared error and standard deviation and potential other usefull metrics for evaluation.

Topic: Computing Standard Deviation
Suppose we have a job monitoring software and we want to implement a module that sends email alerts if a job takes too long to executie. The module should determine what is "too long" autonomously, based on the execution history.

We could calculate the arithmetic mean and standard deviation, and alert if the execution time is e.g. in the high 1%, but:
1) the execution time may depend on e.g. day of week (e.g. working day/weekend)
2) the execution time may have a global (upward) trend
3) the execution time may have sudden jumps due to underlying changes ("from Jan 1, we'll process both cash and card transactions, and the volume will suddenly jump 5x")

Can you outline some ideas on how to implement a system like this and address the bulleted points above?