  Motivated by the recent trend of integrating artificial intelligence into the
Internet-of-Things (IoT), we consider the problem of scheduling packets from
multiple sensing sources to a central controller over a wireless network. Here,
packets from different sensing sources have different values or degrees of
importance to the central controller for intelligent decision making. In such a
setup, it is critical to provide timely and valuable information for the
central controller. In this paper, we develop a parameterized maximum-weight
type scheduling policy that combines both the AoI metrics and Upper Confidence
Bound (UCB) estimates in its weight measure with parameter $\eta$. Here, UCB
estimates balance the tradeoff between exploration and exploitation in learning
and are critical for yielding a small cumulative regret. We show that our
proposed algorithm yields the running average total age at most by
$O(N^2\eta)$. We also prove that our proposed algorithm achieves the cumulative
regret over time horizon $T$ at most by $O(NT/\eta+\sqrt{NT\log T})$. This
reveals a tradeoff between the cumulative regret and the running average total
age: when increasing $\eta$, the cumulative regret becomes smaller, but is at
the cost of increasing running average total age. Simulation results are
provided to evaluate the efficiency of our proposed algorithm.
