Lifelong Open-Ended Probability Predictors
Authors: Omid Madani
Venue: Accepted by TMLR ()

Abstract: We advance probabilistic multiclass prediction on open-ended streams
of items.  In this setting, a predictor must emit items with
probabilities, and adapt to significant non-stationarity, including
new item appearances and frequency changes. The predictor is not given
the set of items that it is to predict a priori, and moreover the
totality of the items can grow unbounded: the space-limited predictor
need only track the currently salient items and their probabilities.
We develop Sparse Moving Average techniques (SMAs), including
adaptations of sparse EMA as well as novel queue-based methods with
dynamic per-item histories.  For performance evaluation, to handle new
items, we develop a bounded version of log-loss. Our findings, on a
range of synthetic and real data streams, show that dynamic
predictand-specific (per connection) parameters, such as learning
rates, enhance both adaptation speed and stability.

Published in Transactions on Machine Learning Research (03/2026)
Lifelong Open-Ended Probability Predictors
Omid Madani
omidmmadani@@yahooo.ccom edu
Reviewed on OpenReview: https: opwp:///eeeevvee...mmmm net/ forun? id= 6430
Abstract
We advance probabilistic multiclass prediction on open-ended streams of items. Tn this setting, predictor must emit items with probabilities, and adapt to significant non- stationarity, including new item appearances and frequency changes. The predictor not given the set of items that it is to predict a priori, and moreover the totality of the items can grow unbounded: the space-limited predictor need only track the currently salient items and their probabilities. We develop Sparse Moving Average techniques (SMAs), including adaptations of sparse EMA as well as novel queue-based methods with dynamic per-item histories. For performance evaluation, to handle new items, we develop a bounded ver- sion of log-loss. Our findings, on a range of synthetic and real data streams, show that dynamic predietand-specifec (per connection) parameters, such learning rales, enhance both adaptation speed and stability.
*Oecasionally, a new knot of significations is formed.. and our natural powers suddenly merge with a richer signification." Merlcau-Ponty (1945)
1 Introduction
The external world inherently dynamic and productive, exhibiting various forms of non-stationarity under different timescales, from periodic cycles of day and night and seasonal changes (Ingold] 2000), to unforescen phenomena including new ideas and inventions and abrupt physical and societal events. This constant change necessitates continual adaptation and renewed learning. As designers of such systems, do not have control over such ezlernal non-slationarily, but hope that those changes are not too rapid and drastic that render learning ineffective (Schlimmer & Granger 1986 Kuh et al. 1990 Helmbold & Long 1991 Widmer & Kubát, 1996 Ditzler et al. 2015 Hinder et al. 2023). In particular, in this paper we foucs on online learning, such as in user interface agents and continual personalization, where the learner should predict with probabilities (Davison Hirsh 1997 Korvernaker & Greiner 2000 Wester el. al. 2011 Mazzetto (024): Probabilit ies are useful for down-stream utility-theoretic decision making. Furthermore, complex learning systems may also exhibit internal, in particular developmental non-stationarity, their adapting subsystems come online and evolve over time. As designers, have some control over this internal change, and an important challenge is balancing system adaptability and stability. Examples of such include systems following the constructivist theories of knowledge development (Piaget 1970 von Glasersfeld 1995 Fosnot 2005), where perceptual or action patterns (schemata) are synthesized (invented') by the learning system over time (Drescher 1991
Thorisson 2012 Strannegård et al. 2018: Madani 2023). In Prediction Games in particular (Madani 2007
2023), concepts, i.e. explicit structures, serve as both predictors and predictands in the system, and the learned set of concepts grows over time. Thus, whether facing internal or external non-stationarity, the set of what is to be predicted may not be known in advance and may expand over time, í.e. open-endedness, and there can be many predictors (features) that do the predicting. The predictors should be space and time efficient, and converge fast when change occurs (sample efficient)).
We advance online fnite-memory multiclass learning methods, which we call sparse moving averages (SMAs), for probability prodiction under non-stationarity. We focus on lcarning and tracking probabilities above a minimum threshold, such as 0.001, acknowledging a trade-off between space constraints and the utility of learning lower probabilities. Key problem properties, and challenges, include:

---

Published in Transactions on Machine Learning Research (03/2026)
Lifelong and open-ended: The imput stream is unbounded, and the set of items encountered is not given to the learner and the number of unique items observed can grow unbounded.
Support non-stationarity as well as wide probability range: Item probabilities change over time, requiring timely adaptation. Subject to the finite space constraints, strive to support several orders of magnitude of probability (e.g. 10 down to 10-5).
Stability vs. plasticity: Balancing fast adaptation, for items that are changing, while maintaining stable predictions (stable probabilities) for other prodictands.
Practical convergence: Achieving balance between convergence speed and accuracy, rather than focusing on highly precise estimation in an asymptotic sense.
As concrete example of the last point, if new larget event of interest occurs about once day, then adequate accuracy, in the predicted probability, in a few days is strongly preferred over weeks or months. Sample efficiency is a necessity for autonomous continual learning. We summarize our contributions, to the best of our knowledge, and paper structure:
A novel problem setting with a formalization for space-bounded predictors facing an open-ended input stream (Sect. 2 both the set of salient items and the proportion of the times that infrequent items are observed (noise for the predictor) can change over time. We use several evaluation scores, and develop a protocol of evaluation with bounded version of log-loss for incomplete distributions handling new or infrequent items (Sect. 2.2.2 proving that the loss remains nearly proper.
We develop several SMAs, focusing on three: sparse EMA (learning-rate based), Qs (counting based) and a hybrid DYAT (Sections 3.1 3.3 These techniques, implicitly or explicitly, need to address several subproblems such as change detection and when to allocate and deallocate parameters for items (non-stationarrity and bouude-bundddeess)).. We establish a bound on sparse EMA convergence and derive a few theoretical properties of Qs. Variants of sparse EMA are used in past work, such Davison & Hirsh (1997); Korvemaker & Greiner (2000); Madani et al.) (2009a); Wester et al.)
(2011). With the novel idea of harmonic decaying of the learning rate (Sect. 3.1.2). make
connection between sparse EMA and the counting technique such as Qs (counting and windowing
are basic techniques for probability estimation, e.g. see Lewis (1998); Rosenfeld (2000); Bifet &
Gavaldà (2007); Gama et al. (2014); Ditzler et al. (2015) ). DYAL combines these ideas, enjoying
The strengths of EMA (stability, memory efficiency) and Qs (fast implicit. change detection).
particular, DYAL keeps few extra parameters, in addition to weight, per predictand (Fig. 9
A variety of experiments, in synthetic and real-world settings (Sections 5 and 6 shed light on the behavior of the different SMAs, providing evidence that DYAL often strikes superior stability- plasticity tradeoff at a cost of a small, constant, overhead.
Code, for number of SMAs including the above three, and supporting funetionality for evaluation and synthetic sequence generation, is provided
Sect. discusses related work. The earlier version of this paper provides further material including more extensive technical properties and proofs (Madani 2024
2 Lifelong Open-Ended Prediction
Our setting is online multiclass probabilistic prediction under non-stationarity. Time is discrete, t 1, A predictor processes stream of items (observations): at each lime point, it predicts (outputting semi- comi-distribution, Sect. 2.1p, then observes and updates (learns). Exactly one discrete item occurs and is observed at f. We use the parenthesized superseript notation for the variable corresponding to the observation at i, i.e. ,1), and similarly for other sequenced objects, for instance an estimated probability á at time
At. Github: https:/ http:://gthnab..om//mmaaae Tet///pareee-moing-aeerrr
2

---

Published in Transactions on Machine Learning Research (03/2026)
[o] N. ctc. infinite & finite sequences, [0] = (o _..' ol is the item observed at t.
SDs and DIs: probability semi mini-distributions (SDs) and distributions (DIs) (Sect. 2.1
SMA: a Sparse Moving Average predictor, e.g. sparse EMA and Qs predictors (Sections 3.3
EMA: : The Exponentiated Moving Average (Sect. 3.1 SMAs are sparse relatives.
P. W, Wi. W2, etc. P is used for a true underlying SD, W and W, are (SMA's) estimates.
[WN;: first N outputs of an SMA (predictor). Each W is a SD (or converted to onc).
sup(W): the support set of SD W. Every SD has finite support (subset, of all items Z).
a a(W), a(W) the allocated probability mass, Eissuprw) W(i), u(W) == 1 - a(W) (unallocated)
p² For a binary event, observation O E {0, 1}, the true probability of the positive, 1, outcome. whenever the target item of interest observed (0 otherwise).
positive outcome,
noise item: an item not seen recently in the input stream sufficiently often (Sect. 2.1.1
Pmén : used for filtering & scaling, and bounding log-loss on noise items (Sect. 2.2.2)
Punin can be interpreted as the minimum probability supported by the SMA.
CNS: (for evaluating) count-threshold used by practical noise-marker algorithm (....2p.
Omin : in synthetic experiments, the minimum number of positive observations of salient
item before it can change probability (a knob on the rate of change, Sections 2.1.1 & 5.2)
B, Imnns Smaz: ... learning rates (for sparse EMA variants), 8 E 0, 1), Bmin is minimum a alwwwed etc.
Table 1: The main notation and terminology, abbreviations and parameters.
W: 0.50
A: 0.70
input stream of observations:
H: 0.20
(m=2)
KJ AJ A A M
B: 0.10
Ø
[t=6)
B: 0.10
A.0.
Jinput
(0me)
W: 0.05
t= 700
1=99
C: 0.02
B BAJAAABAB
prediction method (SMA)
sample, repeatedly iid, from the current distribution
A:0.5 (t:))
(t=1)
Loupput
A:555
B:0.2
W ty ( 1y
BAAAXWAAB. WWBWHCZWH.
(1=8)
output stream (sequence of predictions)
(a) A stream of observations, left to right (items: B, A, J, ...), and prediction outputs, the boxes, by a hypothetical predic- tor, shown at a few time points.
(b) An SMA emits probabil- ities, in effect converting an item stream (observations) into stream of (moving) SDs, [W), f.e. the predictions sequence.
(c) (hidden) categorical distribution generates the (synthetic) data stream, in an idealized setting (IID draws), but also itself changes every so often (here, at time I = 700, 1999, etc).
Figure 1: (a) An example sequence of items together with outputs of a predictor, shown for few of the time points. The sequence is observed from left to right, thus at time t = 1, item B is observed (o(!) = B, o(3) = J, etc). The predictor emits its probabilities before observing and updating. In this example, at times t S 4, nothing predicted. At t 8, the SD WV8 is predicted, where W(8) {A:0.55, B:0.2) (Sect. 2.1. (b) An SMA, in effect, converts a stream of item observations, [o], to a stream of predictions, [W], or oF + [W]Y. (c) We imagine that the input sequence is generated by SD P that changes from time to time (item probabilities may go up or down, including the extreme cases of removal, A, and insertion, H and C, in the above) (Sect. 2.1.1
is denoted ((0), An item is identified by an integer id in the various data structures (sequences, maps, ...).
Open-endedness: an item can be viewed as belonging to an infinite unordered (í.e. categorical and nominal) set (e.g. the infinite set of finite strings). We use letters A, B, C.... and also integers 0, 1,2,3, ... (with no ordering implied) to refer to items in examples. We use the words "sequence" and "stream" interchangeably. A sequence is denoted by the brackets notation []. thus [o] denotes a sequence of N observations: [o()) o²2), [N (also shown as (2))) Fig. [a) gives an example sequence. Table provides a summary of the main notations and terminology used.
3

---

Published in Transactions on Machine Learning Research (03/2026)
2.1 Semi-Distributions (SDs)
A probability is a real number in [0, 1]. A prediction output is semi-distrbution" distribution (SD) W, is a probability-vauued function defined over an infinite item set. I = 1,, 2, ....., J. such that Vi, W(2) E [o, 1], and Σiez W(i) & 1.0. The prediction output at t is denoted W(()) We can imagine that a prediction method is any online technique for converting an item sequence, [o]]" one observation at a time, into a sequence of predictions [W]] (of equal length N) (Fig. [b)) Prediction occurs before updating: W() is output at time & before observing o(t), We use P to refer to an underlying or actual SD generating the observations in the ideal setting and synthetic experiments (see next), and use W to denote estimates and prodiction outputs (often strict SDs). In general, under non-stationarity, W() form moving (changing) SDs.
2 A (categorical) semi-
Let a(W) := Eier W(i) (the allocated probability mass of W), and let u(W) == 1 a(W) (the unallocated or the free mass). A probability distribution (DI) W is a special case of SD, where a(W) = 1. When a(W)= > 0 we say W is non-empty, and when 0 < a(W) 1, we call W a strict SD. The support, denoted sup(WW; is the set of items i with W(i) > O. We simply use [W] for |sup(W)) (the number of positive entries in the map W). The support is finite in this paper: [sup())(()] is up to 100s to 1000s in our experiments depending on the predictor's space limits (or algorithmic parameters, e.g. see Sect. [.2.1]. Let min(W)- ;= min;,sup(((a)) W(i), and defined as 0 when W is empty. A map data structure is used to implement a SD, pruned periodically. W(1) is O if item í is not in the map.
As an example SD, for P = {0:0.5, 1:0.2); P has support of size 2, corresponding to a binary event, and item 0 has probability 0.5, and has probability 0.2, min(P) = 0.2, a(P) 0.7, and u(P) = 0.3, so P is a strict SD in this example.
2.1.1 Idealized Streams and Salience
We have the following idealized non-stationarity in mind for algorithm design. The input stream is the con- catenation of stable subsequences, each generated in a stationary IID manner (independent and identically distributed), sampling from an underlying (semi-)distrribttion (Fig. [c)] 3 The underlying distributions change over time, but each stable period is assumed to be sufficiently long for learning those item prob- abilities that have changed, thus we assume per-item stability. Concretely, in synthetic experiments, we use parameter, Omine to control the degree of non-stationarity, ensuring a (salient) item (described next) occurs 2 Omin times (e.g. Omin = 5) before its probability changes. This allows us to evaluate prediction algorithms under controlled, changing distributions, reflecting real-world scenarios where periods of stabil- ity are followed by change. But real-world streams can also deviate from this idcalization in various ways (e.g. periodicities) and it is important to evaluate on non-synthesized sequences as well.
Our predictors are space- bounded and cannot learn probabilities that are too small. We evaluate the probabil- ities above minimum threshold Pmin (e.g. Pmin {0.01, 0.001} in our experiments). Items with probability above Pmin are salient, otherwise non-salient or noise. Good predictors should quickly detect and learn new salient items while, in effect, ignoring noise (infrequent items) For sequence generation using SD P (e.g. in our senthetic experiments), non-salient items are drawn with probability u(P) = 1 - a(P), often with a unique noise-item id: the item will appear only once in the sequence.
A stationary binary sequence, a special case useful in thcoretical and empirical analyses, can be generated via IID drawing from DI p = { 1: p", 0: 0-1-""}, where p" 0 is the Irue or largel probability to estimate well via predictor that processes the binary sequence. A binary sequence also arises whenever we focus single item in given original sequence containing different items 5
We are borrowing the naming used in (Dupont et al. 2005). Other names include aub-distribution and partial distribution. Thhe concatenation of stable subsequences similar the model of Plasse & Adams (019) who studied change detection for multinomial (whose item-set is known).
"The definition of noise relative to the finite space of the predictor.
5hee original sequence [o] is converted to a binary [os], [o] + [os], in the following manner, say for item A: if ol = A then
% = 1, and of otherwise (or (t) = =[[((]] = A] ]. where [[>]] denates the Iverson bracket on the Boolean condition r).
(t)
4

---

Published in Transactions on Machine Learning Research (03/2026)
2.1.2 Two Example Scenarios (Streams)
Even the ideal generation model described above contains rich challenging possibilities. We describe two simple scenarios below.
Stable-Changing. One item A is stable at constant probability of say 0.5, while other (salient) items change (remaining stable for only relatively short time span). For instance, there could be repeated switching bet ween two underlying distributions, Pi and P2, Pi = P2, where Pi = A:0.5, B:0.5, C:O] and P2 = {A:0.5, B:0, 3 (so items B and C' swap probabilities from time to time). Sect. 5.1 explores a similar sccnario involving oscillations between two probability values for the same item.
Noise-Portion Change. Note that the noise portion of the input stream, i.e. the proportion of the items not predictable by the learner, can range anywhere from 0 to 1, and can change too. Say predictor has space for keeping track of only 100 items, but for some time, e.g. till ti 10?, all items in the stream occur with probability below 0.001 say, Afterwards, for say next 100 time points till time t2, single item A becomes salient with P(4) = 0.7. This period is followed by another one where B becomes salient, at P(B) = 0.4, and A dropping to O, till 13. The noise portion of the stream, for this predictor, is then 1.0 for t S t1, then goes down to 0.3 for t E (t1, t2], and then goes up, to 0.6, for E (t2, ta] (the transitions in the modeled item-probablittes, for a good prodictor, are 0 {A:0.7] - (B:0.4)).
2.2 Evaluating Probabilistic Predictors
A prediction method, effect, converts an observed sequence [o]]", , to sequence of predictions, SDs [W] (Fig. [b) and Scct. 2.1). Here, we develop and discuss mothods for evaluating [W]] for empirical compar- ison of different SMAs.
2.2.1 When True Probabilities are Known: Deviation Rates
Whenever the probability of an item p* is not changing (in an stable period, 2.1.1 the predictor's probability outputs for that item should quickly converge p". We first consider synthetie streams, i.e. when we know the underlying probabilities. Let [p]] denote the sequence of output probabilities for specific item. For choice of a deviation bound d > 1, eg d = 2, we define the deviation-rate dev([p)] p² d) as the following sequence average:
1
dev([o]) D" pd,d)= = deviates(60) ,p'dd) (the deviation rate, with p¹ > 0)
(1)
N
(,, dd) = p 0, or (otherwise), max p' 1]
1 p D p
ist
(2)
where [[x]] is the Iverson bracket (1 when condition x true, otherwise 0). Note that when e is 0, the condition is true and deviates(O.p)) (0,pp"d))))))) d) is for any The smaller the deviation-rate the better, and we report rates for a few choices of d (Sect. 5p. More generally, we track multiple items, and we report two variants: at any time t, when the observed item's estimated probability deviates, and when any (salient) item in P suffers too large of deviation (Sect. 5.2). Deviation rates are easy to understand (interpretable). but often in real-world settings, true probabilities are not available.
2.2.2 Unknown True Probabilities: Adapting log-loss
There exist. much work in diverse domains such meteorology and finance addressing the challenge of unknown underlying probabilities. In particular, the concept of proper scoring rules has been developed to
encourage propricty, i.e. reliable probability forecasts (Bricr) 1950 Good 1952 Toda 1963 Winkler 1969:
Gneiting el al.] 2007 Filho et al. 2021 Tyralis & Papacharalampous 2022). proper rule would lead to
the best possible score (lowest if defined as a cost, as we do here) the technique's probability outputs
matched the true probabilities (assumed exist). In particular, the logarithmic loss (log-loss) incurred by candidate SD W, given an underlying DI P, is proper:
LogLoss(W[P)) := Eo-P In(W(o))
(logarithmic loss of W given P)
(3)
5

---

Published in Transactions on Machine Learning Research (03/2026)
FC(W) // Filter & cap W.
loglossRuleNS((, W, morkedN
1/ W is an item to probability map.
// Parameters: Poii The current // observation is and W is the // predictions (an item to probability map).
// Parameter: Prain E (0,1)
W <- ScaleDrop(W, 1.0, Pmin) // Filter
If a(W) 1.0 Pmin: // Already capped??
W < FC(W) // filter and cap W. prob + W get 0.0)
Return W Nothing left to do. Ponins // Scale down a.
If prob 2 Pmin: // E sup(W")?
a(w")
Return ScaleDrop(W'' a, Pmin))
Return - Lu(pprbb) // plain log-loss.
1/ & sup(W"):: the predictor treats o as // new/noise item. The noise-marker not markedNS: // referee disagrees? Return - II(mmin))
ScaleDrop(W, a, Pmmsa) // Filter and scale.
scaled t {
For item i, probability prob in W:
incur maximum loss. // They both agree o noise, so use the // unallocated mass (potentially lower loss).
prob
prob
If prob 2 Pmin // Keep the salient.
scaled[i] +- prob
Pnoize <- u(W'()) // note: u(W") 2 Pmii
Return scaled
Return (Pogorn)
(a) Filtering & capping the predictions W.
(b) When computing log-loss, handling or noise items (bounded loss when pminn > 0).
For evaluating the output probabilities: (a) CapAndFillee(( is applied to the output of any predictor, at every time point t, before evaluation. It performs filtering (dropping small probabilities below Pmin) and, if necessary, explicit capping, i.e. normalizing scaling down, meaning that the final output will be a SD W where a(W") s 1 Pmin (or u(W") Pmin)- Pmin 0.01 in our experiments. (b) Scoring via bounded log-loss, handling noise itemss: the maximum incurred loss has the ceiling - In(pmin).
Figure 2:
The loss is the expected value of the natural logarithm - II((())),, where the expectation is taken with respect to (wrt) the true DI P. This loss remains sensitive (useful) for assessing smaller probabilities, unlike quadratic (Brier) loss, another commonly used proper loss. But plain log-loss infinite on 0 probabilities and more generally, large losses on infrequent or new items can dominate overall log-loss making uninformative for comparing predictors. We also want preserve propriety as much as possible (including adequately estimating the noise portion of the stream, Sect. 2.1.2). We next explain how we adapt log-loss to handle such cases. Our evaluation protocol has 3 components:
1. Ensuring that the output map of any predictor is a SD W such that a(W) < 1- Pmin with Prein > 0 (Pmin = 0.01 in our experiments), thus some mass, u(W) 2 Puins is left for obscrving a noise item (via filtering and normalixing, the function FC() in Fig.
2. Use of simple noise-marker, 'referee", for noise judgement, such as in Fig. 3
Specification of how to score in the various cases, e.g. whether or not an item marked noise by the referee (Fig. bb), a bounded version of log-loss).
Fig. 2 a) shows the pseudocode for filtering and capping, FC(, applied to the output W of any SMA, before evaluate it. Letting W : FC(W), we have a(W") moise-marker and the FC() function, given any observation o! and the capped predictions Wy(1) (where we have applied FC() to get the SD w(tt) and given the noise status of (4) (via the moise-marker), we evaluate wi( using loglossRuleNS(). and take the average over the sequence:
Puiin and min(W") 2 Pmin [o] Once we have a
ArgLogLossNS((WHF ( N
loglossRuleNS(00) W(,), ()))(
(4)
6nn can make distinction between the minimum probability modeled and the minimum kept noise items (in FC().. When evaluating SMAs, he argued they should he the same [Madani 2024), and we one parameter Pmain-

---

Published in Transactions on Machine Learning Research (03/2026)
isNS( // whether observation o is noise or not. // Parameters: cNs (noise count threshold). flaggedNS +- recent Prrenn/PrMappgellle 0) S cNs // Increment count of o. Increment(o, recent Mapp))
Return flaggedNS // Return the or 1 decision.
Figure 3: A simple noise-marker, a referee for comparing different SMAs, to mark whether an item is noise, via a count map, and unlinited history in this picture.
The scoring rule loglossRuleNS(,, W, isN(()) checks if there is a probability p.p 0 assigned to o, p W(o). If so, must have 2 Pman (from the definition of FCO), and the loss is - In(p). Otherwise, if is also marked noise (there is agreement), the loss is In(u(W))- From the definition of FCQ, u(M) 1 1- a() 2 Pmin- Finally, if is not marked noise, the maximum possible loss, - In(Pmenn) is incurred.
There are two parameters for the noise-marker, the size of the history kept (window size) and the count threshold cNs. For simplicity, by default no bound on length of the history imposed in our experiments (unlimited window), though we report on finite referee window sizes too (e.g. counts of items seen limited the last 500 time points). By default, we set cNS 2, meaning item is marked noise iff it has been seen k S 2 so far in the sequence (so the 3rd occurrence is marked noise too): therefore, to incur loss lower than the maximum - -In(pmin), predictor should provide a probability above Pmnin. for the 4th and subsequent occurrences | When comparing different predictors, via AvgLogLos::S(), we compare on the same exact sequences (with an identical noise- mariee) We summarize the technical propertics of LogLossNS (near propriety), Appendix C and briefly discuss alternatives we considered in Sect. C.2. A basic question is whether the evaluation protocol can be further simplified.
2.2.3 Evaluating a Space-Boundee Predictor under Nonstationarity
We can use the same empirical evaluation log-loss formula of ArgLogLossNS0) (Eq. in the idealized non- stationary case, and as long as each subsequence is sufficiently long, a fast converging prediction method should do well. In synthetic experiments, we try different minimum frequency requirements, Omin: an item nceds to occur Omin times in a sequence before its probability can be changed. The lower the underlying p² the more time points required before p" is changed, as we expect one positive observation every 1 time points. Note that if Omin set too low, e.g. below 5, this would not allow sufficient time for learning a probability well, and the underlying p" loses its meaning. We want a predictor that tracks changes in salient probabilities well: high probability item or items (e.g. above 0.1) may change in probability (e.g. disappear). while a low probability item (e.g- 0.03) may remain stable for long lime, or vice versa (see Sect. 2.1.2 Good predictors should track such too (and, for example, not allocate all their mass onto the salient items). A good evaluation technique should score honest and adaptive predictors well (sce Appondix C.I
With real-world sequences, a variety of phenomena such as periodicity and other dependencies can violate the IID assumptions, and it an empirical question whether, on real sequences, the predictors compare acipattee based on their various, presumed or expected, strengths and weaknesses stemming from consideration of the ideal setting of Sect. 2.1.1 This underscores the importance of empirical experiments on different real-world sequences.
As another example, if ix' O, the predictor should provide probability the 2nd occurrence of item (otherwise it incurs the maximum loss In(pose))).
Note the caveat that propricty is developed under IID assumptious, and we utilize propricty for non-stationary streams composed of stationary subsequences (the assumption of stability per salient item, Sect. 2.1.1.. If sequence substantially violates the IID assumptions, it possible that basing comparisons on proper losses could misleading.
7

---

Published in Transactions on Machine Learning Research (03/2026)
3 Sparse Moving Average (SMA) Predictors
A moving average in its most basic form tracks the changes of scalar value by keeping recency-biased average. Here the observation at time t can be viewed as a sparse (possibly infinite) vector of Os with a single at the dimension equal to the id of the observed item (Madani & Huang, 2008 Wei 2012). and the tochniques developed and studied here are different ways of explicitly keeping several moving averages, i.c. estimated proportions of the items deemed salient, to predict the future probabilistically. The number of proportions that are tracked is kept under a limit for efficiency, i.e. kept sparse. Note that there are two major motivations for having a limited (sparse) memory: one stemming from resource boundedness (space/timm) /tmee efficiency), and another for faster adaptation to non-stationarities, sample efficiency for accurate prediction (which can sometimes trade off). We next present and develop three SMAs: one based on learning rales, another using counting (and queuing based), and, in order to more fully address the challenges of open-ended non-stationarity, a combination.
3.1 The Sparse EMA Predictor
The exponentiated moving average (EMA) update is a convex combination of the present (scalar) observation with the past (moving) average (effectively exponentially decaying the influence of the past values). The FMA update form has been found beneficial for online incremental learning, adapting to change and providing probabilities to predict discrete items, such as actions in adaptive user-interface agents and in large-scale classification and speculative execution (Davison & Hirsh 1997 Korvemaker Greiner 2000 Madani & Huang, 2008; Wester et al., 2011 I Here we further develop and analyze sparse EMA, focusing on its probability prediction capability and convergence speed. In particular, the analysis and (heorem (next) and the technique of decaying the rate and its properties, Sect. 3.1.2 (and used in Sect. 3.3 as well), new to the best of our knowledge.
Fig. 1 presents pseudo code for sparse EMA. The SMA keeps a map W, of item to probability. Initially, at t t=1, 1, W is empty. Each entry in W can be viewed as a connection, or a prediction relation, a weighted directed edge, from the predictor to a predictand, where the weight is the current probability estimate for the predictand. Prediction is straightforward: output the map's key-value can be broken into two phases, a weakening followed by a boost (or strengthening) phase, as in Fig. 5 This picture is useful when we extend EMA (Sect. [3.3). Predicting and updating take the same O(W]] time, and the map W can be pruned periodically (sec also Sect. 3.2.2 and 3.3.3). We next describe probability and SD estimation and accompanied challenges, in particular the speed of convergence us. variance, and handling change (versions of plasticity us. stability trade-offs (Abraham & Robins 2005 Mohri & Medina Mermillod et al. 2013 10 motivating extensions (Sect. 3.1.2 and 3.3
pairs. The update
2012
3.1.1
Convergence
The sparse EMA update, with 3 [0, preserves the SD property: when the probability map before the update corresponds to SD W(1), wir+ (the map after the update) is a SD too. Furthermore, EMA enjoys several convergence properties under appropriate conditions, e.g. the sum of the edge weights (map entries), a(W(!)), increases, converging to 1.0 (i.e. to a DI), and an EMA update can be viewed as following the gradient of quadratic loss (Madani & Huang 2008). Here, we are interested in convergence speed of the map weights as individual probability estimates.
In the stationary IID setting (Sect. 2.1.1 the stability period), show that the probability estimates of EMA converge, probabilistically, to the vicinity of the true probabilities. We focus on the EMA estimates for one item or predictand, call it A: at each time point I, positive update occurs when A is observed, and otherwise it is a negative update (a weakening), which leads to the stationary binary setting (Sect.
'EMA (also, EWMA) and other moving averages can be viewed as type of filter in signal theory and (time) convolution CContributors 2024), and find variely of applications, such in time-series analysis and forecasting (e.g. (WeiB) 2018 Maiti & Biswas 2015 ) and in reinforcement learning for value function updates (in temporal difference learning, (Sutton & Barto
1998)).
computational learning theory similar trade-off has been termed drift error us. statistical (variance) error Mohri &
Medina 2012 Mazzetto 2024
8

---

Published in Transactions on Machine Learning Research (03/2026)
Ema Updaate() // latest observation o.
DecayRate(3)
EmaMop op is item to probability, learning rate 3 E (0, 1/-
Other parameters: 9mn E (0, 1), the minimum allowed learning ratc. Smaz E. (0, 1]. the maximum and the initial learning rate for EMA with harmouic decay.
// Other param:: dofferronic flag.
For each item à Fmma Map:
Epl
(1 - () Mppi // Weaken.
11 Strengthen edge
(insert edge not there).
Ema M op o] Emplo 9 // Boost.
// harmonic detay.
If doloormonicDDecyy reduce rate?
B ++10
8 DecoyRate(())
(b) and Sect. 3.1.2
Return max(3, Bman)
(a) Plain EMA updating.
(b) Harmonic decay of rate.
Figure 4: Pseudo code of sparse EMA: (a) with a single learning rate 8, either fixed ("static" EMA), or (b) decayed with harmonic schedule down to a minimum Bmin ("harmonic" EMA, Sect. 3.1.2). An edge (entry in map W) created if it doesn't already exist (e.g. there are no edges at t 1). See also Fig. 5
Reservoir of svailable probability
Reservoir available probability
1.0 puer
probability flows from all into the reservoir
1.0
probability flows the observed edge
predictor
Increase labiibiiityy
decrease in availability
predictands
(a) First phase of FMA: weaken all edges.
(b) Second phase:
bnost weight of target.
Figure 5: An EMA update can be viewed having two phases: weaken then boost. In the first weakening phase, all existing edges from the predictor to the predictands, entries in the map W, are weakened. This can be viewed as probability flowing from the edges to the implicit reservoir of (unused or available) probability mass (u(W)). In the 2nd boost phase, the edge to the observed item is strengthened.
2.1.11 Let p' denote the target probability. The estimates of EMA for item A, denoted p(t), form a random walk. While often, iee at many time points, can be more likely that g(t) moves away from p", such as when á S p² < 0.5, when á does move closer to p" (upon a positive observation), the gap reduces by a relatively large amount. The following property, particular, helps us show that the expected gap, lp" -l. shrinks with EMA update, when f is not too close to p². It also connects EMA's random walk, the evo- lution of the gaps for instance, to discreto-time martingales, fundamental to the analysis of probability and stochastic processes (e.g. the gap is supermartingale when p( <p") (Williams 1991 Shafer & Vovk 2001
Lemma 1. EMA's 's movements, i.e. changes in the estimate p(0, satisfy the following properties, where 3 E [];
1. Marimum movement, or slep size, no more than 3: Vt, (p(++) - p(00 S 3.
2. Expected movement is toward : Let A (t) = p" p(t). Then,
p) (1 B)(p" - pp)
(1 (1)
3. Minimum expected progress size: With g(t) = |A [A(+1)), (B((0)) > g² whenever f.c. whenever p is not very near )*
A (t)
> B
The lemma is established by writing down the expressions, for an EMA update and the gap expectation, and simplifying (see Appendix A). It follows that ((5l++))pp()) = p*) = p- (or (((++)))A())) 0) = 0, and the erpected direction of movement is always toward p". The maximum movement property implics that when & sufficiently far, at least / away from p", the sign of A does not change, and property 2 implies that the gap is indeed reduced in expectation (property 3), which we
to show probabilistic
9

---

Published in Transactions on Machine Learning Research (03/2026)
0.375
harmanic
2gge Small prouuaiig 0.
harmorit
0.325
prbsbiitis
0.15
probubality
0.075
0.10
0.025
EMA, 0.01 EMA, 0.001 target probabiiity 0.1
FSAh probals eetinases, with barning on Jala))
Rued iite of 0.001 (niel
0.000
2000
4000
Figure 6: Given single binary sequence, 10k long, with target p" 0.1 (the true probability of observing 1), convergence of the probability estimates, f()) by (sparse) EMA variants: (a) for the static or fixed-rate version of EMA, under two different learning rates, 8 = 0.01 and / = 0.001. A higher rate, 8 = 0.01, can lead to faster convergence, but also causes high variance. (b) Convergence faster with harmonic decay.
convergence:
Theorem 1. Sparse EMA, with fixed rate of 3 (0, 1), has expected first-visit time bounded by O(B-3) to wilhin the band y² + B. The required number of updales, for firsl-visil lime, is lower bounded below by 22(81)).
The proof, presented in Appendix A works by using property 3 that expected progress toward p" is at least 3 while our random walker g(0) is outside the band (lv- - p(0) > 3). It would be good to tighten the gap between the lower and upper bounds. We leave further characterizing the random walk, to future work.
The smaller the rate 3 the longer it takes for P to get close to p". A higher rate 3, e.g. 0.01, require 10 times fewer observations than 0.001, or leads to faster convergence, for higher target probabilities (with p" >> B). On the other hand, once sufficiently near the target probability p", desire low rate for EMA, to keep estimating the probability well (Fig. G), or high rate would causes unwanted jitter variance. When we want to learn target probabilities in diverse range, such as [0.001, 1.0], we need to use a low rate to make sure smaller probabilities are learned sufficiently well. So, with a fixed rate (static EMA), accuracy on small probabilities incurs the cost of slow convergence on large probabilities, motivating consideration of changing the rate.
3.1.2 Harmonic Decay and Connection Counting
A variant of EMA, which we will refer to as harmonic EMA, has a rate decaying over time with each update, in a harmonic manner (Fig. (b)). The rate g(t) starts at a high value 3(() = .max (e.g. 8maz = 1.0), and is reduced gradually with each (positive or negative) update, via:
(
)
+1) -1
g((+1) + (00)
(harmonic decay of the learning rate: double reciprocal)
For instance, with g(1) 1, then g(2) = }. and in general g(t) 1, 1,2,3. with some floor 8min E 10 as shown in Fig. [b). Such a decay regime is beneficial for faster learning of the higher probabilities (e.g. p" Z 0.1), as it is equivalent to simple counting and averaging to compute proportions (connecting the rate-based FMA to the counting-basee methods of the next section) II This does not negatively impact convergence or the error-rate on the lower probabilities (such as p" S 0.05). In particular,
Thiss manner reducing the rate equivalent to plain averaging: one can expand and note the telescoping product, or use induction: The estimate after the update at t,t 2 2 B(t) = g(0) + (1- p(1 where B(1) 1, ,B(0) = ;
(60
1- -(t) t-1 and have ((--
(induction hypothesis), and combining, we obtain the simple average:
10

---

Published in Transactions on Machine Learning Research (03/2026)
with harmonic-decay, one requires O²2²) time points instead of (2()) for convergence to within a positive (constant) multiplicative deviation d. Note that the changing rate 3 also indicates the predictor's confidence in its current estimate, assuming the target probability does not change: the lower the compared to an estimate p, the less likely & will change substantially in subsequent updates.
Plain harmonic-decay does not completely address the challenges of open-ended non-stationarity however, as the rate is not raised once lowered: in general one needs to raise the rate to learn fast when there is (substantial) change, e.g. whenever new salient items are observed along the input stream. On the other hand, idcally, the learning of one item should not interfere with the probability already learned for another unrelated stable item (e.g. see the Stable Changiin scenario in Sect. 2.1.2). Therefore, we propose per-edge (per-predictand) learning rates along with (implicit) change detection to address these challenges next.
3.2
SMAs Based on Counting
A simple count-based approach to computing the probability of an item is to keep the count of (positive) observations of the item and divide it by the current time (i.e. the total count). This proportion estimator is unbiased with minimum variance (MVUE), and is also the maximum likelihood estimator (MLE) (Hogg
et al. 2018 Devore 2016) (see also Appendix B This simple running-average approach is not sufficiently sensitive to changes in the proportion of a target item (and assumes we know which items to keep track of).
We need a way to keep track of only recent. history or limiting the window over which we do the averaging. Windowing is a basic tool used for non-stationarity, e.g. (Bifet & Gavaldå 2007 Gama et al.) 2014 Mazzetto 2024). A challenge is that we are interested in a fairly wide range of probabilities, e.g. from 1.0 to 0.001, and a fixed history window of size k of all last k observations, "box" predictor (Sect. B.2 is spatially wasteful and slow to adapt.
3.2.1 The Qs SMA (Count Queues)
The Qs predictor keeps map, qaa of item to queue, where each queue is small list of counts (Fig. 7 At each time i, after outputting predictions using the existing queues, it updates the queues: First, if the observation (( does not have a queue, queue is allocated for it and inserted in g.Mop. For any queue corresponding to an item i * o (( is negative observation, and negative update is performed, while a positive update is performed for the observed item (f (for a positive observation). Every so often, the map is pruned (Sect. 3.2.2). Operations on a single queue are described next (Fig. b 12
A new cell, cel10, at the back of the queue, allocated each time a positive observation occurs, and its counter initialized to 1. With every subsequent negative observation, until the next positive observation, cell0 increments its counter, Cg. The other queue cells are frozen (their counts, C, not changod). Upon posilive observation, before a new back cell (cell0) is allocated, the existing cello, if any, is now frozen (becomes cell1), and the oldest cell, cellk, is discarded if the queue is at capacity qcap. Thus, each cell corresponds to one positive observation and its count is the time 'gap" between one positive observation and the next. Queues of different items with different probabilities, correspond to different lengths of history, the lower the probability, the longer the history (but the same maximum number, qcap, of positive observations)
Recall that a value p derived from a statistical experiment (a random variable) is called an unbiased estimate of p² if E(p) = p", and we call it an upper estimate if E(p) p" (a biased estimator). Lemma summarizes a few properties on few ways that probabilities could be derived from the queue cells, which informs algorithm design (e.g. when to drop items, Sect. 3.2.2). In particular: A single cell's count yiclds an upper estimate, while we can obtain an unbiased est istie from two or more frozen cells.
Lemma 2. Assume a salient item has posilive probubility p < in a time spon T, and its queue cells were allocated and updated in the spon T, cell having count C..
For any queue cell i, T. is an upper estimate of p".
( = o/) Note that the naming 'harmonic EMA' is somewhat inaccurate: this is no longer exponentially weighting of the history (only when the floor 8min > is reached). 12ADWIN (Bifct & Gavaldä 2007), , also based qucuing, could be used here instcad of simpler counts queue. ADWIN is designed for computing mean variances on more general numeric (not Boolean) sequences. We compare with ADWIN
Boolean sequences in Appendix
11

---

Published in Transactions on Machine Learning Research (03/2026)
PredieeUUpaateeViaaa(()))
Queue(cap = 3) // Allocates a queue object, // with various fields (capacity, cells, etc.)
Imput is sequence [o] = [o(1))(()),. ,..)),.....
gdfap ( // An emply map, (lem->quee
g.goap < cap // max size cap, cap > 1. 1/ Array (or linked list) counts.
t 0// Discrete time.
Repeat // Increment time, predict, observe.
q.cella +- [0.....0]
t [ ++1 // Increment time. // Get the predictions. // Use observation at time t, o!t), to
// Current size or number cells (s cap).
GecPPedictiona((MMan
g,nc +
Return 9
Update-Qucess(MMep o(9) update gMap. If % 1000 // Periodically prune Mapp. PruneQs(qMMap) (a heart-beat method).
GetProb(4) // Extract probability from the // queue
1f q.ne S 1: Too few cells (grace period).
GedPredictions(gMep) // Emit probabilities. // Returns a map: item + probability.
Return q.nc-1
Return GetCoumt(g)-1
W + (} // allocate an empty map. For each item i and its queue q in gMap:
Get.Count (g) 1/ Get total count all cells q- q.cells[i)) // sum all cells. 0SS5quNe
// One could remove probabilities here.
Return
Σ
Wii <- CetProb(g)
Return W // The final predictions.
NegativeUppdated) // Increments cello count. I1 The back (latest) cell of is incremented. If g.ne > // Do nothing, if no cells.
TppdateQueuesgMMpp. o) // latest. observation If item o & 9Mpp: // when gaMa, insert.
q.cell[o q.eells 0] +
// Allocate & insert for o.
DositiveUpdaterr
// Adds new (back) cell. // Existing cells shift one position. Oldest cell is // discarded, in effect, when g is at capacity.
gMaplol - <- Queuc()
For each item i and its queue q gMap:
If #
If q.ne < 9.9ccap:
If All but one will negative updates. NegativeUpddate()) // Increments count.
q.nc +- q.nc + // Grow the queue q. For in ... .1............ .g.cc 1}: // Inclusive.
Else: // Exxccly positive update.
q.celle(s] g.cellali 1] // shift (counts).
// Add new cell, with count PositiveUpddate())
q:cells(0) // set newest, coll0, to count
The main functions of the Qs predictor (left), and individual queue operations (right). The Qs SMA keeps one-to-one map of items to queues, g.app: item + queue, where each queue a small list of counts. The counts of queue are combined to yield a probability for the corresponding item.
Figure 7:
For a set S of k 2 2 cells, let Ratio(S) Ratio(S):== k-1 -1 Lie C.' If cello E S, then Ratio(S) is an upper estimate Pf and cell0 S (S contains frozen cells only), then Ratio(S) is unbiased, in particular the minimum-variance unbiased estimator of p"
=
The lemma is established by connecting the cell counts to geometric and negative binomial distributions (Appendix B. and Madani (2024]). To output a probability, for each item that has a queue, we currently use the upper estimate Ratio(S), where all the cells of the queue (including cel10). With our GetProb() function, the Qs technique needs to see two observations an item, in sufficiently close time proximity, to start emiting (positive) probabilities for the item (for an unbiased estimate we would need 3). Our exper- iments did not show a significant difference between including us. excluding cell0 (possibly lower variance when including cell0). The probabilities over all the items in 4a may sum more than 1, violating the SD property. For evaluations, we apply the FC() function in Fig. aaa). Appendix presents a few variants of queuing.
3.2.2 Map Pruning in Qs, and Computational Complexity
The least frequent items are removed from the map every so often, for instance via a method that works like "heart-beat" pattern: map expansion continues until the size reaches or exceeds a maximum allowed 2 capacity 281, e.g. $1 Pmin (some multiple of - We can then remove the items in order of least Pmizy
frequency, f.e. rank by descending count Co in cell0 of each queue, and remove until the size is shrunk back to 81. Lemma 3 is useful in justifying how prune:
Lemma 3. At any time t, across all items with a queue, there is exactly one item, with cel1o count Co For any integer > 1, there is al most one ötem with Co
12

---

Published in Transactions on Machine Learning Research (03/2026)
UpdateDyal(o) // latest observation o.
Eddeen
// Data structures: gp, EmaMap, rateMap.
Weaken weights except for item o. Data structures: qddap, EmaMap and rate.ap. Parameter: Pmni- For each item and rate & in rate If == 0: // Don't weaken o
qPR. gcount < GetQinfo(qMd ap.
UpydateQuennesqMMappp o)
WeakenEdges(o). // Weaken, except for o.
gPR 0: // item is currently noise?
Return
Contimue
emaPR < EmaMap..etto, 0.0) // o if g Map.
1/ Possibly listen (switch) to the queue. PRR, gcount Get (ettQino(gAMa. If Low(EmaMMal)) (i), gPR. goount);
1/ listen queue?
If Q High(ema PPR, gcount):
1f sest initial rate and probability using queue for o.
EmaMap
qPR // Set to q info.
rateMapío] + -countt
Mapl [1] +- gcount
6 + t-gPR emalR
Else: 11 weaken as in EMA.
Else: //EMA update with harmonic decay.
Ema 1] + (1 - B) EmaMapli]
3 rateMaplol
rateMap(1] Decaylaate(3)
6 - (1 emaam)aRRB. 8
Mapl + RecayRate(() Sect. 3.1.2
GetQInfo(gMap. o)
EmaMaplo] 8 emaPR
If o gMap: // queue for o?
Return 0, 0
Q High(ema PR, gPR, gcount) // Uses Binomial Tail
gMap.
1f Parameter: sig thrsh (significance threshold).
Return CetProb(q), GetCount(y)
If em == 0: // when 0, listen to Qs
Return True
Q Low(emaPR, RRR, geount) // Uses Binomial Tail 11 Parameter: sig thrsh (significance threshold).
1PR S emaPR:
Return Palse
If emaPR S qPR:
// Use (approximate) binomial tail (Sect. 3.3.1 Return gcount LlgPR emaPR) 2 sig thrsh
Return False Return gcount KK RR, emaPR) 2 sig thrsh
Figure 8: The DYAL SMA where the weaken-and-boost update logic is similar to sparse EMA. Here each prediction edge has a small queue and its own learning rate, in addition to the probability estimate (a weight) (see Fig. g and during weakening and boosting, the queue estimates could be used to reset both the weight and the learning-rase of the edge.
The proof follows from how the SMA operates: exactly one item is observed at any time (one, new, cello with value 1) all existing cell0 counts are incremented. Some count values may be 'missed' (i.e. cell0 values may not form consecutive increasing integer soquence) because the same item can be observed at different times. The complete proof and further properties the spread of the counts, across all the cells in all the queues (not just cell0), appear in Madani (2024). Therefore, when pruning, an item in position le in the sorted list has co 2 k. One can use the binomial tail to see that the probability that a salient item with true probability p (p 2 Pmmn) is kept and tracked, rapidly goes up the farther p is from Pmin- For instance, for an item with p 2 0.01 the probability that the item in not observed 10 consecutive time points is below 0.01 (sorting and keeping multiple of 1_ is unlikely to prune salient items). See Sect. 3.3.1 for further on the binomial tail. Note that an item that is dropped could be added later. Temporary but possibly long streaks of certain events (e.g. due to non-llI causes) can lead to poor estimates or incorrect pruning, and to address such possibilities techniques such as multilevel updating may be useful (e.g. see Madani et al. 2026)).
Paicn
Count values in queue cells can also be periodically reset to prevent numeric overflow. Predicting and updating take the same Oggaapa apl) O geep) ) (or just O (( )), where we assume 0(1) time for Ponin
summing numbers and taking ratios (qcap is small constant, such as 3 or 5).
3.3 DYAL: Combining Sparse EMA and Qs
The Qs technique can give good roughly unbiased initial estimate of the probability of an item, but it has high variance (e.g. see Fig. (0(c)) unless we use considcrable queue space (many queue cells, and trade off plasticity for stability). With sparse EMA (Sect. 3.I,, and with our assumption that there will be a period of stability when a salient item is observed (stability for that item), one could begin with the Qs estimate and fine tune it to achieve lower variance the estimation. We next show that these complementary benefits of each approach can indeed be put together, and we call the combination DYAL, for dynamic adjustment of the learning (rate), i.c. dialing up and down the learning (Fig.
13

---

Published in Transactions on Machine Learning Research (03/2026)
An update in the DYAL SMA has the simple logical structure of the plain (sparse) EMA, f.c. weaken-and- boost (Fig. 5): weaken all existing edges, then boost (strengthen) the edge to the observed item (Sect. 3.1 However each edge in DYAL, in addition to its weight (a scalar probability), also has other parameters, in particular its own learning rate, which is used for the weakening and boosting of that edge. Fig. 9 shows the parameters kept with each edge, for the three main SMAs of this paper. Associated with each DYAL odge, there is also a queue, and during weakening or boosting an edge, it is possible that the queue estimates
are used: & probability as well as a learning rate are derived from the queue. We refer to this possibility as listening to the queue (Sect. [3..1]. Each learning rate is decayed, via harmonic decay (Sect. (3.122) after an edge update. For visualization of the change patterns in learning rates, see Fig. 10fe) e) and (f) and Fig. 15 The queues are also used for deciding when an item should be discarded. Thus the queues are used as "gates" the interface to the external observed stream, determining what is kepl. track of and what is discarded, and providing rough initial estimates, of the probability and 3, whenever the probability of an item needs to substantially change. Note that in the pscudo code of Fig. [ we use three maps (EmaMap, rateMap, aa),
while in the Fig. (( picture they are combined into one logical graph.
Prediction in DYAL is as in plain EMA, and the EmaMap is output. Like Qs, DYAL does not ensure that the probabilities in EmaMap form an SD, and we apply the normalizing FC( function for evaluations 13
When updating, all the three maps are updated in general, as given in the UpdateDyal() function. Upon each observation o, first the queues information on o is obtained. This is the current quene probability qPR (possibly 0) on O, and the gcount (via GetCount()). Then aaap is updated (a Qs-type update). Finally, all the edges (entries) in the EmaMap, except for o, are weakened. If there was no queue for o, í.e. when qPR is O, o must be new, or noise in general, and nothing more is done, i.e. (FMA) edge strengthening is done (note that queue is allocated for o upon the q.aa update). Weakening, for each odge, is either a plain EMA weakening, or the queue estinates are used, covered next in Sect. [33.1] Similarly, for strengthening, the condition for listening to the queue is examined, as described next, and the appropriate strengthening action is taken, i.e. plain EMA strengthening or listening to the queue.
We can verify from the logic of DYAL that in the stationary setting the probabilities in EmaMap should converge to the true probabilities, except there is some low probability that once in a while resetting to the queue may occur (Sect. 3.3.1 (the queues contain higher variance estimates). We expect that applying the normalizing FC() does not affect the stable salient or higher probabilities substantially (whenever every salient item is imated well), but specially at times of change and resets to queue estimate, where high variance (inaccurate) estimates for one or a few items are used, the effect on stable can be higher. We leave a careful analysis of interactions amongst the probability estimates to future work.
3.3.1 When and How to Listen (Reset) to the Queue Estimates
The queue for each item provides the number of positives observed recently and the total count across cells f.e. the number of trials (roughly, the GetCount() function). A probability estimate qPR is derived from these numbers (GetProb(). We also have an estimate of probability, emaPR, from the EMA weights EmaMap, which we expect to be generally more accurate with lower variance than qPR, but from time to time change occurs and this estimate becomes out-dated and should be discarded (reinitialized). Based on the counts and the queue estimate qPR, we can perform a binomial-tail test that asks whether, when assuming emaPR is the true probability, one can observe the alternative probability pPR in gcount trials, with some reasonable likelihood. The tail can be approximated efficiently in O(1) time when one has the number of trials and the observed probability gPR (Arratia & Gordon 1989 Ash 1990), yielding a basic
tool with diverse applications such as in text analysis (Madani 2021). The tail tells us how likely it is that a binary event with assumed probability emaPR could lead to the observed qPR proportion. As seen in Fig. g the approximation is based on the KT. divergence. When this event sufficiently unlikely, or, equivalently, when the binomial-tail score surpasses a threshold (default 5) DYAL resets to the queue estimate and sets new rate 3 accordingly too (Sect. 3.1.2 Thresholds of 3, 5 (default), and 7 correspond to roughly 90%, 99%, and 99.9% confidences respectively (99% means 0.01 chance of making a false-alarm error, i.e. resetting
In carlier version of DYAL it was cusured that DYAL's output would be SD by transferring probabilitics among the predictands more carefully, requiring extra logic Madani 2024 However leaving this operation to FCQ leads significantly simpler technique, and our experiments do not show a large uniform difference wetween the two versions.
14

---

Published in Transactions on Machine Learning Research (03/2026)
predictands (items) (up to 100s with - Psr 0.01)
learning rate (per odge)
probability
queue
B
(ouend queue
predictor
A
A
A
G
F
(a) A (sparse) EMA map
(b) A Qs map
(c) A DYAL map
Figure 9: Comparing the parameters of each prediction edge by the main SMAs of this paper (3 predictands shown in cach map: items A, G, and F). (a) Sparse EMA maintains single weight, ie. probability we: with each edge, and uses a single learning rate independent of the edges. (b) The Qs SMA (center), keeps and updates a small queue of counts for each edge (integers, up to 3 queue cells in this picture), and has no (explicit) lcarning-rate. (c) DYAL, a combination of EMA and Qs keeps a probability we: a learning-rate Se, and a small queue for each edge. The rate Be normally decays with each update, but at times Be along with we are reset to the estimates derived by the edge's queue (Sect. 3.3
while the EMA estimate should have been kept) See Sect. 5.1.2 Lowering the threshold moves DYAL closer to Qs (higher variance, but faster reaction to change), and increasing the threshold takes it closer to the static sparse EMA (lowers false positives but increases the miss possibility, or the run length), tradeoff common in change detection tasks (Aminikhanghahi & Cook 2016 van den Burg & Williams 2022).
3.3.2 Queue Size and the Granularity of Probabilities
A single queue cell can only provide (over-estimate) probabilities in {1, 1/2, 1/3, ...} (many fractions such as 0.75 are not not representable). Using multiple queue cells generates more variety [Madani 2024). The EMA estimate
also better estimate finer details, as long the learning rate is appropriately small. However, if the true probability of an item goes from say 0.5 to 1.0, the EMA part of DYAL may not adapt sufficiently fast if it has a low learning rate at time of change. The binomial tail test with a conservative threshold (such as 5) and a small queue may not detect the change either. This motivates additional queue cells (e.g. 5 queue cells can be better than 3, e.g. see Fig. b)) as well as keeping a positive floor on the learning rates (the harmonic docay should not go to zero when change is possible)
3.3.3 Computational Complexity of DYAL
The pruning logic of gaap in DYAL is identical to the Qs (Sect. 3.2.2 except that whenever entry is deleted from the queue, its corresponding entries (key-value pairs) in rateMop and EmaMap ap are also deleted, thus ag 2 MrpleMapp = |EmaMap]. Consequently asymptotic space and update times are the same as for Qs and sparse EMA (a fixed overhead).
4 Experimental Set Up
Filtering and normalizing (i.e. the FC() function of Fig. 2 default Pmin = 0.01) is applied to the output of all predictors, for evaluations at every time point. The default parameters for DYAL are qeap of 3 and binomial-tail threshold of 5, and we report the performance of DYAL for different Bmin, often set at 0.01 or 0.001. We are interested Amin < 0.01 because our target range learning probabilities in roughly [0.01, well. For the static (sparse) EMA, we report and change the (fixed) B used, for harmonic EMA
As in Sect. 3.1.2 with each edge having its own rate, the rate rateMapli] can be used as measure of the predictor's uncertainty around the probability estimate EmaMapli]. Initially, when set to the queue estimate, the rate can be relatively high, and is lowered over time (increased confidence in the estimated probability) with rate decay.
15e floor Amin on the learning rate could be function of the current probability estimate P (e.g. & rather than constant (which the case this paper). This provides more plasticity (faster convergence), but at the cost higher variance.
15

---

Published in Transactions on Machine Learning Research (03/2026)
deviation threshold->
1.5
2
1.5
2
change type and Omin
Qs,
Qs.
10
0.025 = 0.25, 10
0.423 + 0.029
0.189 = 0.023
0.395 - 0.021
0.234 + 0.014
0.025 = 0.25, 50
0.382 = 0.038
0.131 = 0.028
0.222 = 0.039
0.062 - 0.017
(4(0.01, 1.0), 10
0.443 + 0.030
0.234 = 0.042
0.468 = 0.028
0.284 = 0.035
U(0.01,1.0)) 50
0.380 + 0.050
0.173 - 0.080
0.247 - 0.060
0.102 + 0.076
static FMA, 0.01
static EMA,, 0.001
0.025 5 0.25, 10
0.510 + 0.024
0.357 + 0.020
0.996
= 0.006
0.760 =: 0.043
0.025 _ 0.25, 50
0.255 + 0.045
0.129 + 0.028
0.705 + 0.032
0.560 + 0.023
((000111..0) 10
0.686 + 0.030
0.477 + 0.035
0.818 + 0.038
0.693 + 0.054
4(0.01,1.0), 50
0.397 + 0.056
0.209 = 0.045
0.775 = 0.085
0.602 + 0.099
harmonic
EMA, 0.01
harmonic
EMA, 0.001
0.025 = 0.25, 10
0.502 + 0.025
0.351 = 0.021
0.957 = 0.021
0.668 + 0.053
0.025 = 0.25, 50
0.247 + 0.046
0.123 = 0.028
0.606 = 0.031
0.494 + 0.018
((00.0,,10); 10
0.684 + 0.033
0.476 = 0.034
0.813 = 0.036
0.683 + 0.050
u(0.01, 50
0.395 + 0.056
0.204 - 0.043
0.761 - 0.079
0.592 + 0.097
DYAL,
0.01
DYAL,
0.001
0.025 5 0.25, 10
0.480 + 0.029
0.332 = 0.025
0.586 = 0.066
0.408 + 0.061
0.025 0.25, 50
0.251 = 0.048
0.150 = 0.031
0.099 = 0.066
0.053 = 0.037
1.0, 10
0.560 + 0.035
0.351 = 0.054
0.529 = 0.052
0.313 + 0.045
4(0.01, 50
0.320 = 0.11
0.189 = 0.128
0.246 = 0.081
0.128 + 0.083
Table 2: Synthetic single-item non-stationary: Deviation-rates (lower is better) where p" either oscillates back and forth between 0.025 and 0.25 (0.025 = 0.25), otherwise is drawn uniformly at random from [0.01, 1.0] (p" ~ (10.001, 1.0)). Each sequence is 10k long, and the deviation-rate is averaged over 500 such sequences. For 0.025 = 0.25, each subsequence (wherein p" is constant) is roughly same length: 400 long for Omin = 10, and 2k for Omin = 50. For (/(001,, 1.0), each subsequence only has to meet the Omin constraint (see Sect. 5.1p. Qs SMAs do best here, and DYAL variants do equally well or come close (in terms of both their best- and worst-case performance).
and DYAL, we change the 8min; and for Qs, the qcap. The referee (noise-marker) uses default cNs 2 and log-loss (in tables and figures) refers to AvgLogLossNS (Sect. 2.2.2 The map size is periodically checked and if above 225 entries, is pruned to 150 (a multiple of Pmnin: Sect. 3.3.3h. All code is in Python (Madani 2024). all paired comparisons, comparing two SMAs on tens of 10k long sequences, take no longer than a minute on an Apple laptop. DYAL is roughly 5x slower than sparse EMA (neither optimized).
5 Synthetic Experiments
Here, we generate sequences knowing the true SDs p(t), D.I] deviation rates when tracking single item in a stationary setting. In such stationary sctting, we find (sparse) EMA variants, including DYAL, do significantly better than Qs, although, for static EMA, the appropriate setting of the fixed rate is
an issue (Sect. 3.1.1p. We next closely examine the behavior of the SMAs, specially DYAL (Fig. 10), when the probability of a single item repeatedly changes, before looking at performance on a multi-item scenario.
5.1 Tracking Single Item, Non- non-Stationary
Here, sequences of 10000 items are generated in cach trial. We report on two main settings for non- stationarity: In the first setting, p" oscillates between, 0.25 and 0.025, thus an abrupt 10x change is guaran- teed to occur and frequently (the Stable- Changl-n example scenario of Sect. 2.1.2 focused on one changing salient item). This oscillation is shown as 0.25 0.025 in Table In the second 'uniform' setting, cach time p' is to change, we draw a new pd uniformly at random from the interval |0.01, 1.0], shown as U(0001, 1.0), and in this setting, some changes large, others small and could be viewed "drifts". The stable period,
16

---

Published in Transactions on Machine Learning Research (03/2026)
during which p" cannot change (to allow time for learning), is set as follows. For both settings, within a stable period, the target item (item 1) has to be observed at least Omin times before p" is eligible to change, where results for Omin E {10,50) are shown in Table 2 Additionally, we impose a general minimum- length constraint (not just on positive observations) for the 0.25 = 0.025 setting: cach stable period has to be Omin
mmnn000025.0.25) 80 that the different periods would have similar length (expected 400 time points when Omin = 10, and 2000 when Omin = 50). In this way, subsequences corresponding to p = 0.25 would not be too short (otherwise, reviation-rate performance when p" = 0.025 dominates). Thus, with 0.25 = 0.025, we get an expected 25 stable su subsequences (or changes in p²) in 10k long sequences with Omin = 10, and 5 switches in p when Omin = 50. For the uniform setting, we did not impose any extra constraint, and respectively with Omin of 10 and 50, we get around 200 and 50 stable subsequences in 10k time points. In this setting, performances in periods when p" is low do dominate the deviation rates.
Here the simple Qs SMA excels, specially when 0min=10. With such non-stationarity, qeap -5 can even work better than qcap =10. In this non non-stationary setting, DYAL is the 2nd best, and performs substantially better than the other EMA variants. Omin of 10 may still be considered on the low side (i.e. relatively high non-stationarity). With Omin 50, every predictor's 's ieviation-rate improves, compared to m sctting. DYAL, with 0.001, performs best for the 0.25 ++ 0.025 and Qs, =10, works best with uniform p"; DYAL remaining second best. When picking from uniform, the change p may not be high, Qs can pick up such change faster, and more naturally, than DYAL, which performs explicit tests and may or may not raise its learning rate depending on the extent of change. However, Qs with 10, despite its simplicity, does require significant extra memory (and we'll find that in the multiple items settings, no setting for Qs works better than DYAL). Appendix B.3 contains comparisons to ADWIN (Biftt& & Gavaldà 2007) in this setting,
5.1.1 Behavior of the Output Estimates of the Various SMAS
Figs. 10 shows plots of the output estimates p(0) along one of the oscillating sequences (0.25 ++ 0.025). As expected, we observe that the Qs technique leads to high variance during stable periods, qcap of 5 substantially higher than qeap of 10 (Fig. [0(d)). Similarity for both (static) EMA and DYAL, the rate of 0.01 exhibits higher variance than 0.001 (Figs. [0(b) and (c)). Static EMA and DYAL with 0.01 are virtually identical in Fig. [0(,), except that DYAL can have discontinuities when p' changes (when it listen to its queue estimates, as discussed next), and converges from both sides of the larget p", while static EMA converges from one side.
5.1.2 Change Causes Spikes in the Learning Rates of DYAL
Fig. 10ke) shows the evolution of the learning rate of DYAL, and in particular when DYAL detects a (substantial or sufficiently significant) change (shown via red dots near the x-axis). At the scale shown, rate increases look like pulses: they go up (whenever DYAL listens to a queue) and then with harmonic decay, they fairly quickly come back down (resembling spikes at the scale of a few hundred time points, for probabilities in [0.01, 1.0)). We also note that this rate increase can happen when a high probability (0.25) changes to low probability (0.025) as well, though the rate increase is not as large, would be expected. DYAL is not perfect: Not all the changes are caught. (via switching to its queues' estimates, Sect. 3.3.1 in particular, there are a few "false negatives" (misses), when a change in p' occurs in the sequence and DYAL continues to use its existing learning rates, slowly adapting its estimates. This may suffice when the 8 at the time is sufficiently high or when the EMA estimate is sufficiently close. In particular, we can observe in Figs. 10 and (c) that a higher rate, 3min = 0.01, leads to fewer switches compared to 0.001, and in Figs. 10 we observe that DYAL behaves almost identically to static EMA when both have 0.01 (there appears to be one switch only at 15000 for DYAL). Fig. f) presents close up of when (significant) change is detected and switch does occur, from a low true probability of p* 0.025 to p" 0.25 in the picture. Note that before the switch, the estimates (the blue line) start rising, but this risc may be too slow. Once sulficient evidence is collected, both the estimate and the learning rale are set to the queues' estimate, which is seen as a more abrupt or discontinuous change in the picture. The estimates then more quickly converge to the new true probability of 0.25. While we have not shown the actual negative and positive occurrences in the picture, it is easy to deduce where they are from the behavior of p(t) (specially after the switch, where the estimates are high): the positives occur when there is an increase in the estimate pi0) (from to t 1).
17

---

Published in Transactions on Machine Learning Research (03/2026)
DYAL, 0.001 true probabilty
dyal, 0.01
true probabiity
0.-
probabitty
0.4
0.3
0.1
0.0
2500
5000
time
37500
2500
5000
7500
10000
32500
15000
17500
20000
(a) Min-Rate of 0.001
(b) Min-Rate Rate of 0.01
0.7
DYAL 0:01
DVAL,
probabiltty
0.8
0.4
DVL EI
0.6
0.3
0.0
2500
7500
10000
12500
15000
17500
20000
2500
5000
7500
10000
12500
13000
17500
20000
(c) DYAL, Min-Rate of 0.001 vs 0.01
(d) Qs with capacity 5 and 10.
leaming actual probability
probabiiity & baaming (ati)
probaisiity
switch Queues time points
Janepers J1 meno
scout paduuityy
0.2
amall proc
0.1
change detected
2500
5000
7500
20000
12500
15000
17500 20005
time
(e) Changes in the learning rate / (DYAL).
(f) close of change in 8 (DYAL).
sequence. Parts (a) to (d) show the probability estimates of EMA, Qs, and DYAL, blue and orange colors, around the true/actual probability p" (green, up down steps). Note the convergences and variances, e.g. Qs with qeap is more variant than 10, in (d) and more variant than EMA or DYAL (note also the y-axes in different figures have different ranges). Parts (c) and (f) show evolution of the learning rate and the rate spikes of DYAL (for a single prediction larget).
Figure 10: Synthetic single-item oscillation experiments, 0.025 = 0.25,
Fig. 15 and Fig. 21 show additional patterns of change the learning rate multi-item real sequences.
Appendix D contains few additional variations.
5.2 Synthetic, Non-Stationary, Multi-ltem
Here, a non-stationary sequence is a concatenation of "stable" (stationary) subsoquences (similar above). each corresponding to true underlying SD p6) and long enough that all its items are observed sufficiently often before changing. We may have minimum (overall) length requirement well. Appendix D.2 describes
18

---

Published in Transactions on Machine Learning Research (03/2026)
the generation process GenSD() which also includes generating noise item. Whenever the underlying SD p(() is changed, we can either use new items ("new items' setting), or reuse but assign new probabilities ("recycle items' in Table 3h.
1.5any
1.5obs
logloss
1.5any
1.5obs
logloss
optimal loss
new items 4
Qs, 5
Qs,
Omin 10
0.94
0.24
1.17
0.88
0.17
1.19
1.040
#0.11
Omin 50
0.92
0.22
1.13
0.78
0.10
1.09
1.028 +0.22
static EMA, 0.01
static
EMA,
0.001
10
0.86
0.20
1.26
1.00
0.95
2.33
1.040
50
0.80
0.06
1.07
0.70
0.32
1.44
1.028
harmonic EMA,
0.01
harmonic EMA,
0.001
10
0.86
0.19
1.25
0.98
0.88
2.25
1.040
50
0.80
0.06
1.07
0.62
0.23
1.34
1.028
DYAL, 0.01
DYAL, 0.001
10
0.94
0.09
1.11
0.89
0.09
1.14
1.040
50
0.89
0.05
1.05
0.59
0.03
1.06
1.028
recycle items J
Qs, 5
Qs, 10
Omin 10
0.91
0.23
1.16
0.83
0.13
1.14
1.040
=0.11
Omin 50
0.91
0.22
1.12
0.76
0.10
1.08
1.028
+.022
static EMA,
0.01
static EMA,
0.001
10
0.87
0.16
1.16
1.00
0.73
1.65
1.040
50
0.80
0.06
1.06
0.77
0.26
1.28
1.028
harmonic EMA,
0.01
harmonic EMA,
0.001
10
0.87
0.15
1.15
0.98
0.66
1.54
1.040
50
0.80
0.05
1.05
0.71
0.18
1.18
1.028
DYAL, 0.01
DYAL, 0.001
10
0.91
0.10
1.11
0.83
0.16
1.18
1.040
50
0.88
0.05
1.04
0.56
0.04
1.06
1.028
Table 3: Synthetic multi-item experiments: performance averages over 50 sequences, ~00k length each, generated using GenSD: the underlying SD P is changed whenever all salient items in P are observed 2 Omin times (Omin E {10, 50)). In top half, items are new when underlying SD changes, and in the bottom half, items are 'recycled' (Sect. 5.2)
Table 3 presents deviation rates, generalized to multiple items (described next), as well as log-loss of various SMAs and the optimal (lowest achievable) log-loss: given item (f and the underlying SD p(t), if (!)) e p()) (o(t) is salient), the optimal loss at
(o(()))) and if (()) is noise (olt) & p(o), then the loss is - -In()(P())) The optimal loss is simply the average of this measure over the entire sequence this loss does not change whether
16 Note that new items recycle items (Table 3), as evaluate the same 50 sequences to evaluate the different methods/settings, so the corresponding optimal losses are identical.
The (multi-item) devialion-rate, given the sequence of underlying SDs [P],, is defined as:
1 N
deo( MWF...l...... I
multider(o W0O) pi0,d) (multi-item deviation).
where for the multidev() function, we explored two options: under the more lenient 'obs' setting, we score based on the observation at time t: multidevssso, W, P,d) d) == deviates(W(o) d))), Un- der the more demanding 'any' setting, we count as deviation if any estimate in W has high deviation:
1f single SD generated [o]N, what we described is an empirical estimate of the SD entropy, i.e. (an estimate of) u(P) In(u(P)) + Enep -P(() In(P(i)). When the underlying SD p changes from time to time, the computed optimal loss is the weighted average of the entropies, weighted by the length of the subsequence each SD was responsible for.
19

---

Published in Transactions on Machine Learning Research (03/2026)
121
1.-
66:31
127
1a 360
1.5
189
3%
6301
123
200
C30
805
8538
300
625
8388
1230
4.00
4100
420
TV (ilkmng:ppponde (54e0000
Tha paamert (9
(a.) now items, Omnn 50
(b) recycle items, Omin
(c) new items, Omin 10
Figure 11: log-loss performance, as the learning rate is changed, in synthetic multi-item experiments, Omin = 50. DYAL least sensitive 3min-
maltidevang(o (o. W, P.P.d. d) == m (eviates(W(i), P(t), d) (o is not used). multidevase is closer to log-loss, as it only considers the observation, and like log-loss, is therefore more sensitive to items with higher proba- bility in the underlying P. multidevany is more strict performance measure. Table shows both deviation types with d = 1.5.
3sows
As expected, when Omin, i.e. the length of the stability period, is increased from 10 to 50, all performances improve, even for Qs with qeap = The performance of Qs with qcap =10 is better than qcap =5, even for high non-stationarity 0min=10 as we change the underlying SD P only when all salient items of SD P pass the Omin threshold. Nevertheless, even with qcap of 10, Qs often trails the best of the sparse EMA variants significantly. Fig. 11 shows that. DYAL is not as sensitive to the setting of the (minimum) rate. We also note that while the log-loss values can seem close, the deviation rates can explain or reveal better why DYAL often does perform better overall. We can also pair two methods and count the number of wins (lower loss the same sequence), and DYAL gets substantially more wins in all pairings. Appendix D.3 contains these paired results and additional sensitivity plots.
6 Experiments On Real-World Data Sources
A variety of complex phenomena combine in the real world to generate the sequences of observations (not necessarily IID), and even if we assume stable distributions generate the data over some durations, in general we do not have access to the underlying probabilitics in order compare predictors. We use log-loss (AvgLogLossNS of Sect. [.2.2] and compare different SMAs on the same sequences. We begin with sequences that contain external non-stationarity (Unix commands), and conclude with a source reflecting internal system non-stationarity.
6.1 Unix Commands Data
The sequences in our Unix data exhibit significant external non-stationarity (Sect. 6.1 each person's pattern of command usage changes over days and months, as daily patterns activities change (project changes, vacation, ...). We look at two sequence datasets here: which we refer to as the 52-scientists data, subset of data collected by Greenberg (Greenberg 1988). and the Masquerade sequences (Schonlau al.
extreme case of non-stationarity, and the item probabilities may lose their meaning. On the other hand, another stationary regime may arise, for instance is picked uniformly from 1] each time, with short enough stable period, assumption, with appropriate learning (or otherwise harmonic EMA), may work best. (resembling the other extreme of high Omin and convergence full stationarity).
In setup, imagine Omin lowered say
if p* 200.5 the whole sequence. A learner making the stationary
20

---

Published in Transactions on Machine Learning Research (03/2026)
52-scientists (52 sequences)
Masquerade (50 sequences, 5k cach)
sequence length
# unique commands per user
# unique commands per user
median
min
max
median
min
max
median
min
max
1.8k
205
12k
106
22
359
101 5
138
Table 4: Statistics on Unix commands sequences, from the two sources.
2001 (Table 4j. Diverse contextual features, derived not just from recently typed commands but also temporal attributes such as time of day and day of week, can serve as predictors of the next command (Davison & Hirsh 1997 Korvemaker & Greiner 2000 Madani et al.) 2009a). As the sequences would be relatively short if we focused on individual commands predictors (at most 100s long in almost all cases); we look at the performance of the "always-active" predictor that trics to predict "moving prior" of which next command is lyped 18
30.
Famone DAA
Fasmalcc nan shardd
tamak
22
2.--
2.5
24-
barmik macos 8.1
20-
25-
200
1001
0800
6202
0100
E200
2013
1.50
1300
5.70
reaning
(a) log-loss rate, Unix 52-scientists soquences.
(b) log-loss rate, Masquerade sequences.
Figure 12: Losses (AvgLogLosNNS() us. on the Unix sequences.
shows the performance the (sparse) EMA variants as a function of the learning rate, and Table 5 reports on Qs performance with a few qeap values, and includes the best of the EMA variants. We observe a similar paltern (compare Fig. 11 a v-shaped performance, and for DYAL, as before, the plots show less of dependence on Amin compared other EMA variants. However, the degradation in performance lower 8min is more noticcable here. DYAL performs better 52-scientists compared to others, but is beaten by static EMA on Masquerade, unless we restrict the history kept by the referee, discussed next.
Fig. 12
6.1.1 Pairing and Sign-Teste Unix Sequences
We pair and compare DYAL to static EMA harmonic and static behave similarly. At 3 of 0.05, best for both DYAL and static EMA, on the 52-scientisl sequences, gel 46 wins for DYAL over static (lower log-loss for DYAL), and 6 wins for static. On average, 13% of sequence is marked noise by the referee, As we increase the referce threshold cNs from to 3 to 4, we get additional wins for DYAL and the log-loss performances improve for all methods, and the fraction sequence marked noise goes up, reaching 18% eNs 4. Conversely if we lower the referee threshold to 1, we get fewer 35 wins for DYAL over static EMA (and 11% marked noise). If impose window, of say 500, for the referee, the head-to-head wins of DYAL over other SMAs remains the same improves.
We also only predict the command stubs, éc. Unix commands such as "Is", "cat", "more", and on, without their arguments (filenmme, options, etc.). Command arguments not available the Masquarade data our experiments with full commands, stubs only, on 52-scientists yielded similar findings.
21

---

Published in Transactions on Machine Learning Research (03/2026)
52- SSieenntssss
Masquerade
Q
DYAL
Qs
static
static" 0.05
DYAL
2
3
10
0.05
2
5
10
0.05
0.05
1.228
2.586
2.629
2. 2.686
2.362
2.769.1..5
2.735
2.754
2.830
2.624
2.56
2.42
Table 5: On Unix sequences, performances of Qs and the best of EMA variants. Lower qeaps work better, suggesting significant (external) non-stationarity. The right-most two columns report log-loss when referee window size is 500 instead of the default unlimited size (Sect. 6.1.1
On Masquerade's 50 sequences, we get only 15 wins for DYAL us. 35 for static (EMA), again at Smin 0.05, where both do their best, which is statistically significant. With the default of cws = 2 only 5% of a sequence is marked noise on average. As in the case of 52-scientists, when we increase the referee threshold from 2 to 3 to 4, we get additional wins for DYAL, and at cNS = 3, DYAL has 32 wins over 18 wins for static (8% marked noise with eNS = 4). Similar to the above 52-scientists case, lowering the threshold to leads to more wins for static. Importantly, if we use a referee that basod on a finite window of the last say 500 (or 200, etc) (rather than our simple unlimited window size) again DYAL becomes significantly superior: 50 wins for DYAL, and 9% marked noise (compared to 5% with unlimited referee window).
Why a fixed and relatively high learning rate of 0.05 docs relatively well here compared to the more dynamic DYAL, on Masquerade sequences (under unlimited referee window)? Any assumption behind the design of DYAL, in particular the sufficient stability assumption, may be partially failing here. For many items, their probability, or appearance frequency, may be high once they appear, but their stability period may be too short, and simple high learning-rate of 0.05 may work just as well, compared to the slow two- tiered approach of first detection and estimation via the queues, and at some point, switching to the queues estimate. As we increase the referee threshold eNS, focus bias the evaluation further on the more stable items in the sequences, and we get better relative results for DYAL, giving more credence to the stability explanation. Limiting the window size (from unlimited to a few hundreds) also lowers the impact of noise (recently unseen) items.
Compared to the Expedition sequences (next, Sect. 6.2), the Unix sequences exhibit higher non-statlonnarity, when we consider a few indicators: the best performance occurs when Smin is relatively high at 0.05 (Fig. 12 Table 5 also shows that Qs does best here with smaller qeap values. Appendix E.1 presents additional visualizations (learning-rate spikes) for further evidence of external non-stationarity.
6.2 104 Expedition Sequences
Expedition is an exploratory system developed for investigating self-supervised cumulative learning of explic- itly represented structures, referred to as concepts (n-grams of characters) (Madani 2023). The sequences extracted from running Expedition exhibit internal non-stationarity as we explain The Expedition system repeatedly inputs a random line of text (averaging ~50 characters 20 and interprets it, involving scarching, predicting, and matching of its concepts (Fig, 13). The probabilistic predictions of the concepts are used to pick high scoring interpretation. The winning interpretation in turn is used to update prediction relations of those concepts participating in the interpretation. Over time, the system also composes new n1-grams out of existing ones, and uses them, which improves the subsequent predictions, and thereby the interpretations (e.g. 'new" is more predictive than 'n' or 'ne' in a corpus of English text). Generating and using n-grams is the source of (internal) non-stationarity: the system begins at the low level of characters, its starter-sct of concepts, and gradually grows its concept set (higher-level concepts). In every episode, once an inter- pretation, f.e. a final concept sequence, is selected, each concept in the sequence acts as a predictor and updates its weight for predicting what concept comes immediately after it to the right (in the final selected interpretation). For example, in the interpretation of Fig. 13 13b), the predictor (concept) 'r' observes the
19 Expedition soquenees are made available with the code, and Unix sequences are publicly available. 20 A text corpus of NSF abstracts was used in extracting the Expedition sequenecs (Dua & Graff 2017). The NSF datasct contains approximately 120k research paper abstracts, yielding 2.5 million English text lines, over 20 million term occurrences, and just under 100 unique characters.
22

---

Published in Transactions on Machine Learning Research (03/2026)
can and an
sand
m
g
nnd play ng ng...
invoke
en
nt
final interpretation
match
... ru.ningggg and playing...
...ing and d...
(a) Bottom to top interpretation (search) paths: invoking compositions (upward) and matching attempts (downward), finding an interpretation.
(b) An example final interpretation, given the imput fragment "rumning and playing'.
Figure 13: (a) The interpretation search process consists of invoking (up arrows) and matching attempts (down arrows), until no more compositions that match the input remain. (b) The final selected interpretation is a sequence of highest level available concepts, n-grams, that match the given input line well.
concept 'un' to its right (and not 'u') and updates accordingly, and (the predictor) 'un' observes 'n' to its right and updates accordingly, and so on.
The sequences were obtained from rumning the Expedition system for four trials, in each trial starting from scratch (character level) and with different random seed, i.e. different sequences of lines are input to the system. Each trial was stopped after 20k to 30k episodes, taking under hour each. In the course of a trial, thousands of new concepts (bigrams, trigrams, ..) are generated and used. Id generation for concepts is incremental via a simple counter, and as most characters are seen first, before any higher-level n-gram is generated, concepts with lower ids tend to correspond to characters (unigrams) and ids above 95 correspond to bigrams and trigrams. In cach trial, for each of a few arbitrary concopt (ids) being tracked"]] (a few below 20, a few around 100 and 500), we collected and created a sequence from what comes after it in the episodes is active in, i.e. it appears in one or more places in the final selected interpretation. For instance, in one trial, the concept (corrosponding to) 't' was tracked, and we obtained sequence of 25k items long for 't', each observation being concept id Initially, in the first few 100s of episodes say, 't' will only see single characters (unigrams) immediately next to its right (concept ids below 95). Later on, bigrams and higher n-grams are generated, a mix of unigrams along with higher level n-grams are observed (after tt). We tracked a few concepts in each run and collected 104 sequences over the different trials, with median sequence size of 1.2k observations, minimum size of 75, and maximum sequence size of 48k. For the less frequent and newer concepts (as predictors) we get shorter sequences. There are nearly 1000 unique concepts (concept ids) in the longest sequences, and 10s of unique concepts in the shorter ones.
22
Note that if Expedition did not generate new concepts, f.e. both the predictors and predictands remained characters (unigrams), this would be stationary task, as input lines are randomly sampled from finite corpus (e.g. the edge or conditional probability P('b'l'a') would not change) 23 The non-stationarity here is internal, due to then generation and use of new concepts.
6.2.1 Overall Performance on 104 Expedition Sequences
Table 6 shows the losses of our 4 predictors, averaged over the 104 Expedition sequences. As before, all the parameters are at their default when not specified, and log-loss refers to AvgLogLossNSO) (Sect. 4 We observe that DYAL does best on average, and as Table shows, pairing and performing sign tests indicates that log-loss of DYAL (with 8men = 0.001) outperforms others (yields the lowest loss) over the great majority of the sequences (high statistical confidence). We also used a referee window size of 500, and obtained similar results (the referee window size is unlimited by default).
The character, and in general concept, gets different id in different trial runs. 22ih implics 't' was active in up 25k episodes (in some episodes, 't' may appear more than once). We also experimented under this stationary multiclass setting and DYAL had faster (superior) convergence curve than the other SMA As and Qs (Madani 2024.
23

---

Published in Transactions on Machine Learning Research (03/2026)
Qs
static
harmonic
DYAL
2
3
10
0.001
0.01
0.01
0.001
logloss
2.65
2.60
2.61
2.70
2.80
2.56
2.71
2.42
Table 6: Loss on 104 Expedition sequences (1.2k median length).
DYAL, 0.001
Qs, 3
static, 0.01
static, 0.005
harmonic, 0.01
CNS = 3
1, 103
3, 101
24, 80
1, 103
CNS 2 (default)
1, 103
3, 101
25, 79
1, 103
CNS = 1
22, 82
35, 69
14, 90
17, 87
CNS = 0
21, 83
21, 83
5, 99
39, 65
Table 7: Number of losses and wins of DYAL, with Amin = 0.001, pairing it against few other techniques, on the 104 Expedition sequences, as we alter the CNS threshold. If observation count I cNs then it is marked noise (Sect. 2.2.2h. 2h Thus DYAL wins over Qs with (qcap 3) on 103 of 104 sequences (top left). The number of wins of DYAL is significant at over 99.9% confidence level in all cases. Also, DYAL wins over static with 3 E {0.01, 0.005), on all the 19 longest sequences, at default cNs = 2.
6.2.2 Sensitivity to Parameters
Table 7 also changes the CNS to assess sensitivity to what is considered noise. Lowering the cns makes the problem harder, and we have observed here and in other settings, that log-loss goes up. For instance, the log-loss performance of DYAL goes from 2.93 at cNs = down to 2.3 with cNs = 3. Of course, with cNs = 1, we are expecting lechnique to provide a good probability estimate even though the ilem has occurred only twice before
Fig. 14 aa) shows the sensitivity to Amin for DYAL and harmonic EMA, and 3 for static EMA. We observe, as in Sect. 5.2 for the case of multi-item synthetic experiments, that DYAL is less sensitive than both of harmonic and static EMA, while harmonic is less sensitive than static when / is set low (and otherwise, similar performance to static). In particular, for longer sequences, lower rates can be better (see Appendix L but for other EMA variants, low (fixed or minimum) rates remain an issue when faced with non-stationarity (i.e. new salient items).
Fig. 14)b shows the sensitivity of DYAL to the choice of the binomial threshold, which controls when DYAL resets to the queue estimates (Sect. 3.3.1). There is some sensitivity, but we posit that at 3 and 5, DYAL performs relatively well. In particular, we see in Appendix that when we look at performance on the longer sequences, the performance as function of binomial threshold becomes more stable, and thresholds 3 and above do best. We also ran DYAL using different queue capacities with Bmin = 0.001 (default is geap = 3), and obtained similar log-loss results (e.g. 2.38 at gcap = 2, and 2.4 for qcap = 5).
6.2.3 Evolution of the Learning Rates, Degrees, etc.
Fig. 15 shows plots of the evolution of maximum (max-rate) and median of the learning rates in the rate of DYAL for two predictors (two sequences). the concept "ten', with just over 200 episodes, and the concept "I" with over 12000 episodes. The number of entries (edges) in the rateMap (and the EmaMap)) or the out- degree, is also reported. We observe that the maximum over the learning rates contain bursts every so often indicating new concopts ncod to be learned (predicted), while the modian rate converges the minimum, indicating that most predictands any given time are in stable state, On the long 12k sequence, we also see the effect of pruning the map every so often (Sect. 3.2.2 and 3.3.3 the number of map entries remain below 100 as old and low probability items are pruned. Appendix E. also includes plots of the max-rate on few additional sequences coneaeoonaatettion)) in exploring evidence for non-stationarity.
24 Withoou extra information assumptions, such as making the stationarity assumption and assuming that there noise items, or using global statistics on similar situations (e.g. past items that were seen once), this appears impossible.
24

---

Published in Transactions on Machine Learning Research (03/2026)
240
of
DVL
30
2.42
JAELEVA EVA
Namonk
238
234
24
DOL vice CO2
02005
0800
0:2100
6,008
3.000
33200
1180
Herning sate paramaterns
(a) Logloss vs learning rate.
(b) Logloss vs binomial threshold, DYAL.
Figure 14: Expedition sequences: Changing the learning rate (left), and the binomial threshold (right) in DYAL, and plotting the AvgLogLossNS performance.
namber of predeturd
0:25
325-
pwidttads
0.20
nar pediie
015
mimbor
312
018
50
808
120
120
200
2000
1900
1000
updaio pirsi
(a) Learning rates, max and median, and out-degree, for concept 'ten' (via DYAL).
(b) Max rate and out-degree for T.
Figure 15: Examples of evolution of the learning rates of DYAL and the number of predictands, or out-degree ((EmaMap)), over time. (a) Maximum (max-rate) and median learning rates and out-degree for the concept 'ten'. (b) Maximum learning rates and number of predictands for the 13k-long sequence for the concept T..
7 Related Work
Our problem lies in the areas of non-stationarity and learning under concept drift (Ditzler et al. 2015
Gama et al. 2014 Hinder et al. 2023), online multiclass learning (Hoi et al. 2018 Madani et al.] 2009bb a
Wester et al. 2011), (non-parametric) density estimation and distribution learning DDvison & Hirsh 1997:
Korvemaker & Greiner 2000; Mazzetto
2024
Mazzetto & Upfal, 2023 Cohen et al.; 2020), probability
forecasting and assessing quality of output probabilities (propricty, calibration) Filloo et al. 2021 Tyralis
& Papacharalampous 2022; Brier 1950; Good 1952 Dawid 1984). , change detection (Aminikaaghaahi &
Cook 2016 van den Burg & Williams| 2022 Ditzler et al.] 2015). strcaming data structures and algorithms
Gaber el al. 2005 Gama el al.) 2009) and lime-series analysis (Weiß 2018 Maiti & Biswas 2015). To the
best of our knowledge, this combination of efficient open-ended probability prediction under statinaaion
has not been studiod before. We provided pointers on relevant work for the tools and techniques we usod
throughout the paper. We further situate our work within the broader context of similar tasks and problem domains, and provide a short history and discussion here.
25

---

Published in Transactions on Machine Learning Research (03/2026)
Non- con-parametric density estimation techniques are often based on kernels or keeping track of specific episodes and assume stationarity (e.g. Cohen et al. (2020): Devroye & Györfi (1987)) Recent work of Mazzetto (2024), in computational-eerriin an open-ended, in particular with infinite-support, non-stationary setting, but space efficiency and empirical performance of the predictor is not locus, and the kept history grows logarithmically in stream length.
theory, comes closest to ours and extends the density estimation task to
Furthermore, absolute loss (total-variational distance) is used to assess the theoretical quality of algorithms (limiting applicability to estimating only high probabilities well, above say 0.1 (see also Sect. B.2 on the Box SMA).
Our task and the DYAL solution involves kind of implicit change detection (CD) (Aminikhaghaah & Cook,
2016 Gama et al. 2014 Ditzler et al.) 2015 Ziffer et al.) 2020): Two main categories of adaptation strategy to change are blind (or implicit) and informed (or explicit) (Gama et al., 2014 Ditzler et al., 2015). In all of Qs, sparse EMA (static or harmonic-decay), and Box, adaptation to non-stationarity is done automatically or without explicit detection. DYAL comes closest to explicit detection, and uses the implicit Qs do so. In our lask, the system need only adjust its output probabilities in timely manner. In some lasks, such monitoring for safety and potential attacks, explicitly pinpointing the (approximate) time of change can be important too (Ditzler et al. 2015 Plasse & Adams. 2019 Change detection is a diverse subject studied in several fields such as system monitoring, psychology (e.g. within human vision) and image processing and time series analysis (e.g. Plasse & Adams (2019): ; Tartakovsky et al.] (2014); van den Burg & Williams
(2022; ; Atto et al. (2021)). Here, we seek a (timely) response and adaptation to a change. Many variants of moving averages are also used in time-series analysis (e.g. the ARMA model). There, the observations are ordinal (such as counts) even if discrete (e.g. /Weiß (2018); Maiti & Biswas (2015)), and stationarity or limited non-stationarity is typically assumed. Similarly, much past work, e.g. on variable window sizes, has addressed numeric data, for instance tracking means and variances (Bilet & Gavaldå 2007] Sebastião et al. 2008). Plasse & Adams (2019) observe that CD is understudied for categorical data, and the authors develop efficient model-based explicit CD techniques for streaming data: the set of items is known and fixed in that work (a multinomial distribution). Their evaluation is based on optimizing change-detection rates (e.g. maximize true-positive rales for an acceptable fixed false-positive rate).
The online observe-update cycle, updating a semi-distribution, has resemblance to online (belicf) state estimation, e.g. in Kalman filters and partially observed Markov models for control and decision making, in particular with a discrete state space (Dean & Wellman 1991]. Here, the goal is pure prediction (1s. action selection or control), though some of the techniques developed here may be useful to that setting, in a changing world. Extensions of the filter, via dynamic window sizes (for numeric sequences), has proved useful for adapting supervised and unsupervised learning techniques, such as Naive Bayes, to non-stationarity [Bifet & Gavaldà, 2007 Ziffer et al., 2021]. More broadly, streaming algorithms aim to compute useful summary statisties, such as unique counts and averages, while being space and time efficient, in particular often requiring single pass over large data set sequence, such the count-min sketch algorithm (Gaber et al. 2005 Gama et al. 2009). Here, we have been interested in computing recent proportions in non-stationary setting, for continual prediction, carried out by cach of many (severoly) resource-bounded predictors. Learning finite state machines in a streaming manner shares similar philosophy and a similar subproblem of change detection (in particular, Fig. 1, "system for continuous learning". " Balle et al. (014))
In typical online (supervised) learning, e.g. Hoi et al.] (0018); Rosenblatt 19558) 0: Littlestone (1988). the
focus is on how best to aggregate the predictors (leatures) for prediction and, for example, on learning a good weighting for linear model. Prior work of Madani et al. (2009b) on large-scale multiclass learning is also open-ended, including handling when parameters are allocated and dcallocated. Subscquent work tackles non-stationarity (Madani et al. 2009a). However, the focus had been on aggregating and ranking classes, while we have focused on learning good independent probability predictors, with improved justification for when allocation and deallocation decisions are made (Sect. 3.2.2 The independence is akin to the (multiclass) Naive Bayes model (Lewis) 1998) and the counting techniques for n-gram language models Rosenfeld 2000), but in a non-stationary setting. Another avenue of work is calibrating classifiers after training (trained a ranking accuracy-related objective) (Filho et al. 2021 : and there may be extensions applicable to our non-stationary open-ended online setting. Similarly, investigating online techniques for
26

---

Published in Transactions on Machine Learning Research (03/2026)
learning good mixing weights, but also handling non-stationarities may prove fruitful future direction, for instance in the mold of (sleeping) experts algorithms (Freund et al. 1997 Hoi et al. 2018 Dani et al. 2006).
Learning-rate decay has been beneficial, e.g. for backpropagation, and there is research work at explaining the reasons (You et al,, 2019 Smith 2018). Here, we motivated decay variants in the context of sparse EMA updates and learning good probabilities fast, and motivated predictand-specifcc (per connection) rates.
8 Summary and Future Directions
We formulated the problem of online open-ended probability prediction, where underlying assumption is that for adequalely long spons of lime, there erist sufficienlly many slable (prediclable) ilems for this task of learning and tracking changing probabilities to be useful. We developed number of sparse moving average (SMA) techniques for finite-ppace predictors. We described the challenges of assessing probability outputs under noise and non-stationarity, and developed a method for evaluating the probabilities, based on bounding log-loss. We showed that different predictors work best for different regimes of non-stationarity, but provided evidence that in the regime where the probabilities can change substantially but only after intermilient, periods of stability, the DYAL SMA which is combination of the sparse EMA and the Qs technique, is more flexible and has advantages over either of the simpler methods: the Qs predictor has good sensitivity to abrupt changes (can adapt fast), but also has higher variance, while plain (static) EMA is slower but is more stable. Combining the two, with predictand-specifie (per connection) parameters, yielding dynamic learning rates, often leads to faster more robust convergence in the face of non-stationarities.
We touched on number of open problems in the course of the paper. We hope to further develop and compare various SMAs in different tasks, e.g. in the unsupervised concept learning of the Expedition system, and for alternative problem formulations, such as when items have different rewards.
Acknowledgments
Thanks to Jana Radhkrishinan for granting the freedom conducive to this work while at Cisco, and to Tom Dean's SATRG reading group, including Brian Burns, Reza Eghbali, Sean Kugele, Georgi Georgiev, Gene Lewis, and Justin Wang, for continued discussions and feedback. Thanks also to James Tee and Aaditya Randas for discussions and pointers on ouline state estimation and change detection. am grateful to John Bowman for a proof sketch of unbiased estimation based on Rao- Blackwelllzaaionn, and related pointers. Many thanks to the anonymous reviewers whose many comments and suggestions led to a better paper.
References
W. C. Abraham and A. V. Robins. Memory retention the synaptic stability versus plasticity dilemma. Trends in Neurosciences, 2005.
S. Aminikhanghahi and D. J. Cook. A survey of methods for time scries change point detection. Knowledge and Information Systems, 2016.
R. Arratia and L. Gordon. Tutorial on large deviations for the binomial distribution. Bulletin of Mathemalical Biology, 51:125 131, 1989.
R. B. Ash. Information Theory. Dover Publications, 1990.
A. M. Atto, F. Bovolo, and L. Bruzzone. Change Detection and Image Time Series Analysis 1: Unsupercised Methods. Wiley-ISTE, 2021.
B. Balle, J. Castro, and R. Gavaldà. Adaptively learning probabilistic deterministic automata from data streams. Machine Learning, 2014.
A. Bifet and R. Gavaldà. Learning from time-changing data with adaptive windowing. In SDM, 2007.
27

---

Published in Transactions on Machine Learning Research (03/2026)
D. Blackwell. Conditional expectation and unbiased sequential estimation. The Annals of Mathematical Stalistics, 1947.
G. W. Brier. Verification of lorecasts expressed in terms of probability. Monthly Weather Review, 1950.
Cohen, Kontorovich, and G. Wolfer. Learning discrete distributions with infinite support. arXiv: Statisties Theory, 2020.
Wikipedia Contributors.
The Tao-BBackwel--ooooogro Theorem, Jensen's Inequality, Negative Binomial Distribution, Moving average, Efficiency (statistics), Kalman Filter, and Change Detection. In Wikipedia, The Free Encyclopedia. 2024.
T. M. Cover and J. A. Thomas. Elements of Information Theory, John Wiley & Sons, 1991.
V. Dani, O. Madani, D. Pennock, S. Sanghai, and B. Galebach. An empirical comparison of expert aggre- gation techniques. In UAI, 2006.
Brian D. Davison and Haym Hirsh. Toward an adaptive command line interface. In Interacción, 1997.
A. P. Dawid. Present position and potential developments: Some personal views: statistical theory: the prequential approach. Journal of the Royal Statistionl Society: Series A (General), 147(2), 1984.
T. L. Dean and M. P. Wellman. Planning and Control. Morgan Kaufmann, 1991.
J. L. Devore. Probobility and Statistics for Engincering and the Sciences. 9th Edition, Cengage, Boston, 2016.
L. Devroye and L. Györfi. Nonparametric density estimation : the L1 view. J. of the American Statistical Association, 82:344, 1987.
G. Ditzler, M. Roveri, C. Alippi, and R. Polikar. Learning in nonstationary environments: survey. IEEE Computational Intelligence Magazine, 10, 2015.
G. L. Drescher. Mode-up Minds construclivisl approach to artificial intelligence. MIT Press, 1991.
D. Dua and C. Graff. UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences, 2017. URL http://archive ics uci adui/du/mm https:/ http://rrchveeeeeeeee setvve.iisuuoi ies.uui..e..... httl//datstt//3/// researh
1bstractt++1990
P. Dupont, F. Denis, and Y. Esposito. Links between probabilistie automata and hidden markov models; probability distributions, learning models and induction algorithms. Pattern Recognition, 2005.
T. Silva. Filho, H. Song, M. Perello-Nieto, R. Caantos-Rddriueee M. Kull, and P. Flach. Classifier calibration: survey how to assess and improve predicted class probabilities. Machine Learning, 112, 2021.
C. Fosnot (ed.). Constructivism: Theory, Perspectives, and Practice. Teachers College Press; 2nd edition, 2005.
Y. Freund, R. Schapire, Y. Singer, and M. Warmuth. Using and combining predictors that specialize. STOC, 1997.
M. M. Gaber, A. Zaslavsky, and S. Krishnaswamy. Mining data streams: A review. SIMOD Record, 34(2), June 2005.
J. Gama, J. S. Aguilar-Ruiz, and R. Klinkenberg. Knowledge discovery from data streams. In Intelligent Data Analysis, 2009.
J. Gama, I. Zliobaite, A. Bifet, M. Pechenizkiy, and A. Bouchachia. A survey concept drift adaptation. ACM Compuling Surveys (CSUR), 2014.
T. Gneiting, F. Balabdaoui, and A. E. Raftery. Probabilistic forecasts, calibration and sharpness. Journal of the Royal Statistical Society Series B: Statistical Methodology, 69(2):243 268, 2007.
28

---

Published in Transactions on Machine Learning Research (03/2026)
I. J. Good. Rational decisions. J. of the Royal Statistical Society: Series B (Methodological), 1952.
S. Greenberg. Using unix: collected traces of 168 users. Technical report, University of Calgary, Alberta, 1988.
P. Helmbold and P. M. Long. Tracking drifting concepts using random examples. In Annual Conference Compulational Learning Theory, 1991.
F. Hinder, V. Vaquet, and B. Hammer. One or two things we know about concept drift a survey on monitoring evolving environments. ArXiv, abs/2310.15826, 2023.
R. V. Hogg, J. W. McKean, and A. T. Craig, Introduction to Malhematical Statistics. 8th Edition, Pearson, Boston, 2018.
S. C. H. Hoi, D. Sahoo, J. Lu, and P. Zhao. Online learning: A comprehensive survey, 2018.
Ingold. The Perception of the Encironment: Essays on Lävelihood, Dwelling, and Skill. Routledge, 2000. (See Chapter 11 on the temporality of the landscape.).
B. Korvemaker and R. Greiner. Predicting UNIX command lines: Adjusting to user patterns. In AAAI/IAAI, 2000.
A. Kuh, T. Petsche, and R. L. Rivest. Learning time-varying concepts. In Neural Information Processing Systems, 1990.
S. Kullback and R. A. Leibler. On information and sufficiency. Ann. Malh. Stal., 1951.
E. L. Lehmann and H. Scheffé. Completeness, similar regions, and unbiased estimation:. The Indian Journal of Slatistics, 1950.
D. D. Lewis. Naive (Bayes) al forty: The independence assumption information retrieval. In Claire Nédellec and Céline Rouveirol (eds.), Proceedings of ECML-98, 10th European Conference on Machine Learning, Chemnitz, DE, 1998. Springer Verlag, Heidelberg, DE.
N. Littlestone. Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Machine Learning, 2(4):285-314 -188 1988.
O. Madani.
Prediction games in infinitely rich worlds. AAAT Fall Symposium, 2007. Yahoo! Research o Tochnical Report, available at: www.omadani.ncc hat.aainntt/palliatiioo.oo ppbliccations.ttmll
O. Madani. Text analysis via binomial tails. In Document Intelligence Workshop at KDD, 2021.
O Madani. An information theoretic score for learning hierarchical concepts. Frontiers in Computational Newroscience, 17, 2023.
Madani. Tracking changing probabilities via dynamic learners. arXin, 2024. github.com/ TomadaniTet///////oooo/ooooaao waaaaaaiieet/spss----eoooo martoee-ovoninigeeeeeee
Code
Madani and J. Huang. On updates that constrain the number connections features during learning. In ACM KDD, 2008
O. Madani, H. Bui, and E. Yeh. Efficient online learning and prediction of users' desktop behavior. In IJCAI, 2009a.
O. Madani, M. Connor, and W. Greiner. Learning when concepts abound. J. of Machine Learning Research (JMLR), 10, 2009b.
O. Madani, B. Burns, R. Eghbali, and T. Dean. When remembering and planning are worth it: Navigating under change. ArXie, 2026. (An early version appears in BICA 2025).
R. Maiti and A. Biswas. Coherent forecasting for stationary time series of discrete data. AStA Advonces in Statistical Analysis, 2015.
29

---

Published in Transactions on Machine Learning Research (03/2026)
J. Marengo and D. L. Farnsworth. A geometric approach to conditioning and the search for minimum variance unbiased estimators. Open Journal of Statistics, 2021.
A. Mazzetto. An improved algorithm for learning drifting discrete distributions. In International Conference on Artificial Intelligence and Statistics, 2024.
A. Mazzetto and E. Upfal. Nonparametric density estimation under distribution drift. In International Conference on Machine Learning, 2023.
M. Merlean-Ponty. The Phenomenology of Perception. Gallimard, 1945.
M. Mermillod, A. Bugaiska, and P. Bonin. The stability-llastiiit dilemma: investigating the continuum from catastrophic forgetting to age-limited learning effects. Frontiers in Psychology, 4, 2013.
M. Mohri and A. M. Medina. New analysis and algorithm for learning with drifting distributions. Tn International Conference on Algorithmic Learning Theory, 2012.
J. Piaget. Psychology and Epistemology: Towards a Theory of Knowledge. Northwestern University, 1970.
J. Plasse and N. M. Adams. Multiple changepoint detection in categorical data streams. Statistics and Computing, 2019.
C. R. Rao. Information and accuracy attainable in the estimation of statistical parameters. Bulletin of the Calcutta Mathematical Society, 1945.
F. Rosenblatt. The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 386)3388,4 1958.
R. Rosenfeld. Two decades of statistical language modeling: Where do we go from here? IEEE, 88(8), 2000.
J. C. Schlimmer and R. Granger. Incremental learning from noisy data. Machine Learning, 1986.
M. Schonlau, W. DuMouchel, W.-H. Ju, A. F. Karr, M. Theusan, and Y. Vardi. Computer intrusion: Detecting masquerades. Statistical Science, 2001.
R. Sebastião, J. Gama, P. P. Rodrigues, and J. Bernardes. Monitoring incremental histogram distribution for change dotection in data streams. In KDD Workshop on Knowledge Discovery from Sensor Data, 2008.
URL https://api.
semanticscholar httpp::/ppp..emaatiicchh.... wtt.:/arr..rg//oouuuDDD..S 29631004
R. Sclten. Axiomatic characterization of the quadratic scoring rulc. Experimental Economics, 1998.
G. Shafer and V. Vovk. Probability and Finance: It's Only a Gome! John Wiley & Sons, 2001.
L. N. Smith. A disciplined approach to neural network typer-parameters. Part learning rate, batch size, momentum, and weight decay. ArXiv, 2018.
C. Strannegård, N. Svangård, D. Lindström, J. Bach, and B. R. Steunebrink. Learning and decision-making in artificial animals. Journal of Artificial General Intelligence, 2018.
R. Sutton and A. Barto. Reinforcement Learning: An Introduction. MIT Press, 1998.
A. Tartakovsky, Nikiforov, and M. Basseville. Sequential Analysis: Hypothesis Testing and Changepoint Delection. Chapman and Hall/CRC, Boca Raton, 2014.
K. R. Thorisson. A new constructivist Al: from manual methods to self-constructive systems. In Theoretical Foundations of Artificial General Intelligence. Springer, 2012.
M. Toda. Measurement of subjective probability distribution. Technical report, State College, Pennsylvania, Institute for Research, Division of Mathematical Psychology, 1963.
H. Tyralis and G. Papacharalampous. A review of probabilistic forecasting and prediction with machine learning. Arxi abs/ abs/2209.08807, 2022. URL /tt.ltttpp///pp..eeaaaiicccooaao http:/ccco......oo.ooopp/a//aaaa
30

---

Published in Transactions on Machine Learning Research (03/2026)
G. J. J. van den Burg and C. K. I. Williams. An evaluation of change point detection algorithms. ArXiv, 2022.
E. von Glasersfeld. Radical Constructivism: A Way of Knowing and Learning. Studies in Mathemalics Education Series: 6. The Falmer Press, 1995.
C. H. Wei. Continuously monitoring categorical processes. Qual. Technology. Quant. Manag., 2012.
C. H. WeiB. An Introduction to Discrete- Valued Time Series. Wiley, 2018.
B. Wester, P. M. Chen, and J. Flinn. Operating system support for application-specffe speculation. In Proceedings of the sirth conference on Computer systems, pp. 249-242, 2011.
G. Widmer and M. Kubát. Learning in the presence of concept drift and hidden contexts. Machine Learning, 1996.
D. Williams. Probability with Martingales. Cambridge University Press, 1991.
R. L. Winkler. Scoring rules and the evaluation of probability assessors. J. of the American Statistical Association, 1969.
K. You, M. Long, and M. I. Jordan. How doos learning rate decay help modern neural networks. ar 2019.
G. Ziffer, A. Bernardo, E. D. Valle, and A. Bifet. Fast incremental Naive Bayes with Kalman filtering, 2020 International Conference on Data Mining Workshops (ICDMW), 2020.
G. Ziffer, A. Bernardo, F. Della Valle, and A. Bifet. Kalman filtering for learning with evolving data streams. 2021 IEEE International Conference on Big Data (Big Data), 2021.
A Convergence Properties of Sparse EMA
The proof of Lemma [ follows. Here, we are the stationary binary setting (Sect. 2.1.1 where the target probability learn, P(1), denoted p", and the estimates of EMA for item 1, denoted ((), form random walk (p is the estimate immediately after the update at time t). Fig. 16 shows the main ideas of the proofs, e.g. the expected step size is 3 towards p' when is not too close to p".
Lemma. EMA 's movements, i.e. changes in the estimate gt0, enjoy the following properties, where B e [0, 1]:
1. Marimum movement, or step size, no more than B: Vt, (p(+1) - 1 SB.
2. Expected movement is loward p²: Let (1) = p² - (()). Then, B(A ((+(++)) = p) = -1133)(p p)
3. Minimum expected progress size: With 8(t)
= |A |((+1)), B(8(0) 2 B² whenever
| (*) N B
fi.e. whenever è is sufficiently far from p²
Proof. (proof of part 1) On negative update, f(0) ((0+1) = -p(() 1000-11 -1 3)p B 8, and on positive update, p(++1)- ((t) (1 () + 3 ((t) 3 ()) B I 3 (as g(t) E [0. 1]).
(part 2) We write the expression for the expectation and simplify: p" of the time, we have a positive update, iee. both weaken and boost ((1-31 - B)p + B), and the rest, 1-p" of the time, we have weaken only ((1- - B)p). In both cases, the term (1 3)p, is common and is factored:
E(A (++1) (1(1 = p) p" (p- ((1 3)p + B)) + (1 p²) (v- - - -1p))
(next, the term, p² - (1 - 3)p, is common and is factored )
(p" + (1 p" (pp (1 8)p) p²
p² (1 B8)p p" p" (1 33 -Bp
(1 =3' p)
31

---

Published in Transactions on Machine Learning Research (03/2026)
1 B
target P
1
target p
8
1
40
2
((0)
|8] s
Elp (3)
4,
p3)
H(0)
|0| SB, B(0) 2
time
(a) Convergence to within band (here, from below p²)
(b) Expected steps and expected positions
Figure 16: A picture of the properties in Lemma upper bounding the expected number of time steps to enter the band p" + 3 by g-2, whether from below or above the band (Theorem where A (t) :=p" p(t) and 8(t) : | - I. (a) At any point, there are two possible outcomes after an update (weaken or boost), and the movement, 8, could be toward or away from p¹ (e.g. 6 < 0), but always |6] S 33. (b) As long
as á is not in the band, there is an expected movement, E(d), of at least B², toward p". (8()) is smallest when P is near p* and largest, up to 8, when ş is farthest (p 0 or 1).
(part 3) Note from our definition of 810, 5(0) > 0 when distance to p is reduced (when JA (1+11) < I))
When p is close to p", e.g. p=p', the expected distance may not shrink, but when outside the band, we can show minimum positive progress: Assume p(() Sp' - 3, then 8(9) below p²) or g(t) = f(1+1) p and:
= pp p(t) (p" - 1p(++) (g(r+1) also
[[p - )(()) = =(((t))
E(p" p((t) - (p" ((++1))) = E6p' ())) E(p" - -b(et1)
(the above rewrite used the linearity of expectation)
(via part 2)
E(p" p()) (1 - B)E((" - -(())= - 338pppp - -(())
2 g2 (via the assumption 10- - 1 2 B)
And similarly for when 8(9) 2 p- + 3, then 50()_ = =p(t) - p((+1).
Note that from property 2 above, there always progress towards p" in expectation, meaning that if (()) <p"; then E(b(+1)) > p(0, and if g(0) p", then E(ß(+1)) < gi0), This is the case even if the probability of moving away is higher than 0.5 (6 $ p" < 0.5). We note however, that the result being in expectation, both the actual outcomes for p(t+1) can be father from p' than pit) (consider when g( =p"). Property 3 puts floor (a minimum) on amount of the progress towards p' in expectation, when á & [p" -B.p" + 3), For instance, when g(0) pp" -3, it puts a minimum on the expected positions E(p(((+))) E((((+2)), E(f((+3)), until one such point crosses the band, Fig. b), and the Theorem follows.
Theorem. EMA, with a fized rate of 3 E (0, 1], has an expected first- time bounded by O(3 2) to within the band p² + B. The required number of updates, for firsl-visil time, is lower bounded below by ((3-1).
Proof. We are interested in the maximum of first-time & when the expected B(((k)) -p" + 3). Using the maximum movement constraint, as long as 1p" p > 8, an EMA update does not change the sign of ppt) (f does not switch sides wrt p", e.g. if greater than p", it remains greater after the update). Thus, before an estimate p(t) < p" changes sides, and exceed p", it has to be within or come within the band p" + 3. Therefore, start with an arbitrary location $(1) outside the band, say g(t) < xp -3 (similar arguments apply p(0)
when ((1) p" + B), and consider the sequence, p()), ,(,),,((), (.)),... p((), where Vt. 1 I I k. bi)))) < <p" B.
We can now lower bound the expected position of g(k). k x=1 2 wrt ((1), to be at least (k- 1).8² above p0u)
32

---

Published in Transactions on Machine Learning Research (03/2026)
B(p(k) - -000000) - pp =B(p(k) - pp
+ p(k-1)
110 p)
(insert all intermediate sequence members)
E
Σ p - 111-11) p)
2515k
= Σ
B(plo
= =p) 2 (k- 1 1) (p0) p(t-1))
25rER
where used the linearity of expeclation, and the 3 lower bound for the last line. With p S 1, an upper bound of 3 on maximum first-visit time follows: k cannot be larger than e if we want to satisfy LA S k B() < p" - B, or one of aft {l,,,, + 1] has to be within the band p" + B.
The lower bound ((8-)) on k follows from the upperbound of B on any advancement towards p'.
The dynamics of the estimates ((() can be likened, in some respects, to oscillatory physical motions such as the motion of a pendulum and a vertically hung spring: the expected movement is 0 at target p", corresponding to the resting length of the spring, or its equilibrium length, where spring acceleration is O, and the expected movement is highest at the extremes (farthest from p"), akin to the accelcration (voctor) of the spring being highest when it's s most stretched or compressed.
B Further Properties of Qs, and Several Qs Variants
We begin this section by reviewing a few properties of Qs, then present two variants, time stamping and Box, and conclude with a comparison to ADWIN (Bifet & Gavaldà 2007).
B.1 Statistical Properties of Qs
A single completed (frozen) cell of a queue in the Qs method, the ideal stationary setting (Sect. 2.1.1 corresponds to the experiment of tossing a biased two-sided coin, with unknown heads probability p" > (the target of estimation) until heads is observed (shift the cell counts by one). Let be the number of tosses until and including the first heads outcome. Then follows the geometric distribution, and is the MLE estimate for p" (Hogg et al. 2018 Devore 2016), but it over estimates and the bias gets worse, 1/c in the ratio sense; p² as p" + O. (Madani 2024). We could repeat. the tossing experiment until we get k > heads outcomes. This corresponds to multiple completed queue cells. The total number of trials, or the total count over all completed cells, minus the number of heads, follows the more general negative binomial distribution (with parameters k and p²), and Rao- Sho-Blackwellization can be used to derive unbiased (and minimum variance) estimates as well as simple upper and lower bounds on p" (Marengo & Farnsworth 2021 Rao 1945 Blackwell] 1947 Lehmann & Scheflé 1950). Further details and other properties Qs (such as the sum and the spread of the probabilities derived from the queues) appear in Madani (2024).
B.2 Variants of the Qs SMA: Time- Some--Stmppng and Box
A small variation, which we call the time-stamp Qs, allows for a more efficient 0(1) updating. Here, the predictor also keeps a single counter, or its own private clock reflecting (update) time, and a queue cell, upon allocation, simply records its creation time using the clock. The time gap (difference) between two consecutive queue cells reflects the count (gap), and can be used to derive probability (similar to Sect. 3.2
The plain Box predictor, also known as sliding window predictor, keeps a history window of fixed size K, of the last K observations for our multiclass setting. The sliding window idea (of fixed or variable size) is a common tool used in various non-stationary problems (Bifet & Gavaldå 2007 Gama et al.] 2014 Ditzler ct al. 2015). For our task, the Box SMA can be implemented relatively efficiently via single queue (plus a count hash map), and thus it has similarities to the Qs technique (one long queue us. several small queues). However, the space consumption is rigid (KK), and unlike the Qs predictor, K could be relatively large
33

---

Published in Transactions on Machine Learning Research (03/2026)
such as 100 1000, depending how small one wants to go in tracking probabilities. This predictor, similar to the static sparse EMA, has convergence us. stability problem 25 depending on K either response time for new items suffers or small probabilities cannot be modeled well (see Sect. [3..1] and and Table 10 The dynamic Qs is more suited to non-stationarity. Our GitHub code implements these variants.
B.3 Numeric vs. Boolean Streams: A Comparison to ADWIN
The ADWIN (Adaptive Windows) technique of Bifet & Gavaldà (007 is streaming method designed for keeping track of a possibly changing mean and variance on numeric sequences, while the input to SMAs are (many-dimensional) sparse Boolean vectors (Sect. 3 ADWIN has similarities to Qs: it uses adjustable window sizes, also making use of quenes, of windows, for detecting and responding to change. Instead of the simple count queues (Sect. 3.2.1), for each item, ADWIN could be used (Bifet & Gavaldå 2007). We compare ADWIN's accuracy in emitting probabilities on binary sequences: the estimated mean j of a recent subsequence is the probability of observing the item (outcome 1), and we use 1 fi for the probability of O. The true probability p* of outcome 1 is picked at random in [0, 1] for periods governed by Omin, in Table 2 We looked at oscillation settings (P) + P2. e.g. P1 = 0.5 and p2 0.05) as well, and obtained similar results. We briefly discuss ADWIN's computational complexity (space/ /time) below. We used the River online suite of algorithms! 20 We used a maximum loss of 5 for logarithmic loss (but 10 and a few higher values do not change results). Therefore, we did not use a referee (no noise ilems). We normalized outputs of Qs and DYAL make sure the two probabilities add 1 (no higher, no less, as for ADWIN). We also include comparisons based on Brier (quadratic) loss (Sect. C.2
We experimented with all the parameters of ADWIN (grace period, M, clock,..),, but the confidence pa- rameter 8 made the most difference (we also needed to use relatively low grace period and clock such as 5, etc). DYAL (8min = 0.001) and Qs (qeap of 5) have default parameters below (wins are head-to-head us. ADWIN, eg DYAL us. ADWIN).
8 = 0.01
8 = 0.1
8 = 0.5
Qs
DYAL
wins Us. ADWIN
Omin = 10, log
0.317
0.310
0.303
0.268
0.289
DYAL & Qs win 300
0min =50, log
0.368
0.360
0.354
0.340
0.345
DYAL & Qs win ~00
0min=500 log
0.388
0.387
0.386
0.402
0.388
mixed
mmin 5500, log
0.541
0.541
0.542
0.576
ADWIN wins ~30 0.545
O Omn 10:---000 10, Brier
0.200
0.194
0.189
0.166
0.183
DYAL & Qs win ~00
O 0mn==00 Brier
0.233
0.226
0.220
0.213
0.217
DYAL & Qs win ~00
Table 8: Comparisons with ADWIN, on 30 binary non-stationary sequences of length 10k cach (probability of outcome changes, as in Table 2): 'log' is log-loss capped at 5 and Brier is quadratic loss. With fast (0min 100 and moderate change (0mi==0)), DYAL and Qs incur lower losses compared to ADWIN.
Very high 8 (above 0.1, f.e. assuming change when confidence on change low) loses its mcaning for ADWIN (or any statistical-signniiiaaee technique): we need 8 to be below 0.1, e.g. 0.01 for reduced false positives (the default in River is 0.002). But with low 5 ADWIN is slower to emit good changed probabilities compared to Qs and DYAL, under fast or moderate non-stationarity. With large, e.g. ô 0.1, ADWIN basically reduces to a regiment akin to Qs (reverting to a small subwindow) and we note Qs does well in these binary experiments. In multi-item synthetic experiments (not 'closed-worldd) Qs begins to trail DYAL substantially. The main ADWIN window can grow unbounded (see below), which is probably the main reason ADWIN has an advantage as we transition to basically stationary settings (e.g. 0min=5000 above) 27
But EMA can be more space efficient, while Box has rigid O(K) space requirement. On the other hand, update time of Box is O(1) when implemented efficiently, while EMA's is [W] (the size of its map). www//atest/api/teee
h0ttps://
(where the better ADWIN2 from the paper is implemented).
increase Omin in these experiments (even though the problem becomes more stable); 27ehe losses in Table x increase corresponding lower probabilitics for outcome 1 longer, and their losses (lower duc to when Omin small, subsequences
lower entropy) tend dominate the overall Joss for each sequence (see also Sect.. 5.2 comments on log- and entropy, and Sect. 5.1 on varying length subsequences).
34

---

Published in Transactions on Machine Learning Research (03/2026)
B.3.1 Computational Complexity
The main window ADWIN, of size W, grows if there is no (detected) change in the stream, and the memory requirement is O(M in W), where M 2 2 is the maximum number of buckets (cells) kept with size each power of 2 up to W (if M
there are up 2 windows of size 2', for each i, until W) (note: each (sub)window is compact keeping summary stats). ADWIN needs to be modified (eg. largest main window cut at max size of a few 1000s if are interested in say Pmán 0.01) so that its memory (and process time) doesn't 't grow unbounded. We expect this could be done (River doesn't 't allow us to limit the total number of windows). We saw a noticeable slow down when testing on 100k sequences. However, even with such capping of main window to say the last 1000 elements observed, the memory requirement tends to be higher than our techniques (over 20 buckets or cells us. say < cells for DYAL or Qs), and the test performance (to dctect change and to pick a subwindow) are more complex ADWIN (designed for handling the more general non-binary case).
C The Near Propriety of Bounded Log Loss
Here we present our analysis that points to the approximate propriety of LogLossNS(). Sect. discusses a few alternatives that we considered for evaluating SMAs.
C.2
briefly
In the IDD generation setting based on drawing from a SD P (defined below), we establish how log-loss is affected under various transformations (e.g.. due to applying FC() via the connection of log-loss with the Kulbeck-Leibler (KL) divergence. Under few assumptions such as small pmin (pmin - Pmin used in FCO), the analysis sheds light when P remains optimal (i.e. leads to lowest loss) near optimal. Specifically, we look at how far minimizer W of the bounded-loss can be from P, i.e. the extent of (item) distortion (see Fig. 17). There are 4 causes or sources of distortion: filtering, capping, bounding the loss, and use of an imperfect noise-marker. We show that any minimizer W is very coistrained: only items in sup(P) with low probability, below or close to Pmán: may have their probability reduced (wrt to P), as LogLossNS has ceiling on the penalty: moving their probability of reduction, their probability is zero in W*, and their probability is proportionalely spread onto remaining (positive probability) items (due to the properties of KL). Lemma I (distortion extent) then follows.
other items results in not lowered loss. In the case
C.1 Summary of the Properties of Bounded log-loss
The proofs are moved to Sect. C.3 to see the flow of arguments and the results below more easily. We begin with the basic propertics surrounding the KL() function, extended to SDs. The correspondence of LogLoss to KL. divergence (Kullback Leibler 1951 Cover Thomas 1991] extends to SDs:
Definition 1. The entropy of non-emply SD P is defored as:
H(P) - -P(D((P)))
(5)
REI
The KL divergence of W from P (asyremetric), also known as the relative entropy, denoted KL(PIIW), is a functional, defined here for non-empty SD and SD W:
P(i) W(i)
KL(PIW) Σ P(O)) In
(6)
In both definitions, by convention (similar to the case for DIs in Cover & Thomas (991 ], when P(i) = 0. we take the product P(()) In(x) to be The divergence KT((P[[W) can also be infinite (denoted +00) when P(i) > O and W(i) = 0.
Lemma 4. Given DI P and SD W, defined over the same finâte set I,
LogLoss(WiP)) H(P) + KI(PIWW.
35

---

Published in Transactions on Machine Learning Research (03/2026)
distortion
Po
Pnz
Per
2
W
Figure 17: When using loglossRuleNS() distortion can oceur, i.e. SD W may obtain a lower loss than the true generating SD P (due to filtering, scaling, and bounding the loss). Section C.1 describes the extent of the distortion. In particular, Lemma characterizes the properties of minimizing SD W": smaller items in P (those with lower probability, such as A, B and C' above) may be zeroed, and have their probability mass transferred (spread, proportionately) onto other items in an optimal W"
This is established when both are DIs, for example by Selten 1998 and the derivation does not change when W is a SD: LogLoss(WW(P)= = Eissuppp) P(i) In((()))()) = Eiesuprr) P(i) In(((i)) - H(P) (i. add and subtract H(P)), and noting that the first two terms is a rewriting of the KL.), establishes the lemma. Note that the entropy term in log-loss is a fixed positive offset: only KL() changes when we try different candidate SDs with the same underlying P. From the propertics of KL (Cover & Thomas 1991]. it follows that LogLoss is strictly proper (and asymmetric). We next formalize drawing itemss using a SD P. which enables us to define taking expectations wrt a SD P.
Definition 2. Definitions of sealing, drawing from a SD, and a perfect marker:
(scaling (D) Let SD P be non-emply, i.e. a(P) E (0, Then, for 0, - a.p); means the SD where sup(P') == sup(P) and vi E sup(P), P'(1) = aP((i. When a a aP is DI and one can repeatedly draw IID from it.
P' oP
Drawiing item from non-emply SD P, i.e. a~ P, means a(P) of the time, drawing from oP, where a(Pj and 1 1-aaap a(P) of the time generating unique noise item. Repeatedly drawing items in this manner N times generates a sequence [o] using P ('IID drawing" from P), and is denoted [o]N ~ P.
(perfect marker) Given a non-empty SD P, perfect noise-marker urt to P, denoted isNSp(, marks an item as noise iff & sup(P).
Thus the perfect marker generates noise markings at about u(P) fraction of the time on the stream [o] P. Next we show that LogLossNS() is equivalent to bounded quasi-divergence denoted KLvs(), similar to plain log-loss reducing to plain KL divergence. First we define augmentation of an SD to corresponding DI that has one extra item with all remaining probability mass. This allows us to draw from P as if it were a DI, in particular for the purpose of evaluating LogLossNS(): We show computing Tog)ossNS(WWPP) is equivalent to computing a bounded KL() on the augmented version
28 of P and FC(W) (Lemma
Definition 3. Definitions of augmenting an SD to a DI, and bounded KL corresponding to LogLossNS():
Given a non-empty SD defined on I fl,...,kk,k 2 1, its augmentation (operation), denoted DI(P), is corresponding DI P', where P' is defined on I' I' = IU (0), P'(0) u(P), and P'()) = P(i), T.
(Bounded KL) For non-empty SD P and SD W(pmin 1]):
P(i)
KL(PIIW)
Σ P(1) In
(ax)))() pmin) (bounded when Pmin 0).
iESUP(P)
We are abusing notation using DI() also denote the augmentation operator applied SD make distribution, in addition to the related notion of referring to distribution.
36

---

Published in Transactions on Machine Learning Research (03/2026)
(KL() for LogLossNS) KLns(PIIW) == KL((DIP))DI(FCOO))) (where the pmin of FC() is used in (T)()).
Both (LI(P))) and (PI)() can be negative even when both P and W are DIs (the minimum is not any more), and they KLNs(PIIP) is not necessarily 0 (FC() is only applied to the 2nd P). As the lemma below shows, the addition of the positive entropy counteracis this growth and LogLossNS() is always in [0, - In(Pmin]]-
divergences, but still useful for scoring and comparisons. In particular,
Lemma 5. For any two SDs and W defined on [1,..,,)) where non-empty, and using perfect noise-marker is (), wrt P:
TogLossNS(WiP) = H(DI(P) + KlNs(P
The next property shows how plain KL() comparison: (defined for SDs in Defn. 6 are affected by scaling, useful in sceing how FC() can affect the minimizer KLNs(.
Lemma 6. (plain KL() scalings) For any non-empty SD P and SDs W, and any 0:
KL(PIaaW)
2. KL(aP|IW) =
8. KTaPPlaWW = (KI(P|))
As a consequence, if (L((I[WW) < KL(PI(W)) then KL(PI]aW,) < KL(PIlaWs). More generally, the following, 'proportionate' spreading properties, can be established.
Corollary 1. Scaling and spreading (adding or deducting mass) should be proportionate to SD P to minimize KL(P(I)):
(P is the unique minimizer over appropriate set) Given non-empty SD P, KL(PI(P) = 0, and for any W * with a(W) S a(P), KL(PIIW) > 0.
2. (same minimizer for P and its multiple) Let non-empty SDs Pı and P2 be such that Pi = aP2 for a scalar > 0, and consider any non-emply sel S of SDs For any two SDs Wi, W2 E S, KL(P)I[W,) < KL(P:|[W)) + KL(Pal(Wi) < KL(P2l(W)) (thus, a SD W E S is a minimizer for both or for neither).
3. Given non-empty SD P, among SD W such that a(W) = the one that minimizes KL(PIIW) is proportionate to (or a multiple of) P, iec Vi E I, W(1) ai P(i) (when = 0 this becomes vacuous).
We can now better describe what a minimizer W of (Pl.)) looks like in terms of P, where a(W") a(P) (and Pmiin > 0): we show that we have all-or-nothing deductions: an item's probability (in W if reduced (compared to P), it. is 0 (zeroed), herwiie stays the same (if no item zeroed) or increases. But, firstly, when a(P) S Pmin (degeneracy) many Ws are minimizers:
Lemma 7. (Minimizer in the degenerate case) Given threshold Pmains Pmin E (0, 1) and nonempty SD P such that a(P) S Pmiin (degenerate), then for any SD W with a(W) S o(P) (including P and the empty SD ), KTx (W) a(P) In(Pmin).
If P is don-degene,,,e, we show that minimizer must exist, and show some of the properties must have (see also Fig. 17
29 this lemma, can extend the definition of KT.) non-negative valued and (no constraint on the sum, unlike plain SDs), or assume such that aW and oP remain SD.
37

---

Published in Transactions on Machine Learning Research (03/2026)
Lemma 8. For threshold Pmin E (0,1) and SD P such that a(P) > Pminy where we seek minimizer of KT& (PIIW) over the set S of SDs W such that a(Wi) S a(P) (and wlog we need only consider sup(W) S
sup(P)):
(all positive probability items are above Pmin) For any Wi E S, if there is an item i where Wi(i) e
(0. Pminl: then there is SD W2 E S, such that Vi E sup(WW), W2(1) > Puing and KL& < (II(P))))). Therefore, in minimizing (TI(PP))), we need only consider the set S2 = (WIW e
S, a.a(W)= = aPP), and vi E sup(W), W(1) > Pminn (S2 is not empty as P is non non-degenerate).
2. (eristence and proportionate increase) The set S S S2 of minimizers of KLs(Pl1.) is not empty,
and for any W E ES", and for some fized maltiple r 2 1, vi E sup(W*), (i) = rP((i.
3. (order is respected) For any minimizer W and any two items i and j, when P(i) < P(i), if (i) > O, then (i) > 0 (and, from part 2, (i) > (i))..
Thus, items in (P) are zeroed in order of their probability magnitude and any non-zeroed item can only increase in probability, and proportionately (unique minimizer W if un-equal item probabilities), and the above suggest simple greedy methods for finding a Wt. The next properties (existence of threshold po) give us additional information about which items can be zeroed.
Lemma 9. With a DI P and Pmin E (0,1), if item i has probability P(i)= 2 Po- where Po such that Punin = =Po(1 - Po) I-Po pa then (1) 2 P(i) (item is not zeroed) in any minimizer W of KL,(P)). If E sup(P) has the smallest probability in sup(P) and P(i) < Pu, then is zeroed in some minimizer W (i) = =) and if it is the unique minimum, then W (i) 0 in any minimizer W
Pmnin = po - po) 1-Bu
70 implies that 2pmin S po S. epinn R 3pmin- Thus, as rule of thumb, as long as p 2 3pmine an item with p not zeroed. Note that the lemma is established for DIs only: for the analysis of extent of distortion below, we can assume P is DI (the first argument in KLNs() is augmented to a distribution). However, we expect the bound can be extended to strict SDs well (this is the case for SDs with only two items).
The following lemma specifies extent of increase in an item's probability in any minimizer of (L,(P)).),, and further characterizes the properties of a minimizer.
Given any DIP and Pmin E (0, 1), and any DIW with sup(W) C sup(P) and with proportionate spread according to P, let Z := sup(P)- sup(W") (the zeroed ilems or the difference of the two support sels), and Ps Pc(W)= ;= Eiez P(i), thus PW()) is the total mass of the zeroed items (the shifted mass), P=(W) 2 O. (part 1) We have for any i E sup(W), W(1) = (WP. (part 2) Purthermore, lel Sg be the set of all such proportionate W with mun P(i) 2 Po- Then any minimizer DIW of KLs(Pll.) is in Sg. and has the largest support and the smallest shifled mass among such (Lee. ps(W") S PxW) and [sup(W")] 2 |sup(W)] for any W E Sg).
Lemma 10.
P(i)
When the total probability shift ps is small, the increase T-ps in probability of any non-zeroed item is also small.
We have assumed a perfect marker (referce) in all the above. Assuming Pmiin = 0.01, a simple practical noise- marker, e.g. keeping a history of the last 100 time points (a box predictor), will have some probability of making false positive markings (a salient item marked noise) and false negative errors (items below Ponn). The error probability goes down an item becomes more salient or more noisy (its probability gets farther from the Pmin threshold), and for items with probability near the boundary, the loss would be similar whether or not In(pmin) is used. Given a SD P on I = (1,2....) },, the ideal- dieal-lireshold moise-marker, or the threshold marker for short, marks an item noise iff P(i) < Pma Considering how LogLossNS() (or loglossRuleNS() works when this threshold marker is used, the marker in effect converts SD P to a DI p' where for any item with P(i) S Pmin: its probability, together with u(P), is transferred to item 0 when computing KI (T'ssPP)l)) (recall that DI(P), in definition 3 only transferred u(P) to item 0). Thus P' is
38

---

Published in Transactions on Machine Learning Research (03/2026)
DI where Vi E sup(P") if i + O, then P'( > Pmin- It is possible that P'( < Pmin: but as use FCO: we could limit our analysis to DI "D' where Vi E sup(P"), (i()) 2 Pmin- The next lemma shows that the distortion ratio (" is upper bounded by about 3 in this sctting (where the probabilities are above Pman). P())
Lemma 11. Given any DI P with mippes P(1)= 2 Pmins for any minimizer W of KLo(PII.), the distortion ratio (
P()) S (1-Po)pman p1)
When Pmin = 0.01 the distortion ratio is bounded by 3.1. A practical marker can be viewed as noisy version of the (ideal) threshold marker. Given threshold Pmii > 0, if we find that the predictions W( have mostly unallocated mass or probabilitios close to Pmins then Pmin and Puain may need to be lowerod (e.g. from 0.01 to 0.001) for improved evaluation as well as better (finer) prediction 30
C.2 Alternatives to Bounded Logloss
Quadratic (Brier) loss is proper and enjoys attractive properties such as symmetry and not blowing up on zcro prodictions, but is not sufficiently sensitive towards low probabilities for supporting a diversity of probability ranges. Take the actual distribution P = {A:0.5, B:0.05, C:0.45), and two proposed ones W2 = [A:0.5, B:0.0, C:O.5), and W3- = {A:0.45, B:0.05, C:0.5)- W2 completely ignores event B (a one in 20 event), but W2 and W3 have the same Brier loss, of 0.55 (equivalently, equal Euclidean distance to P), and W/- = {A:0.46, B:0.0, C:0.54) has lower loss than W3 (for further development, see Appendix A.1 of Madani (024g) log-loss is also sensitive to higher probabilities, but does not suffer this extent of insensitivity. If all probabilities were in a narrower range, say 0.1 to 0.5, perhaps Brier would be adequate. Our GitHub code supports reporting Brier loss as well, and comparisons based on Brier (e.g. in synthetic experiments) has not changed our findings, See also Sect. B.3 which includes Brier scores. We also looked at other options, such as separating zero- and non-zero-probabiiie predictions (e.g. reporting two numbers), but such made comparisons among two or more SMAs difficult and could lead to impropriety.
C.3 Proofs for Section C
first explain the different parts of LogLossNS(W/P)) i.e. Bo-p( loglossRuleNS(,, W, (o)))) to show that it is equivalent to comparing two augmented DIs under KI().
Proof of Lemma b We
Let W = FC(W) and p' = DI(P). An item o ~ P (drawn from P), is salient with probability a(P) (o E sup(P)), and otherwise is marked noise by isNSp( (perfect noise- marker), and it is important to note that o does not oceur in I when marked noise (W" W(o) = P(o) our uniqueness assumption when generating noise items. When we use loglossRuleNS(,, the score in the noise case is - In(u(W")), This case occurs u(P) of the time, and salient item i, i E I, occurs P(i) of (i)) We have
when (pN is true), with
the time, thus LogLoss.NS(WPPP) = -u(P) In (n(max(u("")))n) - -ier that u(W")= 2 Pmii (from the scaling down in FC()), or max(u(W"),, Pmin) = u(W"). Adding and subtracting H(P') establishes the equivalence:
= ((()))(((W)))))
Σ P(i)
(1).(((i)))))) ((()) +HP')
TEl
= uP()
> P(I) In(m Boannax<WWWW (i),P)))) + Σ P'())nn(P(I)) + +H(')
NET
iET
ln In u(P)
P(i)
+ Σ P(i) In
+ +H(P')
u(W)) -KL("IIDI(W") + H(P') KLNs ([INs(P]))+H(DIII
(W"(
The 2nd to 3rd line follow from H(P') = Eier I'())(n((((()) (n((P())) = -u -u(( -(P)) P()
However, this lowering, incurring extra memory, does not guarantee that further salient items are discovered. For instance, all remaining items may be (pure or absolute) noise, appearing only once the stremm.
39

---

Published in Transactions on Machine Learning Research (03/2026)
Proof of Lemma for
6 The a
multiplier comes out, in all cases, multiplier for the
yielding
fixed off-
set
first 2 cases, and
positive
2nd and
3rd
cases:
P(i)
P )
P(i) W(i) + In(1/a))
KL(PIlaW)
Σ
(())
In(
P()
In W(i
Σ
P() (In(
aW(i)
iEI
iE7
iEI
P(i)
Σ iE1
P(i)
In
In(a
Σ
P(i) KL(PIIW) + In(a (n(aa1))(f ()(())
W(1)
iE2
KI(aPIW) Σ (()
= a aP(()) In( aP(i)
In(
ai(i)
W(i)
Σ
P(i)
In
W(i)
P(i)
+ In(aa) ΣP(O
iEI
VEZ
iEI
P(i) In( P(i)
KI((PllaWW) = Σ
iEI aW(i) = a Σ
) = (PI(P)
W(i)
IEI
Proof of Corollary 7 (part 1) When P a DI, among W # P that are DT too, the property (TI(P)))) > 0
holds (Cover & Thomas 1991). If W is a strict SD, on least one item i, W(i) < P(i), and we can repeat increasing all such W(i) in some order until W(1) P(t) or W becomes DI (finitely many such i), lowering
the distance (the log ratio P0)) We conclude for any W * P, KL(PI(W) > KL(PI|P) = 0. When P
is a nonempty strict SD: We have KL(PIIP) = P() In P() P() 0. We can scale P by = a(P) to get DI, and from the property of (T(()))) for DT P, and using Lemma G we conclude that for any other SD W * P, with a(W) I a(P) (and thus aW remains SD), must yield higher (positive) KL(PIIW): (P((I)W) = KI(aP]laW) > 0 (using Lemma and KT(aPllaW) > 0, from first part of this claim).
(part 2) Let A == KL(PL - KL(P: and A2 == KL (M) ) (MM) A = (L(aP2|
(TT(aP2 (22)) = (K2 M)) - (P2))))) (from part 2, Lemma 6 a (n(( (P()) canceling). There-
fore, A < 0 + A2 <0.
(part 3) This is a consequence of parts 1 and 2 where the set S includes Pl, where P1 is proportionate to P2, and P1 minimizes KT.() to itself among W E S (part 1).
Proof of Lemma thus (L,(PIIW)
7
For any such W, (Ls(PI[W) Eiesuper) -P(i) In(pmin) (as Vi, W(1) S a(W) S Pmmin):
= (P) In(pmin)
Proof of Lemma (part 1) Say W, has one or more items with low probability S Pminn call the set Iz. Ie : (i) S )m with total probability b (thus 6 := Eiez, W(i).. We can also assumee a(Wi) = a(P) (if less, we can also shift the difference onto receiving item j in this argument). Then if W1 also has an item j with probability above Pmnn;; shift all the mass to item j. Otherwise, shift all the mass b to a single item j in Ia (pick any item). In either case, we have (Ls(P)()) < (PI(()))) In the first case, the cost. (i.e.- -P(i) In( pmx) is not changed for items in I2 (-bln(pmin)), while for item j the cost is lowered. In the second case, must have b > Pmin (P is non-degencrate), and the cost for the receiving item i improves (as we must have (2 > Pmii), while for others in Iz it is not changed.
P())
The set S2 (from part 1), can be partitioned into finitely many subsets (ie disjoint sets whose union is 52), each partition member corresponding to a non-empty subset of sup(P). For instance all those that have iositive probability on item only (greater than Pmnn by definition of S2) (support of size 1) define one partition subset (we get [sup(P)] such subsets corresponding to singletons). The size of the support set of a SD in S2, k < (P)) can be large lo the exlent that a(P) > Pmin is satisfied (A: = works, but larger k may yield valid SDs in S2 as well. On each such partition set Sk), (LI(P)l)) becomes equivalent to KL(P!I)), in the following sense: If Sy is a partition, then YW, W2 E Sy, (I,((I(QI) (L,(()((22) = KL Q1) KL(P)IQ2). From Corollary [ the total mass from elements that are zeroed (if any), i.e. sup(P) - sup(Wii,, is spread proportionately on sup(Wi) to minimize KT.(PII-) over a partition subset. Since we have finitely
(part 2)
40

---

Published in Transactions on Machine Learning Research (03/2026)
many partitions (the size of powerset at most), and each yields well-defined minimizer of KLs(P)I.), we obtain one or more minimizers for the entire S2 and therefore S.
(part 3) Wlog consider items 1 and 2, P1 = P(1) and P2 = P(2), where P1 < P2, and assume in SD Wi E S2 (with support size [sup(W))] 2 1), item 1 has an allocation T > Pmin (as W1 E S2), while W1(2) < W1(1). We need only consider the case Wi (2) = if (2) > 0, then (2(2) > Pmin as Wi S2, and proportionate increase from part 2 establishes the result. Assuming (2) 0 we can 'swap' items 1 and 2, to get SD W2 S2, and swapping improves KLs(), ie. letting A = KL, (LI()))),)) (L,(P))W)), must have A > 0:
A (P2 in
P2
+ P1 In P1, (p1 In
P1
P2 + p2
(all other terms cancel)
Pmin P2 In(pmin) pi IT()) + pi In(pmin) + p2 In(T) (p2 (In) In(pmin)) >0
Pmin
The last step (conclusion) follows from our assumptiins that p2 > p1 and T > Pmin. Note that W2 can further be improved by proportionate spread (the allotment T to item increased).
For establishing Lemma 9 suppose the DI P has two items, with probabilities and 1 1-p, i.c. P = {1:p, 2:1- p), where p v - p: and > Pmin, and we want to see how high can be and yet distortion remains possible, i.e. item 1 is zeroed and its mass shifted to item 2, and = KLo(PIIP) KLs(PIIW) > how high p, p > Pmin, can be and yet we can get A > 0.
0 (pln pmin + (1 In thus A O,
when p pli(Pmmnn > pln(p) + (1 - p) In - p), or Pmin
> p(1 p) 1-2 > Thus the threshold Po is such that
Le Pmai = -Po(1 Po) P0 Now, with r := (1 (1-Po) Po
mg
1 - Po for Po > 0.5 (r + 1.0 as Po + 1), and = Po 0.5 when po = 0.5) (note that in our set up, Po S 0.5). Thus, indced Pmnin < Po- We can set Po and see how low Pmiin should be (or, otherwise, solve for po). Considering the ratio Pmin po(1 (t-)0/0 me)/r = r, where we have definod r = (1 Po)
< 1-po < when < 0.5 (and r = 1
1-00
P0 and as
Po 0, Pmin (e denotes the base of the natural logarithm), while po 0.5, Pmin po 2: or:
po
2pmin S po S CPmin 3pmin-
The case of more items is similar to two items, and yields the same distortion threshold po: given DI p assume all its items have probability above Pminn and A == KLs(P][P) - KLo(PIIW") = 0 - KLL(PI and we are wondering about the relation of say P1 = P(1) and pmin when A > 0.
(
A = ((p In(ppp/Pmmii + Σ Pa (n(::((p + 1 p1
pt))
12
(the above is the proportionate spread of pı onto others in W*)
Pi In(pmia) - P2 In(p)) - - ps In(p;) +
Σ
In(p: + 1- P1 P1)
122
222
Pi In(pmin)
Σ
Pi (p())
Σ
Pi In(
pi
1-
121
122
pi In(pmin)
-
Σ Pi In(p:)
+
Σ
Pi In(p;)
-In(1 - p1)
Σ Pi
121
122
i22
Pi In(pmin)
- In(1 -P1)
Σ
Ps
Pi
In(
P1
P1 P1 In(Pmin) - In(1 P1) -p In( 1 P1
1
P1
And > 0 implies Pi In(pmin)
> In(1 - pı) + + 1n( or when Pr low enough such that Pmin
(1 P1) 1nn This is the same bound or threshold the two-item case, and we summarize the properties in the following lemma.
41

---

Published in Transactions on Machine Learning Research (03/2026)
Proof of Lemma 9
The result follows from the above derivation and the ordering properties specified in
Lemma
Proof of Lemma 10 For a DI W, let P= denote Pa (WW,, and let 8 Eiesupow) P(1). From the definition
of proportionate spread, we add P(t) Ps to P(i). We have s = Ps, therefore, W(1) = P(i) + P( Ps =
P(a)_ (proof of part 2) From Lemma 8 P. is proportionately spread onto non-zcroed items in W as well. Consider Q1 and W", both in Sg, and assume sup(Wi)= > sup(W") (proof by contradiction). By shifting the lowest probability in S1 to remaining (higher probability) items in the support, and repeating, we should get to W (or an equivalent, in case of tics). But each shift results in an inferior W because of our assumption that all probabilities are 2 Po and the shifts only increase the probability on items that remain (non-zeroed items).
P() 1-p:"
Proof of Lemma 11 Let m be the mass of non-zcroed items in P (m 1 - Ps was also used Lemma 10 then need to bound the distortion ratio m which is the ratio by which each item in
sup(W") goes up by ( (PP())) ). Let j be any item that is zeroed (at least one such item exists, otherwise the distortion ratio 1). Let x := P(), and we have x 2 Pmin- Then spreading 1.0 over one additional item, sup(W") U {j}: proportionately, would fail to take the probability of j to po (by Lemma 10 W
would not be optimal). Proportional allocation over sup(W*) (i} yields xtm = for item j and we must have
po < pot pom > [l-pol2 < and x > pmiin we gel < (I-Po)a POJE (1-Po)pman
]
po
D Additional Synthetic Experiments
We first show comparisons in the stationary setting, then describe the muli-item generation process and present additional results, including pairing (for sign tests) in that setting.
D.1 Tracking a Single Item, Stationary
All the prediction techniques are based on estimating the probability for each item separately (treating all other observations as negative outcomes), so we begin with assessing the quality of the predictions for single item in the binary stationary setting of Sect. 2.1.1 Thus, when p' = 0.1, about 10% of the sequence is 1, the rest 0. Table g presents the deviation rates of Qs, EMA, and DYAL, under a few parameter variations, and for p' E (0.1, 0.05). Sequences are each 10k time points long, and deviation rates are averaged over 200 such sequences
Under this stationarity setting, higher qeap helps the Qs technique: Qs with qcap 10 does better than qeap of 5, but, specially for d = 1.5, both tend to substantially lag behind the best of the EMA variants. EMA with harmonic decay, with an appropriatcly low Biins does best across all p". If we anticipate that the useful items to predict will have probabilities the 0.01 to 1.0 range, in a stationary world, then setting Bmin for harmonic EMA to a low value, 0.01 where k 2 10, is adequate. In this stationary and binary setting, the complexity of DYAL is not needed, and harmonic EMA is sufficient. Still DYAL is the second best. Static EMA is not flexible enough, and one has to anticipate what p" is and set the (fixed) rate appropriately (for instance, when p" = 0.01, FMA with the same B = 0.01 is not appropriate, resulting in too much variance). Finally, we observe that the deviation rates, as well as the variances, for any method, degrade (increase) somowhat as p" is lowered from 0.1 to 0.01. In ten thousand draws (during sequence generation), there are lewer positive observations with lower p", and estimates will have higher variance.
k
D.2 Synthetic Non- non-Stationary Multi-Item Generation
Fig. 18 provides pseudocode of the main functions for generating item sequences under non-stationarity. As in the above, we think of a non-stationary sequence as concatenation of "stable" (stationary) subscquences,
31 The methods keep track of the probabilities of all the items they deem salient, in this case, both 0 and 1, but focus the probability estimates g(t)) for item
42

---

Published in Transactions on Machine Learning Research (03/2026)
deviation threshold
1.5
2
1.5
2
Qs, 5 (qcap of 5)
Qs, 10 (qcap of 10)
p¹ = 0.10
0.385 + 0.026
0.129 + 0.020
0.191 + 0.029 0.026 + 0.010
p" 0.05
0.405 + 0.034
0.142 + 0.028
0.211 + 0.044
0.035 + 0.016
static EMA, 0.01 (3 of 0.01)
static EMA, 0.001 (3 of 0.001)
p² 0.10
0.075 + 0.021
0.013 + 0.006
0.113 + 0.020
0.071 + 0.012
p" 0.05
0.211 + 0.034
0.050 + 0.019
0.118 = 0.029
0.072 + 0.016
harmonic EMA, 0.001 (8min of 0.001)
DYAL 0.001
(8min of 0.001)
p¹ 0.10
0.006 + 0.007
0.002 + 0.003
0.018 + 0.015
0.008 + 0.006
p" = 0.05
0.012 + 0.013 0.005 + 0.005
0.028 + 0.023
0.014 + 0.012
Table 9: Synthetic single-item stationary: Deviation rates, for two deviation thresholds d E {1.5, 2}, averaged over 200 randomly generated sequences of 10000 binary events (0 or 1), for target probability p" E {0.05, 0.1}. As an example, for p" 0.1, about 10% of the items will be 1, the rest are in the sequence, and the SMA predicts a probability p(t) every time point for o(*) (then updates), and we observe from the table that about 38% of time, max( ( p(o) 1.5 (f.e. p00) > 0.15 or p( 1.b), for the Qs SMA with qeap (top left). In this stationary setting, larger queues work better (as expected), and a lower 3mi of 0.001 performs best, for the EMA variants: harmonic EMA does best, and DYAL a close second.
denSSqueecco mmin Imin) // Create & return sequence of subsequences.
GenSD(prenSD) // Generate new SD.
1/ Parameters: recycle, Pmann Pmaz: Pmin- probs <- [ probabilities of the SD.
seg <- 1/// A sequence of items.
preuS +
1/ Repeat while unallocated mass is sufficient.
While len(seg) desiredLen:
While u(probs) > Pmin + Pmnin:
// Extend existing sequence
1/ sample probabilitics.
P + GenSD(preeSD) // get new SD.
Pmaz + min[left Pmin: Pmaa)
seg. "gg.extendGGnnuuSSS((]) Omin, Lenin)
p ~ ul( pmiin.Pmaz)) Pmux)) uniformly at random.
prev8D
append)) // add p to SD.
Return seq
If recycle: // reuse items '1', "2",
Random Random.shufllefe (probl) // random permute I/ item '1' gets probs(0), '2' gets probs[1), etc.
GenSubSeq(P (P, Omin: Amin)
Generate a subsoquence, via repeated
Return MakeMapp(robs))
sampling IID from SD P, long enough all item of P occurs 2 Omin times in it. counts +- () // An observation counter map.
Else: // Allocate new items (umused ids).
Return MakeNewSSDMapprrdde preuSD))
seg + // sequence of items.
DrawItem // Return a salient or noise item.
While min minncounts) Omin or len(seq) Lminn
sump <- 0.0 // via sampling.
DrawItem(P) // item o drawn.
p 2((0 1.0)] // Uniformly random.
seq app.edd)d(()
For o, prob P:
Update counts only for salient items. If P: // increment o's observed count.
sump sump prob
If p sump:
counts]o) (o) +- geetts.geefo, 0) +1
Return o // Done.
Return seg
Return UniqueNoiseltem)) // unique noise id.
Figure 18: Pseudocode for generating a sequence of subsequences. Each subsequence corresponds to a stable period, subsequence j being the result of drawing IID from the jth SD, piu).
subsequence j.j2 1, corresponding to one SD p()) The subsequence is long enough (drawn IID from p() so that min;epo0 count(i) 2 Omin, where count(i) is the number of occurrences of item in the subsequence. We may have a minimum (overall) length requirement as well, Lmin 2 0. Each pO) is created using the GenSD function
32
32'There variety of options for generating SDs and sequences. In particular, also experimented with the option to change only
few items, once they become eligible (their observation count reaching Omin), and obtained similar results. Because such variation is more complex and requires specifying the details of how item's change affects others GenSD(), i.c. changing all the items, and only
(e.g. how replaced by zero when all become eligible. Note that, under the reuse=1 setting, some items" probabilities may not change much, simulating no-change for some items. Experiments real streams provide further scenarios (Sect.
new old items), for simplicity,
43

---

Published in Transactions on Machine Learning Research (03/2026)
DYAL, 0.01 vs. Box, 100
Omin 10, rocycle
10, now
Omin 50, recycle
50, new
losses, wins of DYAL
7, 43
0, 50
8, 42 1.013,1.008
0, 50
log-loss (opt: 1.064 and 0.988)
1.120,1.108
1.182,1.113
1.025,1.010
Table 10: Losses and wins of DYAL, with Amin - 0.01, pairing it against Box (Sect. B.2 with window size
K- = 100 (which performed better than K of 50 or 200), on 50 sequences of 10k each (the setting of Table
B Thus DYAL has the lower loss on 43 of 50 in the recycle mode with Ornin = 10. Average losses are also shown (Ist number is Box's), and average optimal LogLossNS is 1.064 (0min-10 and 0.988 (Omin-50) All wins are highly significant (using the binomial sign test). With the lower Amin 0.001 for DYAL (not shown), DYAL has statistically more wins under the more challenging (for Box) new-items settings but has mixed performance under the recycle settings.
In GenSD, some probability, Pmin. is reserved for the noise items. Probabilities are drawn uniformly from what probability mass is available, initially Pmin: with the constraint that each drawn probability p should be sufficiently large, p 2 Pmin- Optionally, we may impose a maximum probability Pmaz constraint as well (Pmar < 1). Assume we get the set S = {pI,P2. )Pk), once this loop in GenSD is finished, then we will have Ep.spP: Pi S 1-Pmin and each probability satisfies the minimum and maximum constraints.
Once the set S of & probabilities is generated, GenSD() then makes a SD from S. Under the new-items setting (when recycle = 0), new item ids, |S| such, are generated, e.g. an item id count incremented and assigned, and the items are assigned the probabilities (and all the old items, salient in p(--1), , get zeroes, so discarded). Thus, as an example, the first few SDs could be p()) = [1:0.37, 2:0.55, 3:0.065), pl2: (4:0.75, 5:0.21, 6:0.01, 7:0.017}, and p(3) {8:0.8, 9:0.037, 10:0.15]. Under the recycle setting, with k probabilities generated, items 1 through k are assigned from random permutation 33 T of S: P(i) + Px(1) (thus item may gel P3, etc.). For example, with the previous probabilities, we could have p(1) (1:0.065, 2:0.37, 3:0.55) (and 0.015 is left for noise items), p(2) (1:0.75, 2:0.01, 3:0.21, 4:0.017} (a new item is added), p(3) {1:0.037, 2:0.8, 3:0.15]. Thus, under the recycle setting, in addition to changes in probability, the support of the underlying SD P may expand (one or more new items added), or shrink, with every change (i.e. from one stable subsequence to the next one).
D.3 Pairings on Synthetic Experiments and Additional Sensitivity Plots
We pair DYAL with another SMA and count the number of wins and losses, based on log-loss over each of the same 50 sequences of Table 3 (multi-item), and perform statistical sign tests. With Omin and the new-items setting, pairing DYAL with 8min 0.01 (best or near best of DYAL) against any of the other techniques, f.e. static or harmonic sparse EMA (with 3 in {0.001, 0.01, 0.02, 0.05, 0.1)) Qs, DYAL gets a lower log-loss on all 50 sequences. If we lower the Smin rate for DYAL to 0.001, we still get dominating performance by DYAL, but harmonic can win on a few one or two sequences. With 0win==1 and again the new-items setting, pairing DYAL with 3min 0.01 against all others, again we obtain the same dominating results for DYAL.
With the item-recycle setting, the differences between the best of the EMA variants and DYAL variants shrinks somewhat. For instance, with 050 harmonic EMA with Bmin 0.01 gets 9 wins (DYAL., 8min = 0.01, gets the remaining 41 wins), and if we use Bmin 0.001 for DYAL, DYAL loses 42 times to harmonic EMA with 8min = 0.01. Similarly, with =10 (high non-stationarity), in the recycle setting, we need to set 8min = 0.02 to get a dominating performance by DYAL, and setting it lower to Bman = 0.01 (which worked well for 05) yields mixed performance. Thus, the choice of rate 3min for DYAL can make a difference, although the operating range or the sensitivity to Amin is substantially lower for DYAL, than for other EMA variants (Fig. [], in particular when setting 8min to a low value. We observe this improved sensitivity on other data sources and settings as well.
33e probahility generation process tends to generate smaller probahilities with each iteration of the loop, this shuffling ensures that the same items are not assigned consistently high or low probabilities.
44

---

Published in Transactions on Machine Learning Research (03/2026)
108
CVA, B.0,
EVAL, DEL moyI
DVLL 0,00, soythil
CVA, 9.001 recycia Bart
207
105
gureet capeity
(a) loss vs binomial tail DYAL
(b) loss vs queue qcap in DYAL
Figure 19: DYAL performance sensitivity in synthetic multi-item setting: (a) to the binomial tail threshold (b) qeap (on the same 50 sequences of Table 3. under both new-item and recycle generation scttings, and with Omin = 50).
The Box SMA (Sect. B.2p performed somewhat better than static EMA, with an appropriate window size K, which is about 100 for the setting of Table 3 (K =50 and 500 were strictly inferior, and 200 slightly worse, in terms of LogLossNS and other measures). Like EMA, Box is inflexible and requires tuning the fixed K for different input streams, and unlike EMA its rigid space consumption is prohibitive. Table [0 summarizes paired comparisons with DYAL, under the two recycle and new-item settings and Omin 10 and 50. DYAL with 8min = 0.01, consistently outperforms it.
Fig. shows two plots of DYAL sensitivity to the binomial threshold and the queue capacity, showing DYAL performance fairly stable under a reasonable range.
E Additional Experiments on Real-World Sequences
Of the 104 Expedition sequences, there are 19 sequences with length above 5k, median length being 13k. We averaged the log-loss performances of DYAL, as we change its Smin, over these 19, as well as over the sequences with length below 1k, of which there are 46 such, with median of 280 observations. Fig. a) shows that lowering the Amin works better or no worse, for longer sequences, as expected, while the best performance for the shorter sequences occurs with higher 3min 0.01 (increased non-stationarity). A similar patterns is also seen in Fig. 20(a) for Qs tochnique. Larger queue capacitics work better for longer sequences, but because shorter sequences dominate the 104 sequences, overall we get the result that a qcap of 3 works best for this data overall. We also note that the shorter sequences yield a lower log-loss (both figures of 20). This is expected and is due to our protocol for handling noise: for methods that allocate most their initial mass to noise (unallocated) and when this agrecs with the noise marker referoe, one gets low loss. For instance, at L 1, the noise marker marks the next item as noise, and with predictor that has all its probability mass unallocated, log- loss is 0 (ogllssSRlleNS)) in Fig. b)). As the sequence grows longer, and more salient items are discovered, the average loss can (initially) go up (even though learning is taking place) 34
E.1 More on Nonstationarity
A way to see evidence of non-stationarity is to look at the maximum of the learning rates, in the Map of DYAL, as a function of time (similar to Fig. 15 Whenever this max-rate goes up, it is evidence that an item
I4In learning under non-stationarity, it is expected that performance curve is not monotonic and can degrade at times even though learning is taking place all the time. In the above experiments, one can imagine all SMAs beginning with all mass allocated dummy item initially, forcing them learn the appropriate probability allocation the noise portion as well as the salient items, and in that case, we'd observe an initial worst-possible loss and the gradual lowering of the loss.
45

---

Published in Transactions on Machine Learning Research (03/2026)
The capaciy for
sago
Quen 19
332
32-
DM0 10xegeor
30-
330
of 1Dssgg S4
2.8
10
St
Oce1 seessu Or 40005 Dk
48
329-
on-45 sep on IX
OWL, mf-osteor0001,, on 19 sege
2.
527
18
00001
0.003
64030
$5200
leeming 1969 perametes DMUL
lirshel
(a) longer shorter sequences (DYAL and Q5).
(b) Loss on 19 longest sequences us. binomial threshold.
Figure 20: log-loss (a) 19 longest. sequences (above 5k) vs 46 shorter sequences (below 1000). On the longer sequences lower Amin for DYAL, and higher qeap for Qs improves performance. (b) loss binomial threshold on 19 longest sequences. Setting the value in 3 to 7 works well, and 8min = 0.001 does better than Bmin = 0.02 on these sequences.
max rale
0.25
maxium Eates
maximum
0.3
0.0
0.-
0.3
0.05
14000
2000
4000
8500
5000
10000
12000
14200
time
(number
(a) Masquerade, 3x sequence
(b) Masquerade, 3x sequence 2.
Figure 21: The maximum learning rate of DYAL as a function of time, on two sequences from Masquerade, where each sequence is concatenated with itself 3 times. The rate repeatedly jumps up, exhibiting pulsing or spiking patterns, indicating change.
is changing probability, for instance new item is replacing old. We took a sequence and concatenated with itself a few times, 3x in case of Masquerade sequences (Masquerade sequences are uniformly 5k long each). Fig. 21 shows two examples (thus [A, C). i.e. A followed by C, becomes (A,C,A,C,A,C).. If the start of a sequence has a distribution substantially different from the ending then, when concatenated, we should repeatedly see max-rate jump up 35 We looked at many such plots. The max-rate repeatedly goes up in plots of all sequences from the 52-scientists (whether with default. parameters or with Bmin of 0.05), and all except of the 50 Masquerade sequences, and similar pattern on all except the shortest handful of sequences from 104 Expedition sequences. In other experiments (not shown), when updating (learning) is turned off midway along sequence (whether on Unix or Expedition), the loss significantly increases (compared to continuing the updates).
Nonnstaatony the level of predicting the next item given current priors. There may be higher-level stationary regu- larities, e.g. periodicities, that could explain the lower-level or short-term changes.
46


Reviews:
[{"review_id": "GGUVRtgVBZ", "reviewer": "Reviewer_79Vu", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses the problem of online multiclass probabilistic prediction in a \"lifelong\" and \"open-ended\" setting. This setting is characterized by an unbounded stream of items where the set of possible items is not known in advance (open-ended), new items can appear at any time, and the underlying probability distributions are non-stationary (changing over time). The authors focus on resource-constrained predictors that must maintain space and time efficiency while adapting to these changes.\n\n## The key contributions include:\n\n1. **Problem Formalization:** The paper defines the \"lifelong open-ended probability prediction\" problem  , emphasizing the challenges of handling unbounded item sets, non-stationarity, and the \"stability-plasticity\" dilemma (balancing fast adaptation with stable estimation) .\n    \n2. **New Evaluation Protocol:** To handle the evaluation of open-ended probability predictors where the true set of items is unknown and potentially infinite, the authors propose a bounded version of logarithmic loss (log-loss) . This involves a protocol using \"filtering and capping\" (FC) to handle small probabilities and a \"noise-marker\" (referee) to handle new or infrequent items, ensuring the loss remains finite and informative .\n    \n3. **Algorithmic Contributions (SMAs):** The authors develop and analyze \"Sparse Moving Average\" (SMA) techniques :\n    \n    - **Sparse EMA:** An adaptation of Exponentiated Moving Average for sparse vectors, including a \"harmonic decay\" variant for learning rates .\n        \n    - **Qs (Queues):** A count-based method using small per-item queues to estimate probabilities .\n        \n    - **DYAL (Dynamic Adjustment of Learning):** A novel hybrid method that combines Sparse EMA with Qs . DYAL maintains per-item  learning rates that dynamically adjust. It uses the Qs estimator as a \"gate\" to detect changes (via a binomial-tail test based on KL divergence) and reset the EMA weights and learning rates when significant deviations occur.\n        \n4. **Empirical Evaluation:** The authors conduct extensive experiments on both synthetic data (with controlled non-stationarity)  and real-world datasets (Unix command sequences and \"Expedition\" system logs). They compare the proposed methods against baselines (static EMA, Qs) and analyze sensitivity to various parameters.\n\n## Limitations\nThe primary limitations involve the assumption of intermittent stability, as the \"listen-to-queue\" logic requires sufficient stationary periods to accumulate evidence for resets . Furthermore, while DYAL exhibits reduced sensitivity to learning rate selection, its performance remains dependent on the calibration of the binomial-tail threshold and the minimum rate $\\beta_{min}$. Developing more automated or adaptive mechanisms for these hyperparameters would significantly enhance the algorithm's robustness in highly volatile drift environments.", "claims_and_evidence_yn": "Yes", "claims_explanation": "While the paper provides a good mix of theoretical motivation and empirical results, the evidence supports the claims with some caveats:\n\n- **DYAL's Performance:** The claim that DYAL strikes a \"superior stability-plasticity tradeoff\"  is well-supported by the synthetic experiments where the nature of the change is controlled (e.g., oscillating probabilities) . In these settings, the mechanism of DYAL (resetting rates upon change detection) is shown to react faster than static EMA while remaining more stable than pure Qs .\n    \n- **Real-World Applicability:** The evidence on real-world data is mixed but honest. On the \"Expedition\" sequences (internal non-stationarity), DYAL outperforms others significantly. However, on the Unix traces (external non-stationarity), specifically the Masquerade dataset, static EMA with a high learning rate sometimes outperforms DYAL . The authors transparently discuss this, attributing it to the potential lack of sufficient \"stable\" periods for DYAL's mechanism to be advantageous over a consistently aggressive learner . This nuance adds credibility but slightly weakens the claim of universal superiority.\n    \n- **Evaluation Protocol:** The reliance on a \"noise marker\" (referee) introduces hyperparameters (like $c_{NS}$) that significantly influence the absolute loss values . While the authors show sensitivity analysis, the comparison of methods ultimately depends on this specific, somewhat complex, evaluation harness. The evidence is convincing that _within this harness_ DYAL performs well, but the external validity of the harness itself is a complex argument relying on the proofs in the appendix.", "audience_yn": "Yes", "audience_explanation": "The problem of \"open-ended\" learning is increasingly relevant as machine learning moves towards continual, lifelong, and streaming settings where fixed vocabularies cannot be assumed (e.g., in large language models adapting to new tokens, user behavior modeling, or IoT data streams).\nThe specific combination of Sparse EMA with a queue-based change detector (DYAL) and the dynamic per-item learning rates is a novel contribution to the toolkit of online learning algorithms.", "requested_changes": "- **Clarify the \"Noise Marker\" Impact:** The evaluation relies heavily on the \"noise marker\" to bound the loss for new items. While Section 2.2.2 describes it, the impact of the specific choice (default $c_{NS}=2$) on the _relative_ ranking of algorithms should be made more explicit in the main text summary. The paper mentions that varying $c_{NS}$ changes the win counts , but a clearer \"executive summary\" of _when_ (under what noise definition) DYAL fails or succeeds would be helpful.\n\n- **Threshold Sensitivity Analysis**: Provide a more explicit trade-off analysis between \"False Alarms\" (false changes) and \"Missed Detections\" (misses) for the binomial-tail threshold, currently set at a default of 5. This is vital for justifying the 99% confidence level which aims to avoid resetting while the EMA estimate should be kept across diverse drift intensities.\n\n- **Pruning Protections**: Clarify if the \"heart-beat\" pruning logic—which removes items in order of least frequency when the map reaches a maximum capacity—contains mechanisms to protect lower-frequency salient items from being discarded during transient shifts.\n\n- **Baselines Comparison:** The paper compares DYAL primarily against its own components (EMA, Qs) and a \"Box\" (sliding window) predictor . It would be beneficial to briefly explicitly state why more complex concept-drift baselines (e.g., ADWIN mentioned in related work  ) were not suitable or included as direct competitors .", "broader_impact_concerns": "The authors do not explicitly list a Broader Impact statement, but the work is primarily algorithmic and theoretical.\n\n- Positive: Efficient algorithms for open-ended prediction can improve the energy efficiency of learning systems  by allowing resource-constrained devices to adapt without expensive retraining.\n    \n-  Negative:  As with any user modeling technology (e.g., predicting Unix commands), there are potential privacy implications if improved prediction enables more  profiling of user behavior. However, this paper focuses on the fundamental probability estimation mechanics rather than specific surveillance applications.", "additional_comments": "- There's a typo in page 1 introduction section instead of \"We foucs\" it should become \"We focus\"\n\n- Color Blindness Accessibility: Figures 6, 10, and 12 rely heavily on distinguishing \"blue,\" \"green,\" and \"orange\" lines to compare EMA variants and true probabilities. Adding different line styles (dashed, dotted, or markers) would make these critical performance plots more accessible.", "extra_scores": {}, "extra_text": {}}, {"review_id": "RS7ITHl7O0", "reviewer": "Reviewer_Zdsi", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies online multiclass probability prediction on lifelong, open-ended streams, where the set of items is not known in advance and may grow without bound or even shrink. The predictor is space-limited and should only track \"salient\" items above threshold $p_{min}$. Some of the claimed contributions are:\n\n- The proposal of Sparse Moving Average (SMA) techniques. Specifically, they introduce DYAL (Dynamic Adjustment of Learning), a hybrid predictor that combines a Sparse EMA with a Queue-based estimator (Qs). DYAL uses the queue to detect significant changes via a binomial-tail test and resets the EMA learning rates per-item (per-predictand) accordingly\n- The formulation of a \"bounded log-loss\" metric and a protocol (Filter & Cap) to evaluate open-ended probability streams where new or noise items appear frequently, effectively preventing infinite loss on unseen items\n- A convergence bound for sparse EMA in stationary settings, proving expected first-visit time to a probability band is $\\mathcal{O}(\\beta^{-2})$.\n- Evaluations on synthetic data and real-world datasets (Unix commands and an internal \"Expedition\" system), claiming DYAL strikes a superior stability-plasticity tradeoff compared to static EMA or simple queuing.", "claims_and_evidence_yn": "No", "claims_explanation": "The claims in the paper are partially verified, and there are some gaps in the paper. I will summarize some of these gaps as follows:\n- The central evaluation (AvgLogLossNS) depends on a noise-marker referee and hyperparameters ($p_{min}$, $p_{NS}$, $c_NS$), that can change the outcomes. Even the paper itself notes artifacts (e.g., low loss on short sequences due to \"noise\" handling). This makes it hard to interpret improvements as \"better probability prediction\" rather than \"better alignment with the referee.\"\n- The claim of \"superiority\" is insular. The paper completely fails to compare DYAL against established State-of-the-Art (SOTA) methods for concept drift and streaming probability estimation, such as ADWIN (Adaptive Windowing), Hoeffding Trees, or even standard Online Logistic Regression.\n- The claim of superior performance is evaluated based on some 25 years old dataset of Unix commands, which is not satisfying. Modern \"lifelong\" learning involves high-dimensional, rapid-fire streams (e.g., clickstream data, IoT sensor logs, server metrics). Proving performance on small-vocabulary command logs is not convincing evidence of scalability or utility for modern open-ended problems. Even the \"Expedition\" dataset is proprietary, and hence, makes the evaluation harder.\n- The convergence proof applies only to Static EMA, a standard technique. It does not cover the paper's main contribution, DYAL, which involves dynamic resetting of learning rates based on a queue trigger.", "audience_yn": "Yes", "audience_explanation": "Yes, but with a narrow and conditional appeal. This would be an interesting work for researchers working on streaming / online learning under concept drift, especially for discrete or open-vocabulary domains (e.g., event streams, logs, user actions). However, due to the following reasons, the interest would be limited:\n- The novelty is modest, and the main algorithm is a heuristic hybrid of known methods.\n- The main evidence is protocol-dependent and relies on a referee. Readers not already aligned with this framing may be unconvinced or skeptical.\n- Weak engagement with related work, especially in the empirical investigations of the method.", "requested_changes": "- Add at least two strong, conceptually different baselines and re-run core experiments. Some examples are: Adaptive window / drift-aware baseline / Bayesian smoothing baseline for categorical prediction.\n- Compare against a simple Online Logistic Regression (e.g., via SGD) or a basic LSTM/GRU predictor.\n- Evaluation on Modern Public Datasets: Report results on at least one recognized, modern, high-dimensional streaming dataset, like: Click-through Rate Prediction or IoT Sensor Streams.\n- Provide Brier Score or standard Log-Loss (without the referee filter) on the subset of salient items", "broader_impact_concerns": "The current manuscript focuses entirely on algorithmic performance and mathematical bounds. Given the stated applications in personalization and system monitoring, the lack of a dedicated Broader Impact Statement could be of concern.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "KH8AKjyYfx", "reviewer": "Reviewer_XK7T", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies online probabilistic multiclass prediction under lifelong, open-ended, and non-stationary streams with strict memory constraints. It introduces sparse moving averages (SMAs): (i) sparse EMA with fixed or harmonically decaying learning rates, (ii) a queue-based counting method (Qs), and (iii) DYAL, a hybrid that maintains per-item learning rates and small queues with binomial KL-based change tests to reset estimates. For evaluation with emerging items, the paper proposes a bounded variant of log-loss based on filtering/capping predictions and a simple noise referee; it provides a convergence bound for sparse EMA and empirical evidence that per-item dynamic rates (DYAL) often yield superior stability–plasticity trade-offs.", "claims_and_evidence_yn": "Yes", "claims_explanation": "- The DYAL hybrid with per-item harmonic decay and queue-triggered resets via a binomial KL tail test is a pragmatic and interesting design that connects rate-based EMA with counting-based change detection.\n-  Synthetic experiments explore regimes with large and frequent probability shifts versus heterogeneous “drifts,” and report deviation rates and log-loss under controlled stability windows.", "audience_yn": "Yes", "audience_explanation": "- The problem setting is well-motivated: open-ended, space-bounded, probabilistic prediction where the support evolves and can grow unbounded.\n- The setting is practically relevant (UI agents, personalization, large categorical spaces) and under-explored relative to more common frequency/heavy-hitter tasks.\n- The proposed evaluation methodology could be useful for broader work on probabilistic prediction with sparse supports, where unseen items are common.", "requested_changes": "- Theoretical results are limited to convergence of sparse EMA in stationary periods; DYAL’s central mechanisms (per-item rate, reset criteria) lack formal guarantees (e.g., bounds on detection delay, false alarms, regret under drift).\n- Real-world evaluations are only briefly mentioned; details, datasets, and comparative results versus strong baselines are not fully evident in the excerpt.\n- Some typographical errors and occasional digressions (e.g., philosophical epigraphs) distract from the core narrative; the semi-distribution vs distribution terminology could be streamlined.", "broader_impact_concerns": "N/A", "additional_comments": "- How sensitive are results to the referee parameters (c_NS, windowing) and to p_min/p_NS choices? Could you report calibration metrics and log-loss across a grid of these settings to show robustness?\n- In real-world datasets, what are the item cardinalities, memory budgets (as a function of p_min), and throughput? Can you include timing and memory profiles, as well as ablations on DYAL’s significance threshold and beta_min?", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)