Explain what LM Sherlock means in the last message. Whole topic thread: sidereal
5d
I’m a bit late to the party, but I just installed RC2 for Anki 24.10 and optimized my deck with FSRS5. My RMSE went down from 2.78% to 1.57% (with the python version optimizer, 1.61% with built-in), but what concerns me is how nearly all of my cards now have a difficulty of 0%, including cards with reviews:

123123232323223333333
122232233331333
and even 12333

My requested retention is 96% but true is 98.9% for my full 189 days of use (I reformed my habits less than 5 months ago and have ignored reviews since 6/14/2024 when optimizing), and I don’t review tons of cards every day. I know that difficulty won’t affect my reviews, but it certainly doesn’t feel like 0% difficulty each time I see a card.

I even tried to bump up my desired retention to 98% and rescheduled cards on change, but it’s still 0% difficulty for most cards and the peak workload I’ll have to do is 41 cards in 80-89 days with the projected forecast.

I also simulated some reviews on a blank new card and answered 1223 and difficulty went from 65% to 36% to 17% to 0%. Interestingly enough, the FSRS Visualizer says it should be 65.38% to 57.46% to 51.71% to 37.56%. Here are my FSRS v5.2.0 optimized parameters: [6.7006, 18.4477, 18.1612, 18.1421, 6.8842, 0.8254, 1.2354, 0.2736, 1.8441, 0.0, 1.3177, 2.0384, 0.0044, 0.3869, 2.4027, 0.3577, 3.2153, 0.8074, 0.937]

Can I do anything about this? I would love to hear any experts’ thoughts on this and am willing to run tests. I’m not sure if this is a bug or (more likely) it’s just me, but here are some of my stats (after optimization, and, of course, I’m testing with backups):

image
image
716×876 35.8 KB

image
image
718×880 19.2 KB

image
image
719×422 10.8 KB
Before optimization:

image
image
699×874 33.2 KB

1 Reply



Reply

82
views

1
link






Expertium
5d
and have ignored reviews since 6/14/2024 when optimizing

Please keep in mind that ignores entire cards, not just a part of their review history. Try turning it off (setting a very old date).

I even tried to bump up my desired retention to 98% and rescheduled cards on change, but it’s still 0% difficulty for most cards

Increasing desired retention won’t affect D.

the FSRS Visualizer says it should be 65.38% to 57.46% to 51.71% to 37.56%.

It’s outdated.

First of all, the Rust version (built-in) and the Python version are now equally equal in the latest RC, so there is no reason to use the Python version. Previously, the Rust version was mildly worse because of a bug, not anymore.

Anyway, this does seem quite strange. @L.M.Sherlock pinging so that you take a look. It probably has something to do with “Ignore cards reviewed before”.

huggingface.co

Fsrs4anki Previewer - a Hugging Face Space by open-spaced-repetition
Discover amazing ML apps made by the community

With this user’s parameters and ratings=1,2,2,3, I get difficulty history: 0,6.9,4.2,2.5,1.0


Solution




Reply

Keks
5d
For some reason, the w9 parameter affects the complexity of the cards. [Imgur] 0.4072, 1.1829, 3.1262, 15.4722, 7.2102, 0.5316, 1.0651, 0.0234, 1.6160, 0.1544, 1.0824, 1.9813, 0.0953, 0.2975, 2.2042, 0.2407, 2.9466, 0.5034, 0.6567 [Imgur] 0.4072, 1.1829, 3.1262, 15.4722, 7.2102, 0.5316, 1.0651, 0.0234, 1.6160, 0.2544, 1.0824, 1.9813, 0.0953, 0.2975, 2.2042, 0.2407, 2.9466, 0.5034, 0.65…
I also encountered strange behavior of D when using the “Ignore cards reviewed before” function. However, in my case, the date was indicated for the future.

The answer that was given to me:

Is it a problem? It’s intended: Please see the section “Ease Factor to Difficulty”.

GitHub

How does the scheduler work?
A modern Anki custom scheduling based on Free Spaced Repetition Scheduler algorithm - open-spaced-repetition/fsrs4anki


Solution




Reply

sidereal
5d
I tried optimizing again without ignoring any cards and got a lower log loss of 0.0634 and a lower RMSE of 0.98%; however, all cards have 0% D now. Average interval increased to 1.77⁩ years and average stability increased to 4.87 years—from 3.9 months and 3.49 years respectively.

New params: [5.3131, 18.864, 18.9563, 18.5678, 6.7846, 1.0013, 1.2118, 0.2608, 1.9724, 0.0, 1.4316, 2.0897, 0.001, 0.4293, 2.4399, 0.4069, 3.5015, 0.7426, 0.8977]
Percent changes from ignoring reviews to not: [-20.7, 2.3, 4.4, 2.3, -1.4, 21.3, -1.9, -4.7, 7, 0, 8.6, 2.5, -77.3, 11, 1.5, 13.8, 8.9, -8, -4.2]


Solution



Reply

Keks

3
5d
6.7846, 1.0013, 1.2118, 0.2608


GitHub

The Algorithm
A modern Anki custom scheduling based on Free Spaced Repetition Scheduler algorithm - open-spaced-repetition/fsrs4anki

1=6.7846
2=5,0617
3=0,3909
4=-12,30

For some reason, your w7 parameter is very large. Therefore, with any answer, your difficulty tends to D0(4)=0% or 1.

rating history: 1,1,1,1
difficulty history: 0,6.8,2.4,1.0,1.0


Solution




Reply

sidereal
4d
Yeah, w[7] is super high. I’m also not sure why w[9] and w[12] are at 0 (when they were higher with FSRS 4.5), which is causing that stability ballooning. Here are my revlogs if anyone wants to run analysis: Google Drive. Here are also some cursory python plots comparing my FSRS5 params to my previous ones and default

v5 vs Pre-v5
v5 vs Pre-v5
1500×600 60.6 KB

my v5 vs default v5
my v5 vs default v5
1500×600 65.5 KB

Solution



Reply

Gilfar
4d
w[7] steers the distribution of card difficulty, low steers to 100% and high moves it to 0%
in your case the parameter is very high so at high retention all cards will end up at 0%


Solution




Reply

L.M.Sherlock
2h
I don’t think it’s problematic. Your true retention is very high, so FSRS think it’s unnecessary to distingue your cards’ difficulty.

For detail diagnosis, you can use fsrs4anki/fsrs4anki_optimizer.ipynb at main · open-spaced-repetition/fsrs4anki to optimize your collection, and post these figures:

image
image
1920×2151 195 KB

Solution




Reply

sidereal
2h
image
image
640×442 29.1 KB

image
image
1384×997 84 KB

image
image
649×438 23.7 KB

Solution




Reply

L.M.Sherlock
1h
The model fits your reviews pretty well.


Solution




Reply

sidereal

1
29m
I can’t deny that. An RMSE of .96% is astonishing.

I’m unsure if the “removal” of (the power of) a few parameters with D = 1.0 being constant for almost all cards is beneficial for FSRS5 or just a thing it’s dealing with due to the D formula change from v4.5. Would it do better with some sigmoid function or exponential factor? Maybe something will happen in a few years that completely revamps D, or maybe it won’t exist anymore (I know that people have tried tons of things). It’s too bad that I won’t be able to sort cards by the difficulty statistic for now, but at least it doesn’t seem to be limiting FSRS5’s performance for me. I’d love to hear any of your thoughts.


Solution




Reply

L.M.Sherlock
10m
OK. I figure it out. The mean_reversion is using the raw init_difficulty which hasn’t been clamped.

github.com
open-spaced-repetition/fsrs-rs/blob/7477d2bf20749e393c437007564bd87519b7e70d/src/model.rs#L106-L117
fn mean_reversion(&self, new_d: Tensor<B, 1>) -> Tensor<B, 1> {
    let rating = Tensor::from_floats([4.0], &B::Device::default());
    self.w.get(7) * (self.init_difficulty(rating) - new_d.clone()) + new_d
}
pub(crate) fn init_stability(&self, rating: Tensor<B, 1>) -> Tensor<B, 1> {
    self.w.val().select(0, rating.int() - 1)
}
fn init_difficulty(&self, rating: Tensor<B, 1>) -> Tensor<B, 1> {
    self.w.get(4) - (self.w.get(5) * (rating - 1)).exp() + 1
}
I will fix it soon.