# Live Knowledge Tracing: Real-Time Adaptation using Tabular Foundation Models

Mounir Lbath, Alexandre Paresy, Abdelkayoum Kaddouri, Alan André, Alexandre Ittah, Jill-Jên Vie

<figure><img src="image_1.png" /></figure>

## To cite this version:

Mounir Lbath, Alexandre Paresy, Abdelkayoum Kaddouri, Alan André, Alexandre Ittah, et al. Live Knowledge Tracing: Real-Time Adaptation using Tabular Foundation Models. 2026. [hal-05495977](https://hal-05495977v1)

HAL Id: hal-05495977

https://inria.hal.science/hal-05495977v1

Preprint submitted on 5 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Live Knowledge Tracing: Real-Time Adaptation using Tabular Foundation Models

Mounir Lbath<fnref n="1" />, Alexandre Paresy<fnref n="1" />, Abdelkayoum Kaddouri<fnref n="1" />, Alan André<fnref n="1" />, Alexandre Ittah<fnref n="1" />, and Jill-Jënn Vie<fnref n="2" />

**Abstract.** Deep knowledge tracing models have achieved significant breakthroughs in modeling student learning trajectories. However, these architectures require substantial training time and are prone to overfitting on datasets with short sequences. In this paper, we explore a new paradigm for knowledge tracing by leveraging tabular foundation models (TFMs). Unlike traditional methods that require offline training on a fixed training set, our approach performs real-time "live" knowledge tracing in an online way. The core of our method lies in a two-way attention mechanism: while attention knowledge tracing models only attend across earlier time steps, TFMs simultaneously attend across both time steps and interactions of other students in the training set. They align testing sequences with relevant training sequences at inference time, therefore skipping the training step entirely. We demonstrate, using several datasets of increasing size, that our method achieves competitive predictive performance with up to 273x speedups, in a setting where more student interactions are observed over time.

**Keywords:** Knowledge Tracing · Student Modeling · Tabular Foundation Models

## 1 Introduction

Knowledge Tracing is the process of modeling a student's knowledge and skills over time as they interact with learning activities. Deep Knowledge Tracing (DKT) models [1] have achieved impressive performance in modeling student learning trajectories, but they need to be retrained from scratch on new datasets, and may have lower performance in cold-start settings [2], when few interactions are available for new students.

The promise of foundation models is to pretrain deep learning models on large curated datasets so that they can generalize on new datasets without additional training or fine-tuning. By giving examples of tasks in the prompt, foundation models have shown to be able to generalize [3]: this is called *zero-shot prompting* or *in-context learning*. Large Language Models (LLMs) are the most famous example of foundation models, but these techniques have been applied to tabular data [4] or graph data as well, with applications to finance, healthcare, and fairness [5].

<footnote n="1">¹ École Polytechnique, Palaiseau, France first.last@polytechnique.edu</footnote>

<footnote n="2">² Inria, Soda Team jill-jenn.vie@inria.fr</footnote>
In this work, we investigate an alternative paradigm: leveraging tabular foundation models to perform knowledge tracing without training. The goal of this paper is to evaluate their performance in commonly encountered knowledge tracing datasets.

We are particularly interested in a “live” setting where we observe new interactions over time. For example, in programming competitions, contestants solve problems in a limited amount of time by sending their codes to an online judge which evaluates these submissions as correct or incorrect. In this context, contestants need to quickly estimate their likelihood of solving a problem by observing the records of other contestants at a given time, as well as their own performance. As we show, in this setting it is not feasible to retrain from scratch deep learning architectures.

Our contributions are as follows. We introduce a lightweight knowledge-tracing pipeline built on tabular foundation models that operates via in-context learning. In this paper, we define *live knowledge tracing* (liveKT), a setting in which new interactions are observed over time for both train and test students. We describe an online learning protocol that updates the model with batches of student-item interactions, enabling rapid adaptation to new items and sparse data. We empirically demonstrate that the proposed approach is faster to deploy and more sample-efficient than standard deep models while retaining competitive predictive performance.

## 2 Related Work

Early approaches such as Bayesian Knowledge Tracing (BKT) [6] rely on a simple hidden Markov model of latent skill mastery. Over the past decade, neural methods such as Deep Knowledge Tracing (DKT) [1] and attention-based variants like Attentive Knowledge Tracing (AKT) [7] have become popular. While these architectures can be highly accurate, it is unclear what their performance would be on smaller data. Logistic regression has also shown good results [8, 9], notably comparable to DKT when updated in an online way [8]. Previous work has also studied their performance in cold-start settings [2], where the model must make predictions for new students (or items) with little to no interaction history.

Independently, foundation models have shown that pretrained models can adapt to new tasks by conditioning on examples at inference time. Recent attempts of using LLMs for knowledge tracing have yielded mixed results [10, 11]. This is likely because LLMs are mainly trained on raw text, while knowledge tracing data is intrinsically tabular: student & skill identifiers, and binary outcomes. While there is a growing trend toward open-ended knowledge tracing models [11] that incorporate richer data, this may introduce significant privacy risks. In contrast, tabular foundation models (TFMs) can leverage pseudonymized data, and are mostly self-hosted. On some small datasets, TFMs can achieve in a few seconds a performance comparable to gradient boosting decision trees tuned during hours [12]. TabPFN [13], where PFN stands for *prior-fitted network*, is
a transformer-based model pre-trained on millions of synthetic datasets generated from structural priors (e.g., Bayesian neural networks or structural equation models). While most machine learning models fit on training sets, TFMs predict on test sets while feeding the training set at test time: this new paradigm is known as in-context learning. TabICL [4] is another TFM with comparable performance, specialized in classification.

Despite this promise, TFMs remain under-explored for knowledge tracing, especially in streaming or live settings where new items emerge over time. We address this gap in this paper.

## 3 Tabular Foundation Models for Live Knowledge Tracing

In the traditional knowledge tracing framework, we separate train and test students so that information is fully available for train students while test students' samples are observable up to time $T-1$ and the model has to predict the next outcome at time $T$. In what we call *live knowledge tracing* (liveKT), at time $T$, we observe samples for train students up to time $T$ and samples for test students up to time $T-1$ and we have to predict the next outcome for test students. We believe this alternate setting is more realistic and closer to what happens in the classroom, where all students complete tasks over time.

Tabular foundation models receive a table with rows and columns and have to infer at test time the values of a missing column for "test" rows. They compute an embedding for every cell using transformers, and decode the cells to infer. To know more about the architecture of tabular foundation models, we refer to an educational implementation of TabPFN called nanoTabPFN [14].

We now describe how the data is encoded into cells. Rows represent sequences of interactions for a given student, to be continued: each row $i$ contains

$$ \mathcal{R}_i = (q_{1:T}, s_{1:T}, c_{1:T}) = (q_1, q_2, \dots, q_T, s_1, s_2, \dots, s_T, c_1, c_2, \dots, c_T) $$

for student $i$ where $T$ is fixed for all rows, $q_t, s_t, c_t$ represent respectively the question ID, skill ID and correctness at time $t$ for student $i$ (subscript $i$ is omitted in questions, skills, correctness for clarity). If the number of interactions for a given student is less than $T$ then their data is aligned on the right and the remaining cells are filled with NA values, sometimes set to 0 if the model requires it. Students are split into $I_{train}$ and $I_{test}$. The goal is to model for every test student, $p(c_T|q_{1:T}, s_{1:T}, c_{1:T-1}, \mathcal{D}_{train})$ where $\mathcal{D}_{train} = (\mathcal{R}_i)_{i \in I_{train}}$, that is, all train students rows. While TFMs such as TabPFN or TabICL can feed $\mathcal{D}_{train}$ at test time, typical models such as DKT or AKT have to be trained on $\mathcal{D}_{train}$ before they can output a prediction for test users.

## 4 Experiments

We evaluate the proposed approach on several datasets. Data is truncated to the first $T \in [5, 10, 15, 20]$ samples per user to simulate a live setting. Students
are split to 80% train and 20% test. Baselines include standard DKT and AKT, tabular foundation models PFN and ICL and a gradient-boosted decision tree GBM. Metrics are the area under the ROC curve (AUC) and wall-clock latency for producing predictions. Experiments were ran on one NVIDIA RTX 4000 GPU with 20 GB of RAM during at most 115 epochs. We release our code<fnref n="3" />.

## 4.1 Datasets

We use the following datasets described in Table 1:

- ASSISTments2009 [15] and ASSISTments 2012, popular knowledge tracing datasets which stem from an online tutoring system developed at Worcester Polytechnic Institute;

- POJ and Codeforces are datasets extracted from online judges Peking University Online Judge and Codeforces, available online<fnref n="4" />.

<table><caption>**Table 1.** Statistics for the ASSISTments, POJ, and Codeforces datasets</caption><thead><tr><th rowspan="2">Domain</th><th colspan="2">ASSISTments 2009</th><th>POJ</th><th>Codeforces</th></tr><tr><th>Mathematics</th><th>Programming</th><th>Programming</th><th>Programming</th></tr></thead><tbody><tr><td># Students</td><td>4,217</td><td>433</td><td>500</td><td></td></tr><tr><td># Questions</td><td>26,688</td><td>2622</td><td>5857</td><td></td></tr><tr><td># Skills</td><td>26,688</td><td>85</td><td>36</td><td></td></tr><tr><td># Interactions</td><td>346,860</td><td>796,249</td><td>754,772</td><td></td></tr></tbody></table>

## 4.2 Models

The traditional knowledge tracing models are sourced from pyKT [16] with default hyperparameters. **DKT** is based on a recurrent neural network (RNN) that models student knowledge as a hidden state evolving over time. **AKT** [7] is based on an attention mechanism over the sequence length *T*, leading to complexity *O*(*T*²). It selectively focuses on the most relevant past interactions when predicting student performance. Models are run with early stopping on a validation set.

As baselines, **LR** is logistic regression directly fed on features treated as categorical data. **GBM** refers to the histogram-based gradient boosting model provided by the skrub<fnref n="5" /> and scikit-learn libraries [17], similar to LightGBM.

**TabPFN** and **TabICL** are tabular foundation models designed to generalize across diverse tabular tasks. TabPFN refers to the version 2.5 that is proprietary but available for non-commercial use, whereas TabICL is released under a free

<footnote n="3">³ https://anonymous.4open.science/r/liveKT-B356/README.md</footnote>

<footnote n="4">⁴ https://github.com/JonathanSilver/pyKT/tree/main/data</footnote>

<footnote n="5">⁵ https://skrub-data.org/</footnote>
license. TabPFN has complexity $O(N^2T + NT^2)$ while TabICL has complexity $O(N^2 + NT^2)$ where $N$ is the number of users and $T$ is the sequence length. Please note that TabICL and TabPFN were all run on default hyperparameters. The same instance of these models is used for all datasets.

<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Results for Assistments 2009 and POJ datasets. Top is performance as AUC, bottom is total time spent between (optional) training and testing.</figcaption></figure>

# 5 Results

Results are shown in Table 2 and Figure 1. Time refers to the median time per $T$ across $T = 5, \dots, 20$ for training and testing for all epochs. TabPFN and TabICL do not perform training but they look at training data once, at test time. Across datasets, tabular foundation models achieve competitive AUC compared to fully trained deep models while dramatically reducing time. On a massive dataset like Assistments 2009, AKT takes 418 seconds to converge after 115 epochs, while TabPFN finishes in 1.53 seconds. AKT is the slowest model by far, taking around 3.6 seconds per epoch. TabICL is slightly slower than TabPFN, although its
asymptotic complexity is better. On the POJ dataset, tabular foundation models clearly outperform deep knowledge tracing approaches, and are comparable with GBM except for *T* = 20. In particular, DKT has a very low AUC for *T* = 5 which is consistent with prior work that DKT may overfit in a cold-start setting [2]. Based on the currently pyKT implementation, the fact that DKT stops training at epoch 11-15 indicates overfitting and early stopping after looking at the validation set.

<table><caption>**Table 2.** Performance (AUC) of models for different sequence lengths *T*</caption><thead><tr><th rowspan="2">Data</th><th rowspan="2">Model</th><th rowspan="2">Time Epochs</th><th colspan="4">Sequence length T</th></tr><tr><th>5</th><th>10</th><th>15</th><th>20</th></tr></thead><tbody><tr><td rowspan="7">Assistments 2009</td><td>LR</td><td>0.13s</td><td>1</td><td>0.671</td><td>0.702</td><td>0.666</td><td>0.633</td></tr><tr><td>GBM</td><td>0.26s</td><td>1</td><td>0.701</td><td>0.707</td><td>0.717</td><td>0.676</td></tr><tr><td>TabPFN</td><td>1.53s</td><td>1</td><td>0.726</td><td><strong>0.766</strong></td><td><strong>0.766</strong></td><td>0.719</td></tr><tr><td>TabICL</td><td>2.72s</td><td>1</td><td>0.714</td><td>0.743</td><td>0.724</td><td>0.706</td></tr><tr><td>DKT</td><td>8.8s</td><td>22</td><td>0.748</td><td>0.749</td><td>0.751</td><td><strong>0.744</strong></td></tr><tr><td>AKT</td><td>418s</td><td>115</td><td><strong>0.754</strong></td><td>0.756</td><td>0.751</td><td><strong>0.742</strong></td></tr><tr><td rowspan="7">POJ</td><td>LR</td><td>0.01s</td><td>1</td><td>0.709</td><td>0.745</td><td>0.728</td><td>0.706</td></tr><tr><td>GBM</td><td>0.13s</td><td>1</td><td><strong>0.779</strong></td><td>0.750</td><td>0.747</td><td>0.725</td></tr><tr><td>TabPFN</td><td>0.40s</td><td>1</td><td>0.761</td><td><strong>0.758</strong></td><td><strong>0.756</strong></td><td><strong>0.797</strong></td></tr><tr><td>TabICL</td><td>0.50s</td><td>1</td><td><strong>0.773</strong></td><td>0.728</td><td>0.749</td><td><strong>0.798</strong></td></tr><tr><td>DKT</td><td>0.85s</td><td>12</td><td>0.568</td><td>0.728</td><td>0.689</td><td>0.712</td></tr><tr><td>AKT</td><td>31.2s</td><td>65</td><td>0.690</td><td>0.719</td><td>0.749</td><td>0.760</td></tr><tr><td rowspan="7">Codeforces</td><td>LR</td><td>0.01s</td><td>1</td><td>0.594</td><td>0.626</td><td><strong>0.701</strong></td><td>0.545</td></tr><tr><td>GBM</td><td>0.13s</td><td>1</td><td>0.530</td><td>0.566</td><td>0.639</td><td>0.599</td></tr><tr><td>TabPFN</td><td>0.44s</td><td>1</td><td>0.596</td><td>0.643</td><td>0.670</td><td>0.593</td></tr><tr><td>TabICL</td><td>0.56s</td><td>1</td><td>0.599</td><td><strong>0.650</strong></td><td>0.660</td><td>0.593</td></tr><tr><td>DKT</td><td>0.95s</td><td>13</td><td>0.608</td><td>0.636</td><td>0.619</td><td><strong>0.643</strong></td></tr><tr><td>AKT</td><td>21.1s</td><td>40</td><td><strong>0.633</strong></td><td>0.629</td><td>0.625</td><td><strong>0.646</strong></td></tr></tbody></table>

# 6 Discussion

The proposed tabular foundation models (TFMs), TabICL and TabPFN, showed comparable performance to traditional knowledge tracing models, for up to 273x speedups. This opens the door to transformer-based methods that work on smaller datasets, which is most of the data encountered in classrooms. The fact that TFMs do not need to be retrained when new data shows up enables their use in online settings, such as exams or programming competitions.

We now give some insights about how TFMs work. TFMs learn to align testing sequences and training sequences. The attention mechanism learns the most similar sequences in the training set and combines them to infer the next outcomes. While attention models like AKT consider one student at a time and are trained to generalize to new students, TFMs are trained on tabular
datasets and attempt to generalize across datasets. More precisely, in AKT there
is one attention mechanism along the sequence of time steps, while there are two
for tabular foundation models: across features (time steps), and across rows
(students).

The downsides of relying on a pretrained model is that our results depend
on specific weights. Still, at least the weights are downloaded locally, and can be
frozen, which can lead to more determinism and reproducibility than when we
rely on cloud-provided LLMs. In future work, we intend to pretrain our own TFM
from scratch in order to control the full pipeline and develop a new foundation
model that is more specific to the distribution of knowledge tracing datasets,
but not to a dataset in particular.

# 7 Conclusion

We presented a tabular foundation model (TFM) approach to knowledge tracing that removes the need for offline training and is suitable for live environments such as programming competitions. Our method performs consistently well, even for small datasets, and does not need to be retrained on new datasets. Our experiments indicate that TFM can be an efficient alternative to traditional deep knowledge tracing systems.

The proposed approach establishes a robust baseline capable of replacing
traditional logistic-based knowledge tracing models, sometimes at the cost of
interpretability. Still, because the architecture's primary component is a two-
way attention mechanism operating over rows, the model maintains a degree of
transparency. By analyzing the attention weights across students, we can identify
which student sequences have the most influence on a specific prediction. This
would enable identifying "nearest neighbor" students whose past behaviors most
closely align with the current test student, which could be useful for teachers. In
future work, we expect to conduct this analysis and use these models to optimize
instruction and monitor fairness in decision making.

# References

1. Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas J. Guibas, and Jascha Sohl-Dickstein. Deep knowledge tracing. In *Advances in Neural Information Processing Systems*, volume 28, 2015.

2. Indronil Bhattacharjee and Christabel Wayllace. Cold start problem: An experimental study of knowledge tracing models with new students. In *International Conference on Artificial Intelligence in Education*, pages 425–432. Springer, 2025.

3. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. *Advances in neural information processing systems*, 33:1877–1901, 2020.

4. Jingang Qu, David Holzmüller, Gaël Varoquaux, and Marine Le Morvan. Tabic: A tabular foundation model for in-context learning on large data. In *ICML 2025-Forty-Second International Conference on Machine Learning*, 2025.
5. Jake Robertson, Noah Hollmann, Samuel Müller, Noor Awad, and Frank Hutter. Fairpn: A tabular foundation model for causal fairness. In *Forty-second International Conference on Machine Learning*.

6. A. T. Corbett and J. R. Anderson. Knowledge tracing: Modeling the acquisition of procedural knowledge. *User Modeling and User-Adapted Interaction*, 4(4):253–278, 1995.

7. Aritra Ghosh, Neil Heffernan, and Andrew S. Lan. Context-aware attentive knowledge tracing. In *Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining*, pages 2330–2339, New York, NY, USA, 2020. ACM.

8. Kevin H Wilson, Yan Karklin, Bojian Han, and Chaitanya Ekanadham. Back to the basics: Bayesian extensions of irt outperform neural networks for proficiency estimation. In *International Educational Data Mining Society*. ERIC, 2016.

9. Philip I Pavlik, Luke G Eglington, and Leigh M Harrell-Williams. Logistic knowledge tracing: A constrained framework for learner modeling. *IEEE Transactions on Learning Technologies*, 14(5):624–639, 2021.

10. Seyed Parsa Neshaei, Richard Lee Davis, Adam Hazimeh, Bojan Lazarevski, Pierre Dillenbourg, and Tanja Käser. Towards modeling learner performance with large language models. In *Proceedings of the International Conference on Educational Data Mining*. International Educational Data Mining Society, 2024.

11. Naiming Liu, Zichao Wang, Richard Baraniuk, and Andrew Lan. Open-ended knowledge tracing for computer science education. In *Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing*, 2022.

12. Noah Hollmann, Samuel Müller, Lennart Purucker, Arjun Krishnakumar, Max Körfer, Shi Bin Hoo, Robin Tibor Schirrmeister, and Frank Hutter. Accurate predictions on small data with a tabular foundation model. *Nature*, 637(8045):319–326, 2025.

13. Noah Hollmann, Samuel Müller, Katharina Eggensperger, and Frank Hutter. Tabpn: A transformer that solves small tabular classification problems in a second. In *International Conference on Learning Representations (ICLR)*, 2023.

14. Alexander Pfefferle, Johannes Hog, Lennart Purucker, and Frank Hutter. nano-tabpn: A lightweight and educational reimplementation of tabpn. In *EurIPS 2025 Workshop: AI for Tabular Data*.

15. Mingyu Feng, Neil Heffernan, and Kenneth Koedinger. Addressing the assessment challenge with an online system that tutors as it assesses. *User Modeling and User-Adapted Interaction*, 19(3):243–266, 2009.

16. Zitao Liu, Qiongqiong Liu, Jiahao Chen, Shuyan Huang, Jiliang Tang, and Weiqi Luo. pykt: a python library to benchmark deep learning based knowledge tracing models. In *Advances in Neural Information Processing Systems*, volume 35, pages 18542–18555, 2022.

17. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in python. *the Journal of machine Learning research*, 12:2825–2830, 2011.