Tabular Foundation Models
A Hands-On Guide to TabPFN, TabICL, and the Tabular Revolution
Tabular foundation models like TabPFN and TabICL are weird. On the surface, their usage is the same as any machine learning algorithm: fit, then predict. Looking deeper, they work very differently and may change tabular machine learning.
Where To Buy
The book is still in progress, but you can read the in-progress version here:
Summary
Tabular foundation models are pretrained on millions of mostly simulated datasets. To make predictions, you don’t train the model on your specific task, but instead provide the training data during inference time, because these models do in-context learning. Meaning no parameter updates in the model, and no hyperparameter tuning. Increasingly, tabular foundation models beat the state-of-the-art.
I found models such as TabPFN and TabICL puzzling at first. Now, I’m convinced that foundation models change tabular machine learning. This book enables you to:
- Intuitively understand tabular foundation models.
- Apply models like TabPFN and TabICL to your data.
- Understand the difference between traditional machine learning and foundation models.
Who This Book Is For
I strongly feel everyone in tabular machine learning should learn about tabular foundation models. If you know why we split data into train and test, and have at least once in your life written the line “.predict()”, this book is for you. I’m doing my best to make this the absolute best resource on tabular foundation models out there.