Tag: language models

Abstract : The main idea in test time training (TTT) ( 1) is that a model with fixed parameters produces the supervision for another network that is updated during test time (or inference time). This article first reviews the TTT paper. Then, we discuss the problem with TTT and how LaCT addresses them, resulting in ...

This blog post is part 1 of a series that describes my attempt in implementing the Test Time Training (TTT) model proposed by 1, and Titans, proposed by 1. At the time of writing, these two are two strong recurrent language models, but they have not yet open sourced their implementation (TTT has only open sourced th...