In DeltaProduct (Siems et al., 2025), they propose to improve DeltaNet (Yang et al., 2025) by updating the online memory with $n_h$ KVs for each token, which can be seen as performing multiple steps of gradient descent per token. I will explain how this method is almost the same as multi-KV DeltaNet and reveal a potential flaw in the design of DeltaProduct. Introduction DeltaNet We use row-vector notation. ...
This blog post is part 1 of a series that describes my attempt in implementing the Test-Time Training (TTT) model proposed by Sun et al. (2024), and Titans, proposed by Behrouz et al., (2024). At the time of writing, these two are two strong recurrent language models, but they have not yet open-sourced their implementation (TTT has only open-sourced the Jax implementation). Introduction to Test-Time Training Briefly explained, Test-Time Training (TTT) is an RNN model whose hidden state is replaced with an online learner, whose parameters are updated updated through gradient descent during inference. The goal is that this online learner compress contextual information into its parameters. A TTT operator can be expressed as: ...
GitHub | Paper (upcoming) TL;DR: A reader is augmented with a growing notebook that caches all edits in natural texts, and the reader retrieves relevant edits and make inference based on them. This achieves SOTA in model editing in QA and fact-checking. NB: The COLING template in 2024 was very ugly. Introduction This work introduces a model editing method that addresses two issues with existing model editors: ...
Code | Paper The first benchmark for evaluating the effectiveness of LLMs in handling more than 100k tokens! In the paper, we name it $\infty$-Bench, but I will sometimes use “InfiniteBench” in this blog post for better readability. Finally got some time to write this blog, been so busy lately! I have been in a fairly long duration of research hiatus, meanwhile the field of NLP has been revolutionized by an overwhelming number of new LLMs. Finally, I was able to arrive at some productive and meaningful work in this new era of research, as a second author. In this blog post, I will introduce this work that I have been working on recently. ...
The blog post I can’t believe I haven’t read this until now. This is mind-provoking, and the result is an important step towards understanding neural networks. The culmination of this blog post is the exciting work of Activation Addition, which I believe is one important work that inspired the recently Representation Engineering work.
Paper TLDR: Propose ActAdd, a method for controlling model behavior during inference by modifying activations with a bias term that is learned from a pair of prompt. Summary: Propose ActAdd, a method for controlling model behavior by modifying activations at inference time. Steering vectors are computed by taking the activation differences that result from pairs of prompts. The vectors are added as bias during inference. ActAdd provides control over high-level properties of the output, and preserves off-target model performance, and requires little computational and implementational costs. The recently popular representation engineering paper (RepE) seems to be largely inspired by this work. ...
I will be holding a seminar at ModelBest (面壁智能) in Sep 20, 2023 in Beijing, Haidian, 科技园. The seminar will be in Chinese, and it’s called “大模型安全与伦理问题” (translation: Safety and Ethical Concerns of Large Language Models). Below is a list of references. Introduction Galactica: A Large Language Model for Science https://openai.com/research/gpt-4 SafetyBench: Evaluating the Safety of Large Language Models with Multiple Choice Questions Bias and Fairness in Large Language Models: A Survey A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation Evaluation Methods A General Language Assistant as a Laboratory for Alignment, Anthropic Safety Assessment of Chinese Large Language Models Semantics derived automatically from language corpora contain human-like biases StereoSet: Measuring stereotypical bias in pretrained language models Instruction Attacks Toxicity in CHATGPT: Analyzing Persona-assigned Language Models ⭐️ Large Language Models are Zero-Shot Reasoners ⭐️ On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning ⭐️ Prompting GPT-3 To Be Reliable Universal and Transferable Adversarial Attacks on Aligned Language Models ⭐️ Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment ⭐️⭐️ Exaggerated Safety XSTEST: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models ⭐️ Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions ⭐️ Alignment Methods Aligning language models to follow instructions ⭐️ Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback ⭐️ SELF-INSTRUCT: Aligning Language Models with Self-Generated Instructions ⭐️⭐️ Pretraining Language Models with Human Preferences ⭐️ LIMA: Less Is More for Alignment https://openai.com/blog/our-approach-to-alignment-research (Aug 2022) https://openai.com/blog/our-approach-to-alignment-research (Jul 2023) ⭐️ ⭐️: important ...
Code | Paper | Paper (preprints.org) | 知乎 I did this work with my girlfriend, whose research direction is computational fluid dynamics (CFD). We observed that there are numerous research works in applying deep learning (DL) to solve CFD problems. E.g., Pangu-Weather have shown that DL methods can not only be more accurate than the best numerical methods, but can also be multiple magnitudes faster. However, there is no standard benchmark for evaluating the performance of different DL methods. Therefore, we constructed CFDBench. ...