Code | Paper

The first benchmark for evaluating the effectiveness of LLMs in handling more than 100k tokens!

In the paper, we name it $\infty$-Bench, but I will sometimes use "InfiniteBench" in this blog post for better readability.

Finally got some time to write this blog, been so busy lately! I have been in a fairly long duration of research hiatus, meanwhile the field of NLP has been revolutionized by an overwhelming number of new LLMs. Finally, I was able to arrive at some productive and meaningful work in this new era of research, as a second author. In this blog post, I will introduce this work that I have been working on recently.

The blog post

I can't believe I haven't read this until now. This is mind-provoking, and the result is an important step towards understanding neural networks.

Paper

TLDR: Propose ActAdd, a method for controlling model behavior during inference by modifying activations with a bias term that is learned from a pair of prompt.

Summary:

• Propose ActAdd, a method for controlling model behavior by modifying activations at inference time.
• Steering vectors are computed by taking the activation differences that result from pairs of prompts. The vectors are added as bias during inference.
• ActAdd provides control over high-level properties of the output, and preserves off-target model performance, and requires little computational and implementational costs.

今年国庆 🇨🇳 和中秋 🥮 一起放假，我跟 00 一起回来应城参加她堂姐和初中同学的婚礼^[1]， 住在她家里十个夜晚^[2]。第二次见家长，也算是挺顺利，但是每天都会见到陌生人，有点累，庆幸的是，感觉到 00 能接受跟我家人生活在一起。一号到三号我们去武汉玩了三天，超级开心，跟她在一起连逛商场都是开心的！

小县城的氛围

应城跟我想象中的小县城很像，也是很多远房亲戚，习俗也让人很烦。敬酒、随地扔垃圾、室内抽烟、八卦人家的私事、说话粗鄙、脏、说了不要还非要给人家……而且确实能明显感觉到，这里的人的素质的平均水平挺低的，尤其是上一辈。真的很讨厌吃席，00 也是，这些习俗的麻烦程度让 00 都不想结婚了……

I will be holding a seminar at ModelBest (面壁智能) in Sep 20, 2023 in Beijing, Haidian, 科技园. The seminar will be in Chinese, and it's called "大模型安全与伦理问题" (translation: Safety and Ethical Concerns of Large Language Models). Below is a list of references.

之前有过个人主页，但是一直没有弄好，更没有更新。最近我将自己的 GitHub 的用户名改了，导致之前的 GitHub Pages 失效了，就趁机重新搭建个人主页。

兜兜转转，还是决定使用 Hexo。以前用过 Jekyll，觉得还行，但是真的不想用 Ruby，Hugo 又太麻烦。

Code | Paper (on hold by ArXiv) | Paper (preprints.org) | 知乎

I did this work with my girlfriend, whose research direction is computational fluid dynamics (CFD). We observed that there are numerous research works in applying deep learning (DL) to solve CFD problems. E.g., Pangu-Weather have shown that DL methods can not only be more accurate than the best numerical methods, but can also be multiple magnitudes faster.

今天睡到九点才醒来，还是被00打电话叫醒的。去过了个早，然后去上《深度学习》。

今天00的师姐答辩，下午三点去了，当时我在睡午觉。感觉之后她有点emo，但是她不承认，不知道为什么。然后聊了很多关于未来，结婚、生孩子、找工作等事情。感觉也没有很大的问题，但是00总是把东西看得很灰暗，很焦虑。