https://chen-yingfa.github.io/categories/thoughts/Recent content in Thoughts on Yingfa Chen 陈英发Hugo -- 0.146.6en-usSat, 07 Oct 2023 18:03:10 +0000https://chen-yingfa.github.io/research_posts/2023-interpreting-a-maze-solving-network/Sat, 07 Oct 2023 18:03:10 +0000https://chen-yingfa.github.io/research_posts/2023-interpreting-a-maze-solving-network/<p><a href="https://www.lesswrong.com/s/sCGfFb5DPfjEmtEdn">The blog post</a></p> <p>I can&rsquo;t believe I haven&rsquo;t read this until now. This is mind-provoking, and the result is an important step towards understanding neural networks.</p> <!-- more --> <p>The culmination of this blog post is the exciting work of <a href="https://chen-yingfa.github.io/2023/10/07/2023-actadd/">Activation Addition</a>, which I believe is one important work that inspired the recently <a href="https://arxiv.org/abs/2310.01405">Representation Engineering</a> work.</p>https://chen-yingfa.github.io/research_posts/2023-llm-safety-and-ethics/Tue, 19 Sep 2023 18:13:06 +0000https://chen-yingfa.github.io/research_posts/2023-llm-safety-and-ethics/<p>I will be holding a seminar at ModelBest (面壁智能) in Sep 20, 2023 in Beijing, Haidian, 科技园. The seminar will be in Chinese, and it&rsquo;s called &ldquo;大模型安全与伦理问题&rdquo; (translation: Safety and Ethical Concerns of Large Language Models). Below is a list of references.</p> <!-- more --> <h2 id="introduction">Introduction</h2> <ul> <li>Galactica: A Large Language Model for Science</li> <li><a href="https://openai.com/research/gpt-4">https://openai.com/research/gpt-4</a></li> <li>SafetyBench: Evaluating the Safety of Large Language Models with Multiple Choice Questions</li> <li>Bias and Fairness in Large Language Models: A Survey</li> <li>A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation</li> </ul> <h2 id="evaluation-methods">Evaluation Methods</h2> <ul> <li>A General Language Assistant as a Laboratory for Alignment, Anthropic</li> <li>Safety Assessment of Chinese Large Language Models</li> <li>Semantics derived automatically from language corpora contain human-like biases</li> <li>StereoSet: Measuring stereotypical bias in pretrained language models</li> </ul> <h3 id="instruction-attacks">Instruction Attacks</h3> <ul> <li>Toxicity in CHATGPT: Analyzing Persona-assigned Language Models ⭐️</li> <li>Large Language Models are Zero-Shot Reasoners ⭐️</li> <li>On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning ⭐️</li> <li>Prompting GPT-3 To Be Reliable</li> <li>Universal and Transferable Adversarial Attacks on Aligned Language Models ⭐️</li> <li>Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment ⭐️⭐️</li> </ul> <h3 id="exaggerated-safety">Exaggerated Safety</h3> <ul> <li>XSTEST: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models ⭐️</li> <li>Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions ⭐️</li> </ul> <h2 id="alignment-methods">Alignment Methods</h2> <ul> <li>Aligning language models to follow instructions ⭐️</li> <li>Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback ⭐️</li> <li>SELF-INSTRUCT: Aligning Language Models with Self-Generated Instructions ⭐️⭐️</li> <li>Pretraining Language Models with Human Preferences ⭐️</li> <li>LIMA: Less Is More for Alignment</li> <li><a href="https://openai.com/blog/our-approach-to-alignment-research">https://openai.com/blog/our-approach-to-alignment-research</a> (Aug 2022)</li> <li><a href="https://openai.com/blog/our-approach-to-alignment-research">https://openai.com/blog/our-approach-to-alignment-research</a> (Jul 2023) ⭐️</li> </ul> <p>⭐️: important</p>