Ruiqi Zhong

I am currently a PhD student in the UC Berkeley EECS department, and a part-time research scientist at Anthropic. I am co-advised by Prof. Jacob Steinhardt and Prof. Dan Klein.

Before coming to Berkeley, I finished my undergrad at Columbia University, where I worked with Prof. Kathleen McKeown.

Email  /  Google Scholar  /  Twitter  /  Github

profile photo
Research Overview

I work on scalable oversight -- supervising AI systems to accomplish tasks where humans alone struggle to determine the ground truth. Doing so requires human-AI collaborations, a better epistemic foundation, and new algorithmic tools. I currently work on concrete related problems in Natural Language Processing, Machine Learning, and Programming Language. See presentation slides here and my talk here to get a sense of my research interests.

Blogs Representative Work

Non-Programmers Can Label Programs Indirectly via Active Examples: A Case Study with Text-to-SQL
Ruiqi Zhong*, Charlie Snell*, Dan Klein, Jason Eisner
EMNLP 2023
[paper] [code] [demo]
Goal Driven Discovery of Distributional Differences via Language Descriptions
Ruiqi Zhong, Peter Zhang, Steve Li, Jinwoo Ahn, Dan Klein, Jacob Steinhardt
NeurIPS 2023
[paper] [code]
Others
Describing Differences in Image Sets with Natural Language
Lisa Dunlap*, Yuhui Zhang*, Xiaohan Wang, Ruiqi Zhong, Trevor Darrell*, Jacob Steinhardt*, Joseph E Gonzalez*, Serena Yeung-Levy*
CVPR 2024
[paper][website]
Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations
Yanda Chen, Ruiqi Zhong, Narutatsu Ri, Chen Zhao, He He, Jacob Steinhardt, Zhou Yu, Kathleen McKeown
arXiv 2023
[paper]
Goal-Driven Explainable Clustering via Language Descriptions
Zihan Wang, Jingbo Shang, Ruiqi Zhong
EMNLP 2023
[paper][code]
DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation
Yuhang Lai*, Chengxi Li*, Yiming Wang*, Tianyi Zhang*, Ruiqi Zhong*, Luke Zettlemoyer, Scott Wen-tau Yih, Daniel Fried, Sida Wang, Tao Yu
ICML 2023
[paper][data]
InCoder: A Generative Model for Code Infilling and Synthesis
Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Wen-tau Yih, Luke Zettlemoyer, Mike Lewis
ICLR 2023
[paper]
Learning by Distilling Context
Charlie Snell, Dan Klein, Ruiqi Zhong
arXiv 2022
[paper]
Describing Differences between Text Distributions with Natural Language
Ruiqi Zhong, Charlie Snell, Dan Klein, Jacob Steinhardt
ICML 2022
[paper] [code]
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models
Tianbao Xie, Chen Henry Wu, Peng Shi, Ruiqi Zhong, Torsten Scholak, Michihiro Yasunaga, Chien-Sheng Wu, Ming Zhong, Pengcheng Yin, Sida I. Wang, Victor Zhong, Bailin Wang, Chengzu Li, Connor Boyle, Ansong Ni, Ziyu Yao, Dragomir Radev, Caiming Xiong, Lingpeng Kong, Rui Zhang, Noah A. Smith, Luke Zettlemoyer, Tao Yu
EMNLP 2022
[paper]
Meta-learning via Language Model In-context Tuning
Yanda Chen, Ruiqi Zhong, Sheng Zha, George Karypis, He He
ACL 2022
[paper]
The Effect of Model Size on Worst-Group Generalization
Alan Pham*, Eunice Chan*, Vikranth Srivatsa*, Dhruba Ghosh*, Yaoqing Yang, Yaodong Yu, Ruiqi Zhong, Joseph E. Gonzalez, Jacob Steinhardt
NeurIPS 2021 Workshop on Distribution Shifts
[paper]
Approximating How Single Head Attention Learns
Charlie Snell*, Ruiqi Zhong*, Dan Klein, Jacob Steinhardt
arXiv 2021
[paper] [slides] [code] [blog]
Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections
Ruiqi Zhong, Kristy Lee*, Zheng Zhang*, Dan Klein
EMNLP 2021, Findings
[paper][slides][code]
Are Larger Pretrained Language Models Uniformly Better? Comparing Performance at the Instance Level
Ruiqi Zhong, Dhruba Ghosh, Dan Klein, Jacob Steinhardt
ACL 2021, Findings
[paper] [slides] [code]
Semantic Evaluation for Text-to-SQL with Distilled Test Suites
Ruiqi Zhong, Tao Yu, Dan Klein
EMNLP 2020
[paper] [slides] [code]
Semantic Scaffolds for Pseudocode-to-Code Generation
Ruiqi Zhong, Mitchell Stern, Dan Klein
ACL 2020
[paper] [slides] [code] [video]
Detecting and Reducing Bias in a High Stakes Domain
Ruiqi Zhong, Yanda Chen, Desmond Patton, Charlotte Selous, Kathy McKeown
EMNLP 2019
[paper] [poster] [code]
Fine-grained Sentiment Analysis with Faithful Attention
Ruiqi Zhong, Steven Shao, Kathy McKeown
arXiv 2019
[paper]`
Detecting Gang-involved Escalation on Social Media Using Context
Serina Chang, Ruiqi Zhong, Ethan Adams, Fei-Tzin Lee, Siddharth Varia, Desmond Patton, William Frey, Chris Kedzie, Kathy McKeown
EMNLP 2018
[paper] [code]
Subspace Embedding and Linear Regression with Orlicz Norm
Alexandr Andoni, Chengyu Lin, Ying Sheng, Peilin Zhong, Ruiqi Zhong
ICML 2018
[paper] [video] [slides]
GAIA - A Multi-media Multi-lingual Knowledge Extraction and Hypothesis Generation System
Tongtao Zhang, Ananya Subburathinam, Ge Shi, Lifu Huang, Di Lu, Xiaoman Pan, Manling Li, Boliang Zhang, Qingyun Wang, Spencer Whitehead, Heng Ji, Alireza Zareian, Hassan Akbari, Brian Chen, Ruiqi Zhong, Steven Shao, Emily Allaway, Shih-Fu Chang, Kathleen R. McKeown, Dongyu Li, Xin Huang, Kexuan Sun, Xujun Peng, Ryan Gabbard, Marjorie Freedman, Mayank Kejriwal, Ram Nevatia, Pedro A. Szekely, T. K. Satish Kumar, Ali Sadeghian, Giacomo Bergami, Sourav Dutta, Miguel E. Rodríguez, Daisy Zhe Wang
TAC 2018
[paper]
External Presentations
  • [05/18/2023] 由安远和智源社区共同举办，on Scalable Oversight (中文的幻灯片)
  • [04/20/2023] At JHU CS 601, on Scalable Oversight
  • [04/05/2023] At Anthropic, on Scalable Oversight
  • [03/30/2023] At USC NLP Seminar, on Scalable Oversight
  • [10/27/2022] At NYU, on Scalable Oversight
  • [10/25/2022] At Cornell, Sasha Rush's group meeting, on Scalable Oversight
  • [10/24/2022] At Columbia NLP Seminar, on Scalable Oversight
  • [09/07/2022] At Redwood Research, on Scalable Oversight
  • [08/23/2022] At University of Toronto, Roger Grosse's group meeting, on Scalable Oversight
  • [07/28/2022] At Codex Community reading group, on Active Programming by Example with a Natural Language Prior
  • [06/23/2022] At Microsoft Semantic Machines, on Scalable Oversight
Miscellaneous
  • I represented Columbia University in ACM-ICPC and Putnam Math Competition during my Sophormore year (though it seems I was the bottleneck of our teams).
  • I sleep at 11 p.m. and do not respond to later messages. Sometimes, however, I am actually awake; but I pretend not to see them anyways.
  • My favorite animation character and role model is Wenli Yang in Legend of Galactical Heroes.
Awards
  • Berkeley Graduate Student Fellowship
  • Theodore R. Bashkow Award (research), Academic Excellence Award (GPA)
  • CRA Outstanding Undergraduate Research Award Honorable Mention * 2 (2018, 2019)
  • William Lowell Putnam Math Competition top 5% * 3 (2015, 2016, 2018)
Undergrad Advising
Yes, I have a serious commitment towards cultivating future researchers and practitioners who understand AI, expand scientific knowledge, and make the world a better place. I sometimes spend ~3 hours per week with each undergrad whom I closely mentor. Feel free to reach out if you satisfy any of the following condition, and I would love to chat about opportunities.
  • Is passionate about NLP. This is usually evidenced by 1) taking an NLP class, 2) playing around with NLP models on your own, or 3)(self-)learning a substantial fraction of material from this document.
  • Has a strong intellectual interest in social sciences or philosophy.
  • Is from a developing country under-represented in the research community.
  • Excels at competitive programming or math olympiads.
Undergrads that I have mentored:
  • Xinyi Han (now Ph.D. at MIT)
  • Yanda Chen (now Ph.D. at Columbia)
  • Charlie Snell (now Ph.D. at UC Berkeley)
  • Dhruba Ghosh (now Ph.D. at University of Washington)
  • Sicheng Tang
  • Kristy Lee (now 5th year Master at UC Berkeley)
  • Zheng Zhang (now 5th year Master at UC Berkeley)
  • Harry Zhao (now 5th year Master at UC Berkeley)
  • Pulkit Bhasin
  • Dong Yang
  • Peter Zhang
  • Oscar Xu (incoming Ph.D. at University of Pennsylvania)
  • Steve Li
  • JinWoo Ahn
  • Vedant Kumud
  • Heng Wang
  • Anu Soneye

Website design from Jon Barron