Kaiwen Zhou

Research Scientist @ Scale AI · AI Safety & Alignment

Photo of Kaiwen Zhou

kaiwenzhou111@gmail.com

California, USA

I am a Research Scientist at Scale AI, working on frontier AI safety and alignment. Previously, I was a MATS & Anthropic Safety Fellow. I received my Ph.D. from the University of California, Santa Cruz, advised by Prof. Xin (Eric) Wang.

Highlights of past research I led:

  • Reasoning model safety — R1 Safety Eval (IJCNLP-AACL 2025) and SafeKey (EMNLP 2025), two of the first works on the safety of large reasoning models.
  • Embodied AI safety — Multimodal Situational Safety (ICLR 2025), one of the first works on embodied AI safety.
  • LLMs for embodied AI — ESC (ICML 2023), one of the first works on LLMs for embodied agents.

news

Sep 2026 Invited talk at CMU!
Jun 2026 I have joined Scale AI as a Research Scientist!
Jan 2026 Two papers accepted by ICLR 2026!
Jan 2026 Our SIRAJ paper is accepted by EACL 2026 Findings!
Dec 2025 Selected for MATS and the Anthropic Fellows Program for AI Safety Research!
Aug 2025 Our SafeKey paper is accepted by EMNLP 2025!
Jun 2025 Invited talk at Microsoft on safety reasoning!
Jun 2025 Honored to receive the UCSC Dissertation-Year Fellowship to support my research on Trustworthy AI!
Mar 2025 I will join Microsoft as a research intern this summer!
Jan 2025 Our MSSBench paper is accepted by ICLR 2025!
May 2024 Two papers are accepted by ACL 2024!
Mar 2024 One paper is accepted by NAACL 2024!
Jun 2023 Our SlugJARVIS team won third place in the first-ever Amazon Alexa SimBot Challenge!
Apr 2023 Our ESC paper is accepted by ICML 2023!

selected publications

* equal contribution. Full list on the publications page or Google Scholar.

  1. arXiv
    RSI Control: Can AI Models Perform Covert Sabotage in the Self-Improvement Process?
    Kaiwen Zhou, Udari Madhushani Sehwag
    arXiv 2026
    Paper
  2. ICML-W
    Probing the Misaligned Thinking Process of Language Models
    Kaiwen Zhou, Constantin Venhoff, Jonathan Michala, Xin Eric Wang, William Saunders
    Mechanistic Interpretability Workshop @ ICML 2026
    PaperWebsiteCodeData
  3. arXiv
    SafePro: Evaluating the Safety of Professional-Level AI Agents
    Kaiwen Zhou, Shreedhar Jangam*, Ashwin Nagarajan*, Tejas Polu*, Suhas Oruganti, Chengzhi Liu, Ching-Chen Kuo, Yuting Zheng, Sravana Narayanaraju, Xin Eric Wang
    arXiv 2026
    PaperWebsiteCodeData
  4. EACL
    SIRAJ: Diverse and Efficient Red-Teaming for LLM Agents via Distilled Structured Reasoning
    Kaiwen Zhou, Ahmed Elgohary, A S M Iftekhar, Amin Saied
    Findings of EACL 2026
    Paper
  5. EMNLP
    SafeKey: Amplifying Aha-Moment Insights for Safety Reasoning
    Kaiwen Zhou, Xuandong Zhao, Gaowen Liu, Jayanth Srinivasa, Aosong Feng, Dawn Song, Xin Eric Wang
    EMNLP 2025
    PaperWebsiteCodeModels
  6. AACL
    The Hidden Risks of Large Reasoning Models: A Safety Assessment of R1
    Kaiwen Zhou, Chengzhi Liu, Xuandong Zhao, Shreedhar Jangam, Jayanth Srinivasa, Gaowen Liu, Dawn Song, Xin Eric Wang
    IJCNLP-AACL 2025
    PaperWebsite
  7. ICLR
    Multimodal Situational Safety
    Kaiwen Zhou*, Chengzhi Liu*, Xuandong Zhao, Anderson Compalas, Dawn Song, Xin Eric Wang
    ICLR 2025
    NeurIPS 2024 Workshop on RBFM — Oral
    PaperWebsiteCodeData
  8. ICML
    ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigation
    Kaiwen Zhou, Kaizhi Zheng, Connor Pryor, Yilin Shen, Hongxia Jin, Lise Getoor, Xin Eric Wang
    ICML 2023
    PaperWebsite
  9. ECCV
    FedVLN: Privacy-preserving Federated Vision-and-Language Navigation
    Kaiwen Zhou, Xin Eric Wang
    ECCV 2022
    PaperCode

service