){kind=avatar}
Ryo Kamoi
鴨井 遼 (ja)
Ryo Kamoi is a Ph.D. candidate in Computer Science at Penn State University advised by Dr. Rui Zhang. He is also a Student Researcher at Google.
He received his master’s degree in CS from UT Austin where he was advised by Dr. Greg Durrett, and received his bachelor’s degree in Statistics from Keio University where he was advised by Dr. Kei Kobayashi. He previously interned at Microsoft and Amazon.
His recent research work includes:
News
Jul 2026
I was selected as a recipient of the 2026-27 Herbert J. Bailey Graduate Fellowship from the J. Jeffrey and Ann Marie Fox Graduate School at Penn State University!
Jun 2026
I was selected as a recipient of the 2025-26 Vice Provost and Dean of the Graduate School Student Persistence Scholarship from the J. Jeffrey and Ann Marie Fox Graduate School at Penn State University!
Jun 2026
I have started an internship at Google as a Student Researcher! I will be working on improving LLM agents.
Process Reward Models (PRMs) provide step-level verification to LLM reasoning. However, PRM training data in prior work relies on human annotated labels or costly automatic methods, which generate noisy training labels. We propose FoVer, an efficient method to create accurate PRM training data from formal reasoning tasks using formal verification tools like Z3 and Isabelle. Experimental results demonstrate that FoVer improves PRM performance on widely used benchmarks for informal reasoning tasks.
We critically survey broad papers and discuss the conditions required for successful self-correction. Our survey indicates that (1) no prior work demonstrates successful self-correction with feedback from prompted LLMs, except for studies in tasks that are exceptionally suited for self-correction, (2) self-correction works well in tasks that can use reliable external feedback, and (3) large-scale fine-tuning enables self-correction.
ReaLMistake is a benchmark for evaluating error detection methods that detects errors in LLM responses. This benchmark includes errors made by GPT-4 and Llama 2 70B on three tasks (math word problem generation, fine-grained fact verification, and answerability classification). We observe that LLMs still cannot reliably detect mistakes made by LLMs. Strong LLMs like GPT-4 and Claude 3 detect errors made by LLMs at very low recall, and all LLM-based error detectors perform much worse than humans.
Selected publications. For the full list, please see Google Scholar or Semantic Scholar
Efficient PRM Training Data Synthesis via Formal Verification
PDF
Cite
Code
Dataset
Model
Website
(2026). ACL 2026 Findings.
(2026). ACL 2026 Findings.
Evaluating LLM-Simulated Conversations in Modeling Inconsistent and Uncollaborative Behaviors in Human Social Interaction
PDF
Cite
(2026). arXiv preprint arXiv:2603.17094.
(2026). arXiv preprint arXiv:2603.17094.
VisOnlyQA: Large Vision Language Models Still Struggle with Visual Perception of Geometric Information
PDF
Cite
Dataset
Website
Poster
(2025). COLM 2025.
(2025). COLM 2025.
AAAR-1.0: Assessing AI's Potential to Assist Research
PDF
Cite
(2025). ICML 2025. Best Paper Award at the 2nd AI4Research workshop @ AAAI 2025.
(2025). ICML 2025. Best Paper Award at the 2nd AI4Research workshop @ AAAI 2025.
When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs
PDF
Cite
Video
Slides
Paper List
(2024). TACL 2024. Oral at EMNLP 2024.
(2024). TACL 2024. Oral at EMNLP 2024.
Evaluating LLMs at Detecting Errors in LLM Responses
PDF
Cite
Code
Dataset
Poster
(2024). COLM 2024.
(2024). COLM 2024.
WiCE: Real-World Entailment for Claims in Wikipedia
PDF
Cite
Dataset
Slides
(2023). EMNLP 2023. Oral.
(2023). EMNLP 2023. Oral.
Shortcomings of Question Answering Based Factuality Frameworks for Error Localization
PDF
Cite
Dataset
Video
Poster
(2023). EACL 2023.
(2023). EACL 2023.
Why is the Mahalanobis Distance Effective for Anomaly Detection?
PDF
Cite
(2020). arXiv preprint arXiv:2003.00402.
(2020). arXiv preprint arXiv:2003.00402.
Education
Penn State University
— Ph.D. Student in Computer Science
PhD Advisor: Rui Zhang
University of Texas at Austin
— M.S. in Computer Science
Advisor: Greg Durrett, Mentor: Tanya Goyal
Keio University
— B.E. in Statistics
Advisor: Kei Kobayashi
Work Experience
Google, Cloud AI Research
— Student Researcher
Microsoft, Office of Applied Research
— Research Internship
Amazon, Alexa Team
— Applied Scientist Internship
Services
MASC-SLL 2025
— Organizer
NLP Colloquium JP (NLPコロキウム)
— Staff
Awards
2026-27 Herbert J. Bailey Graduate Fellowship
From the J. Jeffrey and Ann Marie Fox Graduate School at Penn State University
2025-26 Vice Provost and Dean of the Graduate School Student Persistence Scholarship
From the J. Jeffrey and Ann Marie Fox Graduate School at Penn State University
Scholarship for alumni of Keio University to pursue degrees at overseas graduate schools
Graduation with highest honors - First place in the Department of Mathematics at Keio University
Media Mentions
Interview about our survey paper on LLM self-correction.
How to Pronounce My Name
- Preferred English Pronunciation:
- Native Pronunciation in Japanese: