Ryo Kamoi

Ryo Kamoi

鴨井 遼 (ja)

Ph.D. Student

Penn State University

He is a CS Ph.D. student at Penn State University advised by Dr. Rui Zhang. He is broadly interested in Natural Language Processing, with a specific focus on building trustworthy NLP systems. He received his master’s degree in CS from UT Austin where he was advised by Dr. Greg Durrett, and received his bachelor’s degree in Statistics from Keio University where he was advised by Dr. Kei Kobayashi.

  • Trustworthy NLP Systems
  • Detecting and Mitigating Errors in Responses from LLMs
  • Evaluation of LLMs
  • Fact Checking
  • Text Summarization

  • Ph.D. in Computer Science, 2023 -

    Penn State University

  • M.S. in Computer Science, 2020 - 2022

    University of Texas at Austin

  • B.E. in Statistics, 2016 - 2020

    Keio University

News


Evaluating LLMs at Detecting Errors in LLM Responses PDF Cite Code Dataset Poster
Ryo Kamoi, Sarkar Snigdha Sarathi Das, Renze Lou, Jihyun Janice Ahn, Yilun Zhao, Xiaoxin Lu, Nan Zhang, Yusen Zhang, Ranran Haoran Zhang, Sujeeth Reddy Vummanthala, Salika Dave, Shaobo Qin, Arman Cohan, Wenpeng Yin, Rui Zhang (2024). COLM 2024.
Direct-Inverse Prompting: Analyzing LLMs' Discriminative Capacity in Self-Improving Generation PDF Cite
Jihyun Janice Ahn, Ryo Kamoi, Lu Cheng, Rui Zhang, Wenpeng Yin (2024). arXiv preprint arXiv:2407.11017.
When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs PDF Cite
Ryo Kamoi, Yusen Zhang, Nan Zhang, Jiawei Han, Rui Zhang (2024). TACL (to appear).
DocMath-Eval: Evaluating Numerical Reasoning Capabilities of LLMs in Understanding Long Documents with Tabular Data PDF Cite
Yilun Zhao, Yitao Long, Hongjun Liu, Linyong Nan, Lyuhao Chen, Ryo Kamoi, Yixin Liu, Xiangru Tang, Rui Zhang, Arman Cohan (2024). ACL 2024.
Fair Abstractive Summarization of Diverse Perspectives PDF Cite
Yusen Zhang, Nan Zhang, Yixin Liu, Alexander Fabbri, Junru Liu, Ryo Kamoi, Xiaoxin Lu, Caiming Xiong, Jieyu Zhao, Dragomir Radev, Kathleen McKeown, Rui Zhang (2024). NAACL 2024.
WiCE: Real-World Entailment for Claims in Wikipedia PDF Cite Dataset Slides
Ryo Kamoi, Tanya Goyal, Juan Diego Rodriguez, Greg Durrett (2023). EMNLP 2023.
Shortcomings of Question Answering Based Factuality Frameworks for Error Localization PDF Cite Dataset Poster
Ryo Kamoi, Tanya Goyal, Greg Durrett (2023). EACL 2023.
Efficient Unknown Object Detection with Discrepancy Networks for Semantic Segmentation PDF Cite
Ryo Kamoi, Takumi Iida, Kaname Tomite (2021). NeurIPS Workshop on Machine Learning for Autonomous Driving.
Alternative methods for fast and stable GAN Cite
Katsuhiro Endo, Ryo Kamoi, Kenji Yasuoka (2020). MIRU.
Out-of-Distribution Detection with Likelihoods Assigned by Deep Generative Models Using Multimodal Prior Distributions PDF Cite
Ryo Kamoi, Kei Kobayashi (2020). The AAAI’s Workshop on Artificial Intelligence Safety.
Why is the Mahalanobis Distance Effective for Anomaly Detection? PDF Cite
Ryo Kamoi, Kei Kobayashi (2020). arXiv preprint arXiv:2003.00402.
Likelihood Assignment for Out-of-Distribution Inputs in Deep Generative Models is Sensitive to Prior Distribution Choice PDF Cite
Ryo Kamoi, Kei Kobayashi (2019). arXiv preprint arXiv:1911.06515.

Work Experience

Amazon - Applied Scientist Internship Jul 2021 – Dec 2021 Cambridge, U.K.
Research on the quality evaluation of Alexa

Invited Talks

Matsuo-Iwasawa Lab at the University of Tokyo (LLM講座, ja)
Lecture about self-correction of LLMs based on our survey paper “When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs”
Oct 17, 2024 The University of Tokyo (Online)
NLP Colloquium (NLPコロキウム, ja)
Talk about our paper “WiCE Real-World Entailment for Claims in Wikipedia”
Nov 29, 2023 Online
Nagoya NLP Seminar at Nagoya University (名古屋NLPセミナー, ja)
Talk about our papers “Shortcomings of Question Answering Based Factuality Frameworks for Error Localization” and “WiCE Real-World Entailment for Claims in Wikipedia”
Jul 3, 2023 Nagoya University

Services

NLP Colloquium (NLPコロキウム) - Staff Mar 2024 – Present

Awards

Keio University Global Fellowship 2020Aug 2020
Scholarship for alumni of Keio University to pursue degrees at overseas graduate schools
Keio Engineering Foundation AwardMar 2020
Graduation with highest honors - First place in the Department of Mathematics at Keio University