I'm a PhD candidate at KAIST Graduate School of AI (expected Feb 2027), advised by Professor Edward Choi.

My research is AI safety evaluation: identifying what must be verified about a model before it enters real-world service, turning those requirements into benchmarks, and measuring state-of-the-art models against them. Across three released benchmarks — Trans-EnV (NeurIPS 2025), KorNAT (ACL 2024), and VisAlign (NeurIPS 2023) — most models I evaluated fell short, and the majority failed to reach the human reference score on value alignment.

I build these benchmarks end to end — from specification and expert-defined label taxonomies through large-scale human annotation (surveys involving over 6,000 participants) to evaluation and scoring — and operate them as public infrastructure: leaderboards accepting external submissions and interactive demos. I am now extending safety evaluation beyond static text benchmarks to agentic and multimodal settings.

🔑 Keywords: AI safety evaluation, agent safety, benchmark construction, cultural alignment

Feel free to contact me!
✉️ jiyounglee0523 at kaist dot ac dot kr

[ Google Scholar] [ GitHub] [🤗 Hugging Face] [📄 CV]

 

🎓 Education

• Sep 2022 – Feb 2027 (expected) — Ph.D. in Artificial Intelligence, KAIST (advisor: Prof. Edward Choi)
• Sep 2020 – Aug 2022 — M.Sc. in Artificial Intelligence, KAIST (advisor: Prof. Edward Choi)
• Mar 2016 – Aug 2020 — B.Sc. in Statistics, Sookmyung Women’s University — GPA 4.08/4.3, ranked 1st in department

 

💼 Industry Experience

• May – Aug 2026 — AI Research Intern at LLM Evaluation Team, Upstage (advisor: Seung (Tony) Shin)
• Jan – Apr 2026 — AI Research Intern at Social Computing Group, Microsoft Research Asia (advisor: Fangzhao Wu)
• Sep – Dec 2025 — AI Engineer Intern at EXAONE Team, LG AI Research (advisor: HeuiYeen Yeen)
• Feb – Aug 2022 — Machine Translation Research Intern at Papago Team, NAVER Corporation (advisor: Cheonbok Park)

 

📚 Publications

2025

• Trans-EnV: A Framework for Evaluating the Linguistic Robustness of LLMs Against English Varieties
  Jiyoung Lee∗, Seungho Kim∗, Jieun Han, Jun-Min Lee, Kitaek Kim, Alice Oh, Edward Choi (∗ denotes equal contribution)
  In Proc. of Neural Information Processing Systems (NeurIPS) 2025 Datasets and Benchmarks
  [🤗 Dataset] [💻 Code] [🚀 Demo]

• Single Ground Truth Is Not Enough: Adding Flexibility to Aspect-Based Sentiment Analysis Evaluation
  Soyoung Yang, Hojun Cho, Jiyoung Lee, Sohee Yoon, Edward Choi, Jaegul Choo, Won Ik Cho
  In Proc. of Conference on Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL) 2025
  [💻 Code]

2024

• KorNAT: LLM Alignment Benchmark for Korean Social Values and Common Knowledge
  Jiyoung Lee, Minwoo Kim, Seungho Kim, Junghwan Kim, Seunghyun Won, Hwaran Lee, Edward Choi
  In Findings of Association for Computational Linguistics (ACL) 2024 — ✅ TTA government-certified, public leaderboard with external submissions
  [🤗 Dataset] [💻 Code] [🏆 Leaderboard] [🪧 Poster] [📊 Slide]

2023

• VisAlign: Dataset for Measuring the Degree of Alignment between AI and Humans in Visual Perception
  Jiyoung Lee, Seungho Kim, Seunghyun Won, Joonseok Lee, Marzyeh Ghassemi, James Thorne, Jaeseok Choi, O-Kil Kwon, Edward Choi
  In Proc. of Neural Information Processing Systems (NeurIPS) 2023 Datasets and Benchmarks
  [🤗 Dataset] [💻 Code] [🏆 Leaderboard] [🪧 Poster] [📊 Slide]

• Exploration into Translation-Equivariant Image Quantization
  Woncheol Shin, Gyubok Lee, Jiyoung Lee, Eunyi Lyou, Joonseok Lee, Edward Choi
  In Proc. of International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 2023 🏅 (Oral Presentation, Top 3% Paper Recognition)
  [💻 Code]

2022

• Specializing Multi-domain NMT via Penalizing Low Mutual Information
  Jiyoung Lee, Hantae Kim, Hyunchang Cho, Edward Choi, Cheonbok Park
  In Proc. of Empirical Methods in Natural Language Processing (EMNLP) 2022
  [🪧 Poster] [📊 Slide]

• Unifying Heterogeneous Electronic Health Records Systems via Text-Based Code Embedding
  Kyunghoon Hur∗, Jiyoung Lee∗, Jungwoo Oh, Wesley Price, Young-Hak Kim, Edward Choi (∗ denotes equal contribution)
  In Proc. of Conference on Health, Inference, and Learning (CHIL) 2022
  [💻 Code]

• Automatic Detection of Noisy Electrocardiogram Signals without Explicit Noise Labels
  Radhika Dua, Jiyoung Lee, Joon-myung Kwon, Edward Choi
  In International Workshop on Pattern Recognition in Healthcare Analytics (PRHA) 2022

2021

• Conditional Generation of Periodic Signals with Fourier-Based Decoder
  Jiyoung Lee, Wonjae Kim, Daehoon Gwak, Edward Choi
  In Deep Generative Models and Downstream Applications Workshop at NeurIPS 2021
  [💻 Code]

2019

• Breast Cancer Subtype Classification utilizing Multi-Omics Data Integration based on Neural Network
  Joungmin Choi, Jiyoung Lee, Jieun Kim, Jihyun Kim, Heejoon Chae
  In Journal of Korean Institute of Information Scientists and Engineers (JOK), Vol. 46 No. 02, pp. 0476–0478, Dec. 2019

📝 Technical Reports

• EXAONE 4.5 Technical Report
  LG AI Research, 2026 — contributor (safety data)
  [🤗 Models]

• K-EXAONE Technical Report
  LG AI Research, 2026 — contributor (safety data)
  [🤗 Models]

 

🤝 Academic Service

• Reviewer — NeurIPS 2025, ACL 2025, ACL Rolling Review (May), ACCV 2022
• Advisory Committee Member — SelectStar AI NLP Team, LLM Dataset Construction (Sep 2023 – Feb 2024): advised on benchmark construction methodology and quality protocols

 

🧑‍🏫 Teaching & Talks

• Invited Lectures — KoSAIM Summer School: Chest X-ray Classification with CNNs (Aug 2025); NumPy & PyTorch Introduction (Aug 2020)

• Teaching Assistant, KAIST — AI504: Programming for AI (Fall 2020–2024)

  AI612: Machine Learning for Healthcare (Spring 2021, 2023)

 

🏆 Awards & Scholarships

• NeurIPS 2023 Travel Award
• Google Conference Scholarship 2022
• National Science & Technology Scholarship (2018–2020)