I'm a 4th-year PhD student at KAIST Graduate School of AI, advised by Professor Edward Choi.

My research focuses on assessing AI models before deployment to ensure they align with human values and perception across multiple dimensions, including uncertainty, social ethics, and linguistic diversity. I primarily investigate whether (1) AI models exhibit robustness to human variations, such as dialects, and (2) they can recognize and interpret the world in a manner similar to humans.

Through my work in constructing benchmarks, I have extensive experience conducting human surveys involving over 6,000 participants, actively collaborating with experts from different fields, and applying statistical methods to build robust datasets.

keywords: benchmarks construction, AI-human alignment, safety, social ethics

Feel free to contact me!
e-mail: jiyounglee0523 at kaist dot ac dot kr

[Google Scholar] [Github] [CV]

 

Work Experience

• Jan – June 2026 — Research Intern at Social Computing Group, Microsoft Research Asia (advisor: Fangzhao Wu)
• Sep – Dec 2025 — Research Engineer at EXAONE Lab, LG AI Research (advisor: HeuiYeen Yeen)
• Feb – Aug 2022 — Research Intern at Papago Team, NAVER Corporation (advisor: Cheonbok Park)

 

Professional Service

• Sep 2023 – Feb 2024 — Advisory Committee Member at SelectStar AI NLP Team, LLM Dataset Construction

 

Publications

2025

• Trans-EnV: A Framework for Evaluating the Linguistic Robustness of LLMs Against English Varieties
  Jiyoung Lee, Seungho Kim, Jieun Han, Jun-Min Lee, Kitaek Kim, Alice Oh, Edward Choi
  In Proc. of Neural Information Processing Systems (NeurIPS) 2025 Datasets and Benchmarks
  [Dataset] [Code]

• Single Ground Truth Is Not Enough: Add Linguistic Variability to Aspect-based Sentiment Analysis Evaluation
  Soyoung Yang, Hojun Cho, Jiyoung Lee, Sohee Yoon, Edward Choi, Jaegul Choo, Won Ik Cho
  In Proc. of Conference on Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL) 2025

2024

• KorNAT: LLM Alignment Benchmark for Korean Social Values and Common Knowledge
  Jiyoung Lee, Minwoo Kim, Seungho Kim, Junghwan Kim, Seunghyun Won, Hwaran Lee, Edward Choi
  In Findings of Association for Computational Linguistics (ACL) 2024
  [Official Webpage] [Poster] [Slide]

2023

• VisAlign: Dataset for Measuring the Degree of Alignment between AI and Humans in Visual Perception
  Jiyoung Lee, Seungho Kim, Seunghyun Won, Joonseok Lee, Marzyeh Ghassemi, James Thorne, Jaeseok Choi, O-Kil Kwon, Edward Choi
  In Proc. of Neural Information Processing Systems (NeurIPS) 2023 Datasets and Benchmarks
  [Code] [Poster] [Slide]

• Translation-equivariant Image Quantizer for Bi-directional Image-Text Generation
  Woncheol Shin, Gyubok Lee, Jiyoung Lee, Eunyi Lyou, Joonseok Lee, Edward Choi
  In Proc. of International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 2023 (Oral Presentation, Top 3% Paper Recognition)
  [Code]

2022

• Specializing Multi-domain NMT via Penalizing Low Mutual Information
  Jiyoung Lee, Hantae Kim, Hyunchang Cho, Edward Choi, Cheonbok Park
  In Proc. of Empirical Methods in Natural Language Processing (EMNLP) 2022
  [Poster] [Slide]

• Unifying Heterogenous Electronic Health Records Systems via Text-Based Code Embedding
  Kyunghoon Hur∗, Jiyoung Lee∗, Jungwoo Oh, Wesley Price, Young-Hak Kim, Edward Choi (∗ denotes equal contribution)
  In Proc. of Conference on Health, Inference, and Learning (CHIL) 2022
  [Code]

• Automatic Detection of Noisy Electrocardiogram Signals without Explicit Noise Labels
  Radhika Dua, Jiyoung Lee, Joon-myung Kwon, Edward Choi
  In International Workshop on Pattern Recognition in Healthcare Analytics (PRHA) 2022

2021

• Conditional Generation of Periodic Signals with Fourier-Based Decoder
  Jiyoung Lee, Wonjae Kim, Daehoon Gwak, Edward Choi
  In Deep Generative Models and Downstream Applications Workshop at NeurIPS 2021
  [Code]

2019

• Breast Cancer Subtype Classification utilizing Multi-Omics Data Integration based on Neural Network
  Joungmin Choi, Jiyoung Lee, Jieun Kim, Jihyun Kim, Heejoon Chae
  In Journal of Korean Institute of Information Scientists and Engineers (JOK), Vol. 46 No. 02, pp. 0476–0478, Dec. 2019