Jiayi Pan

Jiayi Pan

ę½˜å®¶ę€”

University of Michigan - Ann Arbor

Hi šŸ‘‹

I am Jiayi Pan. A senior student in Computer Science. My research goal to build language agents that operate in physical environments and interact with humans. Towards this goal, my current research focused on language grounding to vision and robotics.

Iā€™m fortunate to have worked with Professors Joyce Chai, Dmitry Berenson, Fan Wu through my undergraduate research.

Outside of work, I love reading, arts, sports on water, implementing random ideas, playing/developing games, being connected with interesting people. Feel free to contact me and have fun via email :)

šŸ“¢: I am applying for PhD programs and Summer Intern for 2023. If you have opportunities, please feel free to contact me via email.

Interests
  • Grounded AI
  • Natural Language Processing
  • Computer Vision
  • Robotics
Education
  • B.S.E in Computer Science, 2021-2023

    University of Michigan, Ann Arbor, U.S.A.

  • B.S.E in Electrical and Computer Engineering, 2019-2023

    Shanghai Jiao Tong University, Shanghai, China

Publications & Manuscripts

šŸ¤« Three of my papers are under anonymous peer review and will be released here once they are accepted šŸ˜‰
* denotes equal contribution
Data-Efficient Learning of Natural Language to Linear Temporal Logic Translators for Robot Task Specification

Jiayi Pan, Glen Chou, Dmitry Berenson. ICRA 2023.

We present a learning-based approach to translate from natural language commands to LTL specifications with very limited human-labeled training data by leveraging Large Language Models. Our model can translate natural language commands at 75% accuracy with about 12 annotations and when given full training data, achieves state-of-the-art performance. We also show how its outputs can be used to plan long-horizon, multi-stage tasks on a 12D quadrotor.
Data-Efficient Learning of Natural Language to Linear Temporal Logic Translators for Robot Task Specification
DANLI: Deliberative Agent for Following Natural Language Instructions

Yichi Zhang, Jianing Yang, Jiayi Pan, Shane Storks, Nikhil Devraj, Ziqiao Ma, Keunwoo Peter Yu, Yuwei Bao, Joyce Chai. EMNLP 2022, Oral.

We propose a neuro-symbolic deliberative agent that, while following language instructions, proactively applies reasoning and planning based on its neural and symbolic representations acquired from past experience. Our deliberative agent achieves 70% improvement over reactive baselines on the challenging TEACh benchmark. Moreover, the underlying reasoning and planning processes, together with our modular framework, offer impressive transparency and explainability to the behaviors of the agent.
DANLI: Deliberative Agent for Following Natural Language Instructions
FairEdit: Preserving Fairness in Graph Neural Networks through Greedy Graph Editing

Donald Loveland, Jiayi Pan, Aaresh Farrokh Bhathena, Yiyang Lu. Manuscript, 2022.

We introduce FairEdit, the unexplored method of edge addition, accompanied by deletion, to promote fairness. FairEdit performs efficient edge editing by leveraging gradient information of a fairness loss to find edges that improve fairness. Our results show that FairEdit performs comparably to many state-of-the-art methods, demonstrating FairEdit's ability to improve fairness across many domains and models.
FairEdit: Preserving Fairness in Graph Neural Networks through Greedy Graph Editing

Selected Honors/Awards

Tang Jun Yuan JI Scholarship
  • 2 out of 100+ students per year | Highest honor for JI’s Dual Degree students
  • Endowed by Cytus Tang Foundation.
John Wu & Jane Sun Excellence Scholarship
  • 15 out of 1000+ students per year | Highest honor for JI undergraduate students
  • Endowed by Mr John Wu and Mrs. Jane Sun.
Fan Xuji Sholarship
  • 10 out of all undergrad students (over 10k) per year | Highest honor for SJTU undergraduate students
  • Endowed by various alumni that were supported by Former University President Fan Xuji.
Chunt-sung Research Scholarhip
  • 50 research projects initiate per year | Top undergraduate research program for SJTU students.
  • Endowed by Tesung-Dao Lee and his wife Hui-Chun Chin.
Second Prize in Chinese Physics Olympiad 2017, 2018
  • Top 100 in Sichuan Province

Back in my highschool days, I was really tired of and bored with all the naive coursework. Just for the fun of it, instead, I spent most of my time learning university-level physics and preparing for the CPhO.

This prize might not look big compared to others listed above, but it is exactly where my passion for science starts. After the “failure” in CPhO, I went on to prepare for the College Entrance Examination (GaoKao) and ranked top 0.1% in 2019.

Now as an AI researcher, I still constantly benefit from the training I had for CPhO. Not only for the domain-specific knowledge I learnt, but also on how to think in the first principle, how to approach, build intuition with, and understand complex systems, etc…

Contact

Schedule:


Direct Contact

  • Email: jiayipan [DOT] umich [DOT] edu