Guoheng Sun

Profile

Hi 🙋, My name is Guoheng Sun, and I am a first-year Ph.D. student in the Department of Electrical and Computer Engineering at the University of Maryland (UMD), advised by Professor Ang Li. I received my bachelor's degree from Sichuan University. My research interests lie in large language models, with a particular focus on efficiency and trustworthiness.


Feel free to reach out for collaboration or discussion:
Github
Google Scholar
LinkedIn
ghsun@umd.edu





Publications

CoIn: Counting the Invisible Reasoning Tokens in Commercial Opaque LLM APIs

CoIn: Counting the Invisible Reasoning Tokens in Commercial Opaque LLM APIs

Guoheng Sun, Ziyao Wang, Bowei Tian, Meng Liu, Zheyu Shen, Shwai He, Yexiao He, Wanghao Ye, Yiting Wang, Ang Li

ABSHTMLPDFCODE
Invisible Tokens, Visible Bills: The Urgent Need to Audit Hidden Operations in Opaque LLM Services

Invisible Tokens, Visible Bills: The Urgent Need to Audit Hidden Operations in Opaque LLM Services

Guoheng Sun*, Ziyao Wang*, Xuandong Zhao, Bowei Tian, Zheyu Shen, Yexiao He, Jinming Xing, Ang Li

ABSHTMLPDF
Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQL

Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQL

Zhewei Yao, Guoheng Sun, Lukasz Borchmann, Zheyu Shen, Minghang Deng, Bohan Zhai, Hao Zhang, Ang Li, Yuxiong He

ABSHTMLPDFCODE
What matters in transformers? not all attention is needed

What matters in transformers? not all attention is needed

Shwai He*, Guoheng Sun*, Zheyu Shen, Ang Li

ABSHTMLPDFCODE
VeriReason: Reinforcement Learning with Testbench Feedback for Reasoning-Enhanced Verilog Generation

VeriReason: Reinforcement Learning with Testbench Feedback for Reasoning-Enhanced Verilog Generation

Yiting Wang*, Guoheng Sun*, Wanghao Ye, Gang Qu, Ang Li

ABSHTMLPDFCODE
Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild

Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild

Xinyu Zhao*, Guoheng Sun*, Ruisi Cai*, Yukun Zhou*, Pingzhi Li*, Peihao Wang*, Bowen Tan, Yexiao He, Li Chen, Yi Liang, Beidi Chen, Binhang Yuan, Hongyi Wang, Ang Li, Zhangyang Wang, Tianlong Chen

Advances in Neural Information Processing Systems (NeurIPS) 2024

ABSHTMLPDFCODE
Flora: Federated fine-tuning large language models with heterogeneous low-rank adaptations

Flora: Federated fine-tuning large language models with heterogeneous low-rank adaptations

Ziyao Wang, Zheyu Shen, Yexiao He, Guoheng Sun, Hongyi Wang, Lingjuan Lyu, Ang Li

Advances in Neural Information Processing Systems (NeurIPS) 2024

ABSHTMLPDFCODE
Shed: Shapley-based automated dataset refinement for instruction fine-tuning

Shed: Shapley-based automated dataset refinement for instruction fine-tuning

Yexiao He, Ziyao Wang, Zheyu Shen, Guoheng Sun, Yucong Dai, Yongkai Wu, Hongyi Wang, Ang Li

Advances in Neural Information Processing Systems (NeurIPS) 2024

ABSHTMLPDFCODE