Zhang-Wei Hong

Research: My research focuses on advancing reinforcement learning (RL) methods to overcome the challenges of applying RL to computational discovery problems. Discovery problems span a range of applications, from identifying materials that optimize power density in science to designing robot controllers for complex tasks. These problems involve finding solutions that optimize specific objectives using interaction data from systems with unknown dynamics in black-box settings. I believe RL is particularly well-suited for solving discovery problems because it learns through interaction, akin to how humans discover new knowledge through trial and error. However, existing RL methods face critical challenges that limit their effectiveness for discovery:

• Limited learning signals: RL relies heavily on reward signals from interaction to guide learning, but sparse rewards make it challenging to learn effectively. For instance, when designing a robot controller, rewards might only occur upon successfully completing specific tasks, leaving most interactions useless feedback. While RL theory suggests that agents can asymptotically converge to optimal solutions despite sparse rewards, in practice, RL algorithms often fail to learn the desired solutions within finite time in such scenarios. (References: NeurIPS'24, NeurIPS'23, ICML'23, ICML'23, ICLR'23, ICLR'22, and ICLR'22)
• Lack of diversity: Standard RL algorithms typically aim to find a single optimal solution, whereas many discovery problems require generating a diverse set of high-quality solutions. For example, in drug discovery, identifying multiple candidate compounds with distinct properties is often more valuable than finding a single best compound. (References: ICLR'24)

Bio: Zhang-Wei is a final year Ph.D. student in Electrical Engineering and Computer Science (EECS) at the Massachusetts Institute of Technology (MIT), advised by Prof. Pulkit Agrawal. In 2024, Zhang-Wei was awarded the Qualcomm Fellowship for North America. He completed both his B.S. and M.S. degrees at National Tsing Hua University, working closely with Prof. Chun-Yi Lee and Prof. Min Sun. Previously, Zhang-Wei had the opportunity to collaborate with Prof. Jan Peters at TU Darmstadt in Germany and worked at Preferred Networks.

template from jonbarron