Independent Reading · 2024
Transition-Informed Reinforcement Learning for Large-Scale Stackelberg Mean-Field Games
Pengdeng Li, Runsheng Yu, Xinrun Wang, and Bo An · AAAI
Stackelberg GamesMean-Field GamesReinforcement Learning
Full text
Read the Paper
Use View PDF or Download PDF to read the full paper on this device.
Paper Information
- Authors
- Pengdeng Li, Runsheng Yu, Xinrun Wang, and Bo An
- Venue
- AAAI · 2024
Summary
This paper studies how a leader can learn incentives for very large populations of homogeneous, self-interested followers.
Core Idea
Empirical transition information can accelerate reinforcement learning in Stackelberg mean-field games without requiring complete environment dynamics.
Method
The authors propose Stackelberg mean-field updates, transition-informed reinforcement learning, and regularized variants evaluated with up to 100,000 followers.
Key Takeaways
- Mean-field structure enables leader-follower models to scale to large populations.
- Transition information improves data efficiency over purely model-free learning.
- Regularization reduces sensitivity to leader-policy initialization.
Questions
- How robust is the method to heterogeneous follower populations?
- Can transition estimates remain reliable in non-stationary environments?