Independent Reading · 2024

Transition-Informed Reinforcement Learning for Large-Scale Stackelberg Mean-Field Games

Pengdeng Li, Runsheng Yu, Xinrun Wang, and Bo An · AAAI

Stackelberg GamesMean-Field GamesReinforcement Learning

Full text

Read the Paper

Use View PDF or Download PDF to read the full paper on this device.

Paper Information

Authors
Pengdeng Li, Runsheng Yu, Xinrun Wang, and Bo An
Venue
AAAI · 2024
Open official paper

Summary

This paper studies how a leader can learn incentives for very large populations of homogeneous, self-interested followers.

Core Idea

Empirical transition information can accelerate reinforcement learning in Stackelberg mean-field games without requiring complete environment dynamics.

Method

The authors propose Stackelberg mean-field updates, transition-informed reinforcement learning, and regularized variants evaluated with up to 100,000 followers.

Key Takeaways

  • Mean-field structure enables leader-follower models to scale to large populations.
  • Transition information improves data efficiency over purely model-free learning.
  • Regularization reduces sensitivity to leader-policy initialization.

Questions

  • How robust is the method to heterogeneous follower populations?
  • Can transition estimates remain reliable in non-stationary environments?