Independent Reading · 2020

Multi-agent Reinforcement Learning in Bayesian Stackelberg Markov Games for Adaptive Moving Target Defense

Sailik Sengupta and Subbarao Kambhampati · arXiv preprint

Stackelberg GamesReinforcement LearningAI Security

Full text

Read the Paper

Use View PDF or Download PDF to read the full paper on this device.

Paper Information

Authors
Sailik Sengupta and Subbarao Kambhampati
Venue
arXiv preprint · 2020
Open official paper

Summary

This paper models adaptive moving target defense as a Bayesian Stackelberg Markov game with incomplete information about strategic attackers.

Core Idea

Combine a sequential leader-follower security model with multi-agent reinforcement learning so a defender can learn movement policies without a complete reward or transition model.

Method

The authors introduce Bayesian Strong Stackelberg Q-learning and study its convergence to a Strong Stackelberg Equilibrium in the proposed game model.

Key Takeaways

  • Sequential security decisions require both strategic reasoning and adaptation over time.
  • Incomplete information about attacker types can materially change defensive policies.
  • Game-theoretic structure can be incorporated into reinforcement-learning algorithms.

Questions

  • How sensitive are learned defense policies to inaccurate assumptions about attacker types?
  • How does the approach scale as the number of system configurations and attacker models grows?