Independent Reading · 2019
Manipulating a Learning Defender and Ways to Counteract
Jiarui Gan, Qingyu Guo, Long Tran-Thanh, Bo An, and Michael Wooldridge · NeurIPS
Security GamesAdversarial LearningDeception
Full text
Read the Paper
Use View PDF or Download PDF to read the full paper on this device.
Paper Information
- Authors
- Jiarui Gan, Qingyu Guo, Long Tran-Thanh, Bo An, and Michael Wooldridge
- Venue
- NeurIPS · 2019
Summary
This paper shows how an attacker can manipulate a defender that learns payoff information from observed best responses.
Core Idea
A defender can counter manipulation by committing to a higher-level policy that maps learned information to strategy commitments.
Method
The authors provide a polynomial-time optimisation method for a finite setting and a heuristic that also applies when the attacker payoff space is infinite or unknown.
Key Takeaways
- Learning algorithms can create a new attack surface when opponents control the observations.
- Naive learning may collapse to a maximin outcome under strategic manipulation.
- Policy commitment can improve defender utility when attacker manipulation is anticipated.
Questions
- How can manipulation be detected before it significantly changes the learned policy?
- What policy classes balance protection, computational cost, and interpretability?