Independent Reading · 2019

Manipulating a Learning Defender and Ways to Counteract

Jiarui Gan, Qingyu Guo, Long Tran-Thanh, Bo An, and Michael Wooldridge · NeurIPS

Security GamesAdversarial LearningDeception

Full text

Read the Paper

Use View PDF or Download PDF to read the full paper on this device.

Paper Information

Authors
Jiarui Gan, Qingyu Guo, Long Tran-Thanh, Bo An, and Michael Wooldridge
Venue
NeurIPS · 2019
Open official paper

Summary

This paper shows how an attacker can manipulate a defender that learns payoff information from observed best responses.

Core Idea

A defender can counter manipulation by committing to a higher-level policy that maps learned information to strategy commitments.

Method

The authors provide a polynomial-time optimisation method for a finite setting and a heuristic that also applies when the attacker payoff space is infinite or unknown.

Key Takeaways

  • Learning algorithms can create a new attack surface when opponents control the observations.
  • Naive learning may collapse to a maximin outcome under strategic manipulation.
  • Policy commitment can improve defender utility when attacker manipulation is anticipated.

Questions

  • How can manipulation be detected before it significantly changes the learned policy?
  • What policy classes balance protection, computational cost, and interpretability?