Knowledge archive

Paper Library

A growing collection of reading notes across multi-agent learning, reinforcement learning, game theory, online learning, bandits, and robustness.

Independent Reading2020

Multi-agent Reinforcement Learning in Bayesian Stackelberg Markov Games for Adaptive Moving Target Defense

Sailik Sengupta and Subbarao Kambhampati · arXiv preprint

This paper models adaptive moving target defense as a Bayesian Stackelberg Markov game with incomplete information about strategic attackers.

Stackelberg GamesRead notes
Independent Reading2021

Markovian Persuasion

Ehud Lehrer and Dimitry Shaiderman · arXiv preprint

This paper studies Bayesian persuasion when the state of nature evolves according to a Markov process.

Bayesian PersuasionRead notes
Independent Reading2019

Imitative Follower Deception in Stackelberg Games

Jiarui Gan, Haifeng Xu, Qingyu Guo, Long Tran-Thanh, Zinovi Rabinovich, and Michael Wooldridge · ACM EC

This paper examines how a strategic follower can imitate another type and manipulate a leader that is learning from observed responses.

Stackelberg GamesRead notes
Independent Reading2026

Learning in Bayesian Stackelberg Games With Unknown Follower’s Types

Matteo Bollini, Francesco Bacchiocchi, Samuel Coutts, Matteo Castiglioni, and Alberto Marchesi · arXiv preprint

This paper studies online learning in Bayesian Stackelberg games when the leader initially knows neither follower payoffs nor the distribution over follower types.

Stackelberg GamesRead notes
Independent Reading2020

Decoding the Imitation Security Game: Handling Attacker Imitative Behavior Deception

Thanh H. Nguyen, Nam Vu, Amulya Yadav, and Uy Nguyen · ECAI

This paper studies attackers that manipulate historical data by imitating a behavioural model, causing a defender to learn an ineffective strategy.

Security GamesRead notes
Independent Reading2019

Manipulating a Learning Defender and Ways to Counteract

Jiarui Gan, Qingyu Guo, Long Tran-Thanh, Bo An, and Michael Wooldridge · NeurIPS

This paper shows how an attacker can manipulate a defender that learns payoff information from observed best responses.

Security GamesRead notes
Independent Reading2021

Partial Adversarial Behavior Deception in Security Games

Thanh H. Nguyen, Arunesh Sinha, and He He · Conference paper

This paper considers a population of attackers in which only part of the observed behaviour is controlled by a deceptive adversary.

Security GamesRead notes
Independent Reading2024

Online Bayesian Persuasion Without a Clue

Francesco Bacchiocchi, Matteo Bollini, Matteo Castiglioni, Alberto Marchesi, and Nicola Gatti · NeurIPS

This paper studies repeated Bayesian persuasion when the sender knows neither the prior over states nor the receiver’s utilities.

Bayesian PersuasionRead notes
Independent Reading2025

Bayesian Persuasion with Externalities: Exploiting Agent Types

Jonathan Shaki, Jiarui Gan, and Sarit Kraus · AAAI

This paper examines how a principal can inform and coordinate multiple agents whose utilities depend on one another’s actions.

Bayesian PersuasionRead notes
Independent Reading2020

Online Bayesian Persuasion

Matteo Castiglioni, Andrea Celli, Alberto Marchesi, and Nicola Gatti · NeurIPS

This paper studies repeated persuasion against receivers whose unknown types may be selected adversarially over time.

Bayesian PersuasionRead notes
Independent Reading2022

Inverse Game Theory for Stackelberg Games: The Blessing of Bounded Rationality

Jibang Wu, Weiran Shen, Fei Fang, and Haifeng Xu · NeurIPS

This paper studies how a leader can infer unknown follower utilities from observed behavior in Stackelberg games.

Stackelberg GamesRead notes
Independent Reading2023

Actions Speak What You Want: Provably Sample-Efficient Reinforcement Learning of the Quantal Stackelberg Equilibrium from Strategic Feedbacks

Siyu Chen, Mengdi Wang, and Zhuoran Yang · arXiv preprint

This paper considers learning a quantal Stackelberg equilibrium in episodic leader-follower Markov games from follower actions.

Stackelberg GamesRead notes
Independent Reading2022

Choices Are Not Independent: Stackelberg Security Games with Nested Quantal Response Models

Tien Mai and Arunesh Sinha · AAAI

This paper extends quantal-response security games to represent correlations among an attacker’s choices through a nested decision model.

Security GamesRead notes
Independent Reading2016

Combinatorial Multi-Armed Bandit with General Reward Functions

Wei Chen, Wei Hu, Fu Li, Jian Li, Yu Liu, and Pinyan Lu · NeurIPS

This paper generalizes combinatorial bandits to nonlinear rewards that may depend on complete outcome distributions rather than only their means.

Multi-Armed BanditsRead notes
Independent Reading2022

Contextual Bandits with Large Action Spaces: Made Practical

Yinglun Zhu, Dylan J. Foster, John Langford, and Paul Mineiro · arXiv preprint

This paper develops practical contextual-bandit algorithms for large or continuous linearly structured action spaces.

Contextual BanditsRead notes
Independent Reading2024

Global Rewards in Restless Multi-Armed Bandits

Naveen Raman, Zheyuan Ryan Shi, and Fei Fang · arXiv preprint

This paper extends restless multi-armed bandits to objectives with global, non-separable rewards across selected arms.

Restless BanditsRead notes
Independent Reading2011

X-Armed Bandits

Sébastien Bubeck, Rémi Munos, Gilles Stoltz, and Csaba Szepesvári · Journal of Machine Learning Research

This paper studies stochastic bandits over general measurable action spaces with local smoothness expressed through a known dissimilarity function.

X-Armed BanditsRead notes
Independent Reading2025

Non-Stationary Lipschitz Bandits

Nicolas Nguyen, Solenne Gaucher, and Claire Vernade · NeurIPS

This paper studies continuous-action Lipschitz bandits whose reward functions may change arbitrarily over time.

Non-Stationary BanditsRead notes
Independent Reading2023

PyXAB: A Python Library for X-Armed Bandit and Online Blackbox Optimization Algorithms

Wenjie Li, Haoze Li, Jean Honorio, and Qifan Song · arXiv preprint

This paper presents PyXAB, an open-source Python library for X-armed bandit and online black-box optimization algorithms.

X-Armed BanditsRead notes
Independent Reading2024

Transition-Informed Reinforcement Learning for Large-Scale Stackelberg Mean-Field Games

Pengdeng Li, Runsheng Yu, Xinrun Wang, and Bo An · AAAI

This paper studies how a leader can learn incentives for very large populations of homogeneous, self-interested followers.

Stackelberg GamesRead notes
Independent Reading2024

Uncoupled Learning of Differential Stackelberg Equilibria with Commitments

Robert Loftin, Mustafa Mert Çelikok, Herke van Hoof, Samuel Kaski, and Frans A. Oliehoek · AAMAS

This paper develops decentralized learning dynamics for differential Stackelberg equilibria when agents cannot access one another’s payoff functions.

Stackelberg GamesRead notes
Independent Reading2025

Playing Repeated Stochastic Security Games Against Non-Stationary Attackers

Ling Chen and Runfa Zhang · Mathematics

This paper studies repeated security games where an attacker’s latent behavior type changes stochastically and without notification.

Security GamesRead notes