Our reading group recently wrapped up a sequence of papers under the theme Beyond Bayesian Bandits — or as I like to think of it, “3B.” (Yes, like the composers: Bach, Beethoven, Brahms. Also sometimes a bit boring.)
Anyway, here’s what we’ve covered so far. For some books and papers I don’t attach links but they should be google-able.
📚 Completed Readings
- Kleinberg: Introduction to Multi-Armed Bandits Slides (Cornell CS6840, 2017)
- Dumitriu, Tetali, Winkler: Playing Golf with Two Balls
- Whittle (1980): Multi-armed Bandits and the Gittins Index
- More useful Gittins Index books:
- Gittins, Glazebrook and Weber (2011) Multi-armed Bandit Allocation Indices (Second Edition)
- Qing Zhao (2019) (Section II and III of) Multi-Armed Bandits: Theory and Applications to Online Learning in Networks
- Hadfield-Menell & Russell (UAI 2015): Multitask Inverse Reinforcement Learning PDF
- Guha, Munagala, Shi: Restless Bandits with Constraints FOCS 2007 / SODA 2009
- Doval & Scully (2024, under review): Local Hedging in Bandits arXiv
- Chawla, Christou, Harlev, Scully (2025, in submission) arXiv
🔍 Future Readings
- Gupta, Jiang, Scully, Singla: The Markovian Price of Information arXiv
- Hajiaghayi, Krysta, Mahdavi, Shin (EC 2025): Delegation with Costly Inspection
- Banihashem, Hajiaghayi, Krysta, Shin (EC 2025): Delegated Choice with Combinatorial Constraints
- Ziv Scully & Alexander Terenin: Tutorial: The Gittins Index as a Design Principle (Seems like a solid conceptual anchor for everything above.)