Our reading group recently wrapped up a sequence of papers under the theme Beyond Bayesian Bandits — or as I like to think of it, “3B.” (Yes, like the composers: Bach, Beethoven, Brahms. Also sometimes a bit boring.)
Anyway, here’s what we’ve covered so far. For some books and papers I don’t attach links but they should be google-able.
📚 Completed Readings Kleinberg: Introduction to Multi-Armed Bandits Slides (Cornell CS6840, 2017) Dumitriu, Tetali, Winkler: Playing Golf with Two Balls Whittle (1980): Multi-armed Bandits and the Gittins Index More useful Gittins Index books: Gittins, Glazebrook and Weber (2011) Multi-armed Bandit Allocation Indices (Second Edition) Qing Zhao (2019) (Section II and III of) Multi-Armed Bandits: Theory and Applications to Online Learning in Networks Hadfield-Menell & Russell (UAI 2015): Multitask Inverse Reinforcement Learning PDF Guha, Munagala, Shi: Restless Bandits with Constraints FOCS 2007 / SODA 2009 Doval & Scully (2024, under review): Local Hedging in Bandits arXiv Chawla, Christou, Harlev, Scully (2025, in submission) arXiv 🔍 Future Readings Gupta, Jiang, Scully, Singla: The Markovian Price of Information arXiv Hajiaghayi, Krysta, Mahdavi, Shin (EC 2025): Delegation with Costly Inspection Banihashem, Hajiaghayi, Krysta, Shin (EC 2025): Delegated Choice with Combinatorial Constraints Ziv Scully & Alexander Terenin: Tutorial: The Gittins Index as a Design Principle (Seems like a solid conceptual anchor for everything above....