The paper “Approximation Algorithms for Restless Bandit Problems” by Guha, Munagala, Shi (2009) designed a $2 + \epsilon$-approximation algorithm for a special class of RMAB (“Feedback MAB”, and generalized to “Monotone MAB”). The algorithm is fundamentally different to the classical Whittle Index. The paper’s analysis uses a duality-based algorithmic technique—it is vastly different compared with Weber (1988)’s proof for RMAB’s asymptotic optimality, hence the $2 + \epsilon$ approximation outcome doesn’t requires asymptotic....

There’s no gene for the human spirit. — Gattaca (1997) Vincent Freeman is born into a genetically engineered society but conceived naturally, deemed ‘in-valid’ due to a heart condition. Determined to defy his genetic fate and reach space, Vincent assumes the identity of Jerome Morrow, a former elite athlete, to join Gattaca’s space program. Gattaca is the stunning directorial debut of Andrew Niccol, who also wrote The Truman Show (1998)....

The visible world is no longer a reality and the unseen world no longer a dream (Kandinsky)

This is beautiful, beautiful, beautiful!

Evoking the rush of water, the stroke of oars and the motion of the ocean, the Barcarolle was a folk song sung by Venetian gondoliers (the word comes from “Barca” meaning “boat”). Characterised by a rocking rhythm, suggestive of the movement of the gondola, a Barcarolle is usually of moderate tempo scored in compound time (often 6/8, 9/8 or 12/8). The genre has been used by many composers to great expressive effect....

Multi-armed bandits with switching costs are a special case of the restless-bandit model. Setup Consider the infinite-horizon, discounted MAB problem with finite state space $\mathcal S$, binary action set ${0,1}$ per arm ($1$ = pull, $0$ = idle), discount factor $0\le\beta<1$, arms evolve only when pulled (i.e. “static” when $a_i=0$), per-pull reward $r_i(s)$. We now add two costs for each arm $i$: switch-in cost $c_i$: paid (once) whenever we switch to arm $i$,...

Are you listening to anything the last time you read/wrote a paper? Sounds of science: how music at work can fine-tune your research Nature | https://www.nature.com/articles/d41586-023-00984-4 Researchers describe how listening to music at work can boost (or hamper) productivity, and share the tunes that keep them focused. TLDR: music cheers you up—almost like a mental massage, dopamine boosters. So it makes tedious, repetitive work less unenjoyable. But music also takes up the brain’s processing power, especially for people with musical training....

“The literature on the RMABP, whether on its theoretical, algorithmic, or application aspects, is currently vast to the point where it is virtually infeasible for researchers to keep up to date with the latest advances in the field.” True. Markovian Restless Bandits and Index Policies: A Review José Niño-Mora | Mathematics, 2023 The review is organized as follows. Section 2 surveys the antecedents to the RMABP, in particular, the classic MABP and the Gittins index policy....

Someone wrote about Swan Lake music: Once ballet music leaves the stage and enters the concert hall or recording, it becomes a kind of group-form symphony. Yet, it lacks the weight of a true symphony or concerto, as the dance drama itself is inherently “light.” Composers have never used ballet to convey grand themes—after all, a fictional prince can leap and twirl, but imagine Peter the Great or Napoleon doing the same; the image borders on the absurd....

We’re back! Last time, we explored unconstrained first-order methods (FOMs), where gradient descent works well and its time-traveling cousins (momentum and acceleration) helped even more. Now adding constraints: Here’s how FOMs extend to constrained problems, especially equality constraints. We’ll walk through two major methods: The Augmented Lagrangian Method with Multipliers (ALMM) and its smoother, modular evolution—ADMM. For constrained problems like this: $$ \min_x ; f(x) \quad \text{s.t.} \quad h(x) = 0,; x \in X $$...