Gneiting and Raftery (2007) | Strictly proper scoring rules, prediction, and estimation

Scoring rules are really cool, and useful. This seminal survey paper has over 6k citations as of April 2025 :o Strictly proper scoring rules, prediction, and estimation tandfonline [pdf] Scoring rules assess the quality of probabilistic forecasts, by assigning a numerical score based on the predictive distribution and on the event or value that materializes. A scoring rule is proper if the forecaster maximizes the expected score for an observation drawn from the distribution F if he or she issues the probabilistic forecast F, rather than G ≠ F. It is strictly proper if the maximum is unique. In prediction problems, proper scoring rules encourage the forecaster to make careful assessments and to be honest. In estimation problems, strictly proper scoring rules provide attractive loss and utility functions that can be tailored to the problem at hand. This article reviews and develops the theory of proper scoring rules on general probability spaces, and proposes and discusses examples thereof. Proper scoring rules derive from convex functions and relate to information measures, entropy functions, and Bregman divergences. In the case of categorical variables, we prove a rigorous version of the Savage representation. Examples of scoring rules for probabilistic forecasts in the form of predictive densities include the logarithmic, spherical, pseudospherical, and quadratic scores. The continuous ranked probability score applies to probabilistic forecasts that take the form of predictive cumulative distribution functions. It generalizes the absolute error and forms a special case of a new and very general type of score, the energy score. Like many other scoring rules, the energy score admits a kernel representation in terms of negative definite functions, with links to inequalities of Hoeffding type, in both univariate and multivariate settings. Proper scoring rules for quantile and interval forecasts are also discussed. We relate proper scoring rules to Bayes factors and to cross-validation, and propose a novel form of cross-validation known as random-fold cross-validation. A case study on probabilistic weather forecasts in the North American Pacific Northwest illustrates the importance of propriety. We note optimum score approaches to point and quantile estimation, and propose the intuitively appealing interval score as a utility function in interval estimation that addresses width as well as coverage. ...

April 16, 2025

Expectation of the Maximum Order Statistic of a Power Law Distribution

Didn’t know calculating expectation would take this long… Consider $n$ i.i.d. random variables $X_i$ drawn from Power Law distribution with support lowerbound $x_\text{min} = 1$ and the shape parameter $\alpha > 2$—in other words, the pdf of $X_i$ being $$ f_{X_i}(x) = (\alpha - 1) x^{-\alpha }, $$ and $$ F_{X_i}(x) =1 - x^{-(\alpha - 1)} \quad \text{ for }x \ge 1. $$ Btw, for the heavy-tailed distribution family, it can be more comfortable (and general) to understand random variable $X$ as $$ \Pr[X > x] \sim x^{-\alpha}. $$ see Ibragimov and Walden (MS'10)’s Section 3 “Heavy-Tailed Distributions” for a cooler, less intuitive way of seeing it. ...

April 15, 2025

YouTube Ads for an Ad-Blocker that Pays You to View Ads

So are advertisement good or bad? There’s no fixed answer. One view says it’s great cause advertisers have to pay for ads—so only sellers who believe that their own products/services are of good quality will use ads (if ads price and the ads market adjust appropriately)—then ads become informative and useful signals, and they would help consumers distinguish valuable products out of the sea of alternatives. Another view is that ads are more like competitors for attention—and it becomes a bad equilibrium now when seller would have to buy ads to at least get any attention. ...

April 14, 2025

C&A 2025 interesting paper collections

Four papers (topic) from the C&A (complexity & algorithms) workshop 2025 that I personally found interesting (or, perhaps, just understood), hosted by Sun Yat-Sen University, Guangzhou China. Two EconCS papers: Persuasive Calibration Yiding Feng and Wei Tang, 2025 (Arxiv link) The paper studies a persuasion problem consisting of an information sender and a receiver. It’s somewhat like information design—but calibration replaced commitment to an information policy. The model is restricted to a state in [0, 1] (unknown success probability of a Bernoulli experiment). Calibration means when the sender sends an signal $q$ it’s close to the expected value of the real state. ...

April 13, 2025

Mendelssohn's Scherzo from A Midsummer Night's Dream

I’ve blogged about Mendelssohn’s Scherzo from a A Midsummer Night’s Dream previously (two interesting Mendelssohn paraphrases—Rachmaninoff’s genius piano transciption of Mendelssohn’s incidental music). But today let’s talk about the piece itself, in its most authentic orchestral form. Mendelssohn is a big fan of Shakespeare and particular the Midsummer play. The composer’s sister Fanny explained it this way. “From our youth on we were entwined in A Midsummer Night’s Dream, and Felix [Mendelssohn] particularly made it his own. He identified with all the characters. He re-created them, so to speak, every one of those whom Shakespeare produced in the immensity of his genius.” ...

April 12, 2025

Theoretical Computer Science (TCS) Special Column

My undergraduate advisor Pinyan Lu has published a special introductory column for theoretical computer science in the Communication of CCF (China Computer Federation). The Chinese version is available here. TCS field is not only home to many powerful tools (and brilliant minds), but it also boasts a beautiful, elegant philosophical methodology—a way of thinking that is deeply fundamental. Snippets from the article: [TCS has] its unique internal way of thinking. I believe that most work in theoretical computer science follows a three-step pattern: ...

April 11, 2025

the Ultimate iPhone Decision Problem Wait-or-Buy?

See previous post for the idea: My Ex, Apple PR, and the Economics of (Not) Buying an iPhone — for iteratively updated products like iPhone, do consumers strategically decide on Wait-to-Buy? It turns out that the market does anticipate upcoming product and update expectations w.r.t. new information. When Trump first announced the Tariff raise, Apple Pro Fans quickly reacts and adjusted their policy for iPhone Purchase: Should you buy an iPhone right now? Chance Miller 9to5mac ...

April 10, 2025

Apple Air — When iPhones Fly First Class

Companies react strategically to tariff raise: Apple transported five planes full of iPhones and other products from India to the US in just three days during the final week of March. The urgent shipments were made to avoid a new 10% reciprocal tariff. Gadgets Now Although Apple’s headquater is in California, iPhones are manufactured overseas—China, India. Apple currently assembles the entire iPhone 15 and iPhone 16 lineups in India as well as China. A 10% baseline tariff on all imports into the United States kicked in on Saturday. On April 9, the tariffs that Trump has falsely labeled as “reciprocal” will kick in. This will raise the tariff rate on imports from China to 54% and imports from India to 27%. ...

April 9, 2025

Multiobjective Optimization and Tchebycheff Scalarization (and something about Pareto)

I always like attending job talks. You learn something, be awed by cutting edge research, and from time to time, reflect on skills of academic presentations… Today’s topic is Multi-Objective Optimization (MOO). When sometimes for an optimization problem there are two or more conflicting objectives (e.g. fairness vs. efficiency), it is not guaranteed that a single solution simultaneously optimizes each objective. Naturally, we consider Pareto Efficiency as a major concept for solution—a solution where none of the objective functions can be improved in value without degrading some of the other objective values. By the way, quick overview for Pareto: ...

April 8, 2025

A Meme about Machine Learning Papers

When will we get a “XXX is all you need” paper published in an economic journal? (This comes from an ML paper ‘Attention is All You Need’, which introduced ‘Transformers’—the tech behind ChatGPT. Since then, there’s been a flood of papers claiming ‘X is all you need’ especially in computer science papers… ) source: Types of Machine Learning Papers Reddit

April 7, 2025