6 ideas
Every activity can be placed on a line running from pure luck (roulette, lotteries) to pure skill (chess, running races). Where an activity sits determines how much a single outcome tells you about ability. It also determines whether you should judge by results or by process, and how large a sample you need before drawing conclusions.
As participants in a competitive field become more skilled and more uniformly trained, the variance in skill across competitors shrinks. Luck then plays a larger role in determining who wins, even though absolute skill has risen. Declining spreads in batting averages or fund returns are evidence of this rising parity, not of declining talent.
Observed outcome variance can be split as observed variance = skill variance + luck variance. You estimate the luck component by simulating what pure chance would produce, for example with coin flips over the same number of trials. Comparing that luck-only spread with the actual spread shows how much of the outcome dispersion reflects genuine skill differences.
Extreme results in any domain tend to move back toward the average. The rate at which they do so measures how much luck produced them. Activities dominated by skill show high persistence of results from one period to the next, while luck-dominated ones revert quickly. The correct forecast is therefore to shrink any extreme observation toward the mean in proportion to luck's contribution.
Near the skill end, results reliably reflect ability, so deliberate practice with fast, accurate feedback is the path to improvement. Near the luck end, outcomes are noisy, so you should evaluate and improve the decision process, such as checklists and base rates, rather than reacting to results. In the middle, both matter, and discipline means not overlearning from small samples.
A good performance statistic must correlate strongly with itself over time, showing it captures skill rather than noise. It must also correlate with the outcome you actually care about, such as winning or profit. Many popular metrics fail one of these two tests. For example, they may be predictive but not persistent, or persistent but disconnected from results.