Skill Tags Beat Win Rates 2-to-1 in 700 Strategy Game Logs

When researchers analyze thousands of competitive matches, the obvious metric—who won—turns out to be a surprisingly blunt instrument. In a dataset of 700 annotated strategy game logs, descriptive skill tags predicted player improvement and match outcomes roughly twice as reliably as raw win rates did. The finding raises a precise question: if outcome data is so abundant and so easy to collect, why does it explain so little about the people producing it?

The Measurement Problem With Outcomes

Win rates are seductive because they're cheap. Every match generates one, they aggregate cleanly, and they seem to settle arguments. But a win rate is a compressed signal: it collapses thousands of decisions into a single binary, then averages away the context that made those decisions good or bad. Two players at 55% can be having entirely different seasons—one grinding out narrow victories with conservative play, the other alternating brilliant wins with self-inflicted collapses.

This is a familiar problem in behavioral research. Psychologists have long distinguished between outcome measures and process measures, and the gap between them is where most of the interesting variance lives. A student who guesses correctly on a multiple-choice test and one who reasons correctly produce the same score; only a process measure separates them. Strategy game logs, because they record every action in sequence, are unusually rich process data—if you bother to code them.

The 700-log analysis did exactly that. Rather than treating each match as a win or loss, coders tagged observable behaviors: resource allocation discipline, scouting frequency before commitment, recovery speed after a setback, willingness to abandon a losing position. These tags are descriptive, not evaluative—closer to "did the player re-scout after losing vision" than "was the player good." That distinction matters, and it's where the 2-to-1 predictive advantage came from.

What the Tags Actually Captured

Commitment Under Uncertainty

The strongest single tag family concerned timing of commitment: how long a player gathered information before locking into a strategy, and whether they revised that strategy when new information arrived. Players who committed early and never revised had win rates indistinguishable from players who committed late—but wildly different trajectories over a season. The early committers plateaued; the late committers improved, even when their current results were worse.

This maps onto a well-documented finding in judgment research. Kahneman and Tversky's work on anchoring and adjustment showed that people under-revise from initial estimates; the strategy game logs show the same pattern in miniature, with players anchoring on an opening build and adjusting too little when the opponent's behavior contradicted it. The tag "revised plan after contradictory scouting" is, in effect, a behavioral measure of adjustment—and it predicted future performance better than the outcome it eventually produced.

Loss Aversion in Reverse

Loss aversion—the tendency to weigh losses more heavily than equivalent gains—usually gets discussed as a bias to correct. In competitive play it has a stranger expression. Players who had invested heavily in a position (units built, territory taken, time spent) were measurably slower to abandon it, even when abandonment was clearly correct. The logs show this as a distinctive pattern: continued reinforcement of a failing front while a viable alternative sat unused.

The behavioral economics literature calls this the sunk cost effect, and it's robust across domains. What the game logs add is a measurement: you can timestamp the moment a position became unwinnable and the moment the player left it, and the gap between those two numbers is a clean, quantitative index of sunk-cost susceptibility. That index correlated with seasonal improvement more strongly than win rate did. Players who learned to shorten the gap got better; players whose win rate was high but whose gap was long were, in a sense, winning on borrowed time.

Variable Reinforcement and Its Discontents

One tag family deserves separate treatment because it's where the behavioral psychology literature is most directly relevant: response to intermittent reward. Strategy games, like many competitive environments, deliver unpredictable payoffs—a risky play sometimes works spectacularly, sometimes fails. This is the structure of variable-ratio reinforcement, the schedule that produces the most persistent behavior in the operant conditioning literature, from Skinner's pigeons onward.

The logs show two distinct responses to a lucky success. Some players tagged the strategy as validated and repeated it with increased confidence; others treated it as a sample of one and kept testing. The first group's win rates were, for a while, higher—luck compounds early—but their skill tags stagnated. The second group's win rates were noisier and their tags improved steadily. This is the practical core of the 2-to-1 finding: outcome data rewards the lucky, process data tracks the learning, and over a long enough horizon the second predicts the first.

Why Win Rates Fail as a Learning Signal

There's a statistical story here as well as a behavioral one. Win rates are high-variance estimators early in a season, and the variance doesn't shrink fast enough to guide decisions at the pace players actually make them. A player who loses five close matches and wins five blowouts has the same record as one who wins five close matches and loses five blowouts, but the two are not the same player and should not train the same way.

Process tags reduce this noise because they're measured per-decision rather than per-match. A hundred tagged decisions yield a far tighter estimate of a player's current habits than ten match results do. This is the same logic that led medical educators toward structured clinical assessments rather than relying on patient outcomes alone: outcomes are confounded by case difficulty, and process measures isolate the skill you're trying to teach.

The concrete reference point is the deliberate practice literature. Ericsson's work on expert performance argued that improvement depends on feedback that is specific and immediate—not on the aggregate result of a performance. A win rate is neither specific nor immediate; it arrives after the fact and tells you nothing about which decision to change. A skill tag is both. The 700-log study is, in effect, an empirical demonstration of why that distinction matters at scale.

Designing for Process Feedback

The forward-looking implication is that any competitive environment generating outcome data can probably generate process data too—and should. The tags in this dataset weren't exotic; they were codable from existing logs by people with domain knowledge and a rubric. The bottleneck is attention, not instrumentation.

Three design principles follow. First, tag decisions, not results: "scouted before committing" is a tag; "won the engagement" is not. Second, timestamp the gap between information and response—revision latency is one of the most predictive tags in the dataset and one of the easiest to compute automatically. Third, separate luck from skill explicitly in the feedback loop, so that a single spectacular success doesn't get encoded as a validated strategy. The variable-ratio literature is unambiguous about how sticky that encoding becomes.

For readers of books on decision-making, the broader lesson will feel familiar: the outcome is a noisy sample, the process is the signal. What the strategy logs add is scale—700 matches' worth of evidence that the signal is there, measurable, and roughly twice as informative as the number everyone was already looking at. The next step for anyone collecting performance data is to ask which process tags their environment already supports, and to start coding them before the season ends.