How TrainRun Measures Learning Progress

Why Scoring Matters for Learning

Most educational platforms reduce performance to a single number - a test score, a percentage correct, a letter grade. This conflation obscures critical distinctions: a student who scores 80% through genuine understanding looks identical to a student who scores 80% through memorization of surface patterns. Yet their future learning trajectories are entirely different.

TrainRun separates performance into three independent axes, each measuring a fundamentally different aspect of learning. These axes are never mixed or cross-compared - they provide orthogonal views into what happened during a run.

The Three-Axis Scoring System

Every run produces three independent scores:

// Three-axis scoring - orthogonal measurement
// From TrainRun's engine (scoring.ts)

COMPETITIVE SCORE - execution quality × challenge rating
  Base: pickups collected, chains completed, obstacles avoided
  Multiplier: challenge rating (higher difficulty = higher ceiling)
  Speed bonus: collecting at 2x speed earns double points per pickup
  Chain streak: consecutive chain completions build multiplier

LEARNING SCORE - per-run knowledge signal
  Concept coverage: unique concepts encountered and collected
  Chain accuracy: how many concept chains completed vs broken
  Collection rate: successful retrievals / total opportunities
  → Feeds into long-term mastery via EMA aggregation

COMPLETION SCORE - content consumption
  Pickups: collected / total available
  Chains: completed / total attempted
  Progress: how much of the content timeline was traversed

Competitive score measures how well you executed under the challenge conditions you chose. It rewards skill - fast reactions, strategic lane changes, consistent chain completions - and scales with the difficulty level. This is the axis that drives leaderboards and personal bests.

Learning score measures the knowledge signal from the run - how many unique concepts you engaged with, how accurately you completed concept chains, and your overall collection rate. This is the axis that drives mastery tracking. A run where you collected 95% of pickups at low difficulty produces a strong learning signal even if the competitive score is modest.

Completion score measures content consumption - how much of the material you actually traversed. This is a pure progress metric: did you finish the run, or did you exit early?

Mastery Tracking with Diminishing Alpha

Individual run scores are volatile - one good run doesn't prove mastery, and one bad run doesn't prove incompetence. TrainRun aggregates learning scores over time using an Exponential Moving Average (EMA) with a diminishing alpha:

// Mastery aggregation: EMA with diminishing alpha
// Prevents score inflation from grinding

function aggregateMastery(currentMastery, newRunScore, runCount) {
  // Alpha decreases with experience: min(0.3, 1/runCount)
  // First run: alpha = 1.0 (new score dominates)
  // Third run: alpha = 0.33 (equal weighting)
  // Tenth run: alpha = 0.1 (incremental updates)
  const alpha = Math.min(0.3, 1 / runCount)

  return currentMastery * (1 - alpha) + newRunScore * alpha
}

// Why diminishing alpha?
// - Early runs have high influence (bootstrapping)
// - Later runs have low influence (stability)
// - Prevents grinding: replaying 100 times doesn't inflate mastery
// - A single bad run doesn't destroy established mastery

This design encodes an important educational principle: mastery is demonstrated through consistent performance over time, not through a single peak. The diminishing alpha means early runs establish a baseline quickly, while later runs can only shift mastery gradually. You cannot game the system by grinding - each additional run contributes less than the previous one.

Challenge Rating

The competitive score is multiplied by a challenge rating that reflects the difficulty conditions during the run. Challenge rating is computed from spatial difficulty components:

• Density - how many pickups and obstacles per unit of track length.
• Reaction window - how much time you have to react to obstacles.
• Lane pressure - how frequently you must switch lanes.

Playback speed is deliberately excluded from the rating formula because it is already applied per-pickup at collection time (collecting at 2x speed earns double points). This prevents double-counting - speed influences scoring once through the per-pickup multiplier, and spatial challenge influences scoring once through the rating.

Personal Bests and Difficulty Categories

Personal bests are tracked per difficulty category: linear, scaling, and custom modes each have their own leaderboard. Scores from different modes are never compared because they measure performance under fundamentally different conditions.

This separation means you can hold a personal best in linear mode (consistent challenge) and a separate personal best in scaling mode (progressive ramp) without them competing. Each category represents a different skill: linear rewards endurance, scaling rewards adaptability, and custom rewards mastery of specific challenge configurations.

Scoring Rewards Skill Over Grinding

A deliberate design decision: each run is independent with no cumulative bonuses. There are no experience points that accumulate over time, no leveling system, no unlock mechanics that reward time spent rather than skill demonstrated. Replaying a module only matters if you beat your personal best in that difficulty category.

This anti-grinding philosophy aligns with learning science: if the goal is durable mastery, then replaying content that has already been mastered produces diminishing returns. The scoring system reflects this by making each run's value independent of prior runs. The spaced repetition model determines when review is actually needed; the scoring system only measures how well you perform when you do review.

Connecting Scoring to Confidence

The three-axis system provides the quantitative foundation, while confidence calibration in quizzes provides the qualitative layer. Together, they paint a complete picture: the scoring axes tell you what happened (how many concepts collected, at what speed, how completely), while confidence data tells you the metacognitive quality of your knowledge (whether you knew that you knew it, or just got lucky).