Athletic performance offers a uniquely quantifiable setting for studying cultural evolution, linking record statistics and evolutionary theory. Analyzing Olympic records and endurance sports, we find broadly universal patterns: linear progress in record values and counts across much of the 20th century. Population growth explains part of this trend but cannot alone account for sport evolution. Instead, progress reflects both expanding participation and evolutionary dynamics of innovation, inheritance, and selection. Applying concepts from population genetics, we find that variance predicts response, consistent with a heritability-like signal in performance. At the level of extremes, world records occur in clustered bursts and we propose a record process with restart reproducing this burstiness.