I spent the first 13 years of my career developing software for a major telecommunications company. The kids—er, I mean “early career professionals” — that I work with these days would probably be shocked to learn that once upon a time, I used to be able to code like the wind. This skill has served me well in the second chapter of my career.
A few months ago, we were examining some software written in Ada when it became apparent that no one in the room was familiar with the syntax of that language. Since I was the only person alive during Ada’s heyday, some assumed that I would be the obvious choice to close that knowledge gap. Unfortunately, Ada proficiency is mostly useless nowadays, with rare critical exceptions. In any case, I won’t be spending any time in what is left of my career learning a defunct programming language.
In computer science, the principle of “garbage in, garbage out” (GIGO) is a foundational concept that many recently minted software professionals don’t seem to understand. No matter how sophisticated or powerful an algorithm may be, it cannot transform flawed, inaccurate, or incomplete data into reliable results. A perfect algorithm processing bad data will produce bad results. Every time. Accurate outcomes depend on clean, well-structured, and accurate data.
In tennis, match performance data is a perfect example of a fundamentally dirty data set. Tennis is a game of high variance where player performance can swing wildly from one match to the next based on many factors. These variables are difficult —if not impossible — to quantify. From my external perspective, the NTRP algorithm and WTN number don’t even attempt to account for that natural variance. A player who looks unbeatable one week can suddenly struggle the next, not due to a change in skill but because of shifting circumstances that the data can’t capture.
In addition to the legitimate factors influencing tennis performance and results, ratings and ranking algorithms suffer from another fundamental GIGO constraint. Specifically, many players change their competitive behavior as incentivized by the ratings systems put before them.
You get what you measure.
Once any metric becomes the focus of tennis performance, people start optimizing for the metric itself rather than the broader intention it was created to support. In the context of tennis ratings, players often begin making decisions not based on improving their game or seeking out tough competition but rather on what will best protect or enhance their rating. When a system places too much weight on a number, it inadvertently shifts behavior all in service of the metric, not the sport.
This dynamic is particularly challenging for the World Tennis Number (WTN), which is a noble attempt to provide a unified performance ratings system that serves both adult and junior players. The messy reality is that adults and juniors are motivated very differently by that ratings system. Both groups manage their ratings, but the incentives driving their behavior aren’t the same. With two populations of players behaving very differently, the data feeding the algorithms is anything but clean. GIGO ensues.
With that in mind, I’ll be spending the rest of the weekend digging deeper into how juniors and adults each manage their ratings . Watching two very different cohorts of players respond in ways that serve their own unique incentives is fascinating. That mess highlights the importance of developing a better understanding of why the data going into these systems is often so unreliable.
It is a critical first step in developing a clearer picture of how the USTA’s rating systems and competitive frameworks could be improved to better incentivize more consistent high-performance behavior. Tweaking the algorithm can never solve the problem. The most accurate performance ratings data can only come from a clean data set. Until that transpires, garbage in, garbage out will continue.
