I spent last weekend at USTA Texas 18+ League Sectionals. A side effect of that is spending a fair amount of time hearing people grouse about NTRP ratings. Strong performances at Championship events by both self-rated and long-time computer-rated players inevitably generate conversation. Sometimes those complaints may be entirely justified. At other times they may reflect ordinary variability, rapid improvement, an unfavorable matchup, or sour grapes following a loss.
Underpinning those conversations is an inherent assumption that a single objectively correct rating can be identified. A player that someone eyeballs as “too strong” for their designated NTRP level evokes perceptions of impropriety. That runs from intentional self-rating malfeasance to the practice colloquially referred to as “ratings management.” It is unfortunate that those attitudes are so prevalent, but it is an indelible part of the culture of NTRP leveled play.
I have recently come to appreciate a different facet of the NTRP cultural conundrum from an unexpected source: retirement planning. There are an astonishing number of variables to consider when determining when to retire and how to finance the decades that follow. Investment returns, taxes, Social Security, a pension (which I am fortunate to have), healthcare costs, inflation, spending, and longevity can all be modeled and projected.
The temptation to dive down a rabbit trail of spreadsheets comparing different assumptions and running projections extending decades into the future is real. The exercise is useful because it exposes tradeoffs and helps identify decisions that appear more or less reasonable under different circumstances. However, there is no spreadsheet sophisticated enough to tell us exactly what inflation will do twenty years from now, how markets will perform, what tax law will look like, how long me and the Trophy Husband will live, or what our priorities will evolve to in the future.
Adding more variables and increasingly sophisticated analysis can improve the understanding of the range of possibilities without ever revealing the one objectively correct retirement plan, because that does not exist. In other words, my retirement planning is a real-world useful example of the distinction between a system or problem that is complicatedversus one that is complex.
The world tends to use those words interchangeably, but they describe fundamentally different kinds of problems. A complicated problem contains many aspects, but each of those is possible to decompose into deterministic solutions. The pieces can be separated, understood, analyzed, and eventually put back together. Expertise and additional information generally help because cause and effect can be understood well enough to determine what should happen under a particular set of conditions. Complicated problems may be extremely difficult, but sufficient analysis can produce a solution.
Complex systems are a different beast. The interactions among the components matter as much as the individual pieces. Conditions change, feedback alters future behavior, and the same action may produce different outcomes under different circumstances. More information can improve our understanding of the system without necessarily producing a single answer that remains correct over time.
When people are part of the system, the dynamic variability rises dramatically. People observe rules, interpret incentives, form strategies, and adapt. Changes intended to influence one part of the system can alter behavior somewhere else, which then changes the environment in which the original intervention occurred.
This brings me to the assertion that the NTRP ratings system and the associated competitive ecosystem are a complex system. The actual algorithm that calculates the rating is arguably complicated, but the competitive aspects make it complex.
Tennis is a game of high variance. Player performance varies from day to day and changes over time. Matchups and doubles partners matter. Injuries, fitness, coaching, confidence, age, and experience can all influence results. On top of all that, players and captains understand that ratings impact their competitive opportunities and prospects, so they respond to those consequences.
NTRP ratings determine eligibility for both league and tournaments. It gates partner matchups and advancement opportunities. In many cases, NTRP is a demarcation point between a player who has the opportunity to compete and where those opportunities do not exist at all. An NTRP rating isn’t just a metric of playing ability. It has consequences, and those consequences become inputs into future behavior.
The NTRP self-rating questionnaire takes incomplete information about a human being and produces a range of acceptable discrete levels. That is necessary because NTRP-leveled competition defines those boundaries. However, a side effect of that is a false sense of precision about what an NTRP rating actually represents. It is much the same withcomputer-rated players.
Two tennis players assigned to the same NTRP level can look nothing alike. A player who looks ordinary against one opponent may appear dominant against another. Someone can improve materially over the course of a season. A self-rated player may enter the system at a completely reasonable level and become much stronger a few months later. Conversely, a player can produce very good tennis during a championship match, but that does not mean that the original rating was improper.
I am not saying that egregious self-rating never occurs. Intentional or accidental misclassification is a real possibility. So is legitimate and dramatic performance improvement. However, it is always important to understand that exceptional performance by itself does not establish which mechanism produced it.
This is where I think the complicated-versus-complex distinction becomes useful. If NTRP were merely complicated, we could reasonably expect that enough data, a sufficiently sophisticated algorithm, or a perfectly constructed self-rating questionnaire would eventually produce the correct answer. That seems to be what the players who sit around grousing about NTRP ratings expect.
Complexity explains why that expectation isn’t realistic. The objective of the NTRP system is to produce a reasonable discrete classification from imperfect information and then continue to observe what happens as the player interacts with the system.
That helps explain why ratings change, and why mechanisms exist to address classifications that no longer appear appropriate. The NTRP algorithm is not producing a permanent truth about a player. Rather, it is continually updating an assessment as additional information becomes available.
In other words, I have come to suspect that some of the frustration surrounding ratings comes from expecting more certainty than the underlying system can provide. When a self-rated player dominates at Sectionals, the immediate conclusion is often that someone got the rating wrong. Sometimes that may be exactly what happened. In other cases, we may simply be observing the normal behavior of a complex system populated by human beings who improve, fluctuate, respond to circumstances, and occasionally surprise us.
Complexity does not excuse poor rating decisions or mean that the system cannot be improved. In many respects, it makes continuous improvement more important. A complex system cannot be perfected through one brilliant fix because any intervention occurs within a system that continues changing after the decision is made.
I have long asserted that the first step in solving any problem is fully understanding it. This adds another layer to that idea. Before trying to solve a problem, we also need to understand the nature of what we are confronting. Complicated problems are difficult because the answer is hard to find. Complex systems are even more challenging because there is no provably correct answer that persists over time.
This perspective is an additional nuance of what good problem solving looks like. A complicated problem can be solved through expertise and analysis. With a complex problem, we can only make the best intervention we can based on the information available, observe what actually happens, and remain prepared to adjust as the system responds.
That brings us back to another aspect of NTRP that I want to explore further tomorrow. The people being measured by the ratings system understand that the measurement carries consequences. Players and captains respond to the incentives those consequences create, sometimes in ways that do not fit comfortably within our traditional ideas about good sportsmanship. Once a measurement begins influencing the behavior it is intended to measure, the problem becomes even more complex.
- The Critical Difference Between Complex and Complicated, Theodore Kinni, MIT Sloan Management Review, June 21, 2017. This article appears to be behind a paywall, but you can set up a free account to access three articles per month. This is a useful source for anyone who wants to further explore the differences between complicated and complex problems.
