Fiend at Court Unplugged
Yesterday I wrote about the strengths of the USTA NTRP System. Today we are examining the flip side of that coin, which are the weaknesses. While there are legitimate criticisms that can be made over the fidelity or granularity of the system, the primary issues with the NTRP system are the incentives of the league framework that surrounds it, rather than with the system itself.
Initialization of NTRP Ratings
One of the challenges with any computer rating system is what to do with a player who is new to the system and does not yet have match play data on which to base a rating. In computer engineering this is referred to as initialization. In fact, it is helpful to examine NTRP from the perspective of computer science.
The NTRP Rating is what computer programmers would refer to as a variable, which is a named unit of data that contains a value. It is called a variable because… follow me closely on this… the value of the data can vary over time. In computer programming, all variables have to be set to an initial value before they can be used in meaningful calculations.
The NTRP playing level for an individual player is essentially an attribute of the player that indicates the level of competitive play. Initialization of that variable is required before it can be used in NTRP calculations. That initial value is critical to the calculation and recalculation of the NTRP attribute of the player, partner, and opponents, which occurs with every match recorded into the system. For every match, the NTRP attribute factors in on the dynamic calculation for every player involved with that match.
In the USTA NTRP system, there are actually two attributes of playing level logically assigned to each player. The published NTRP level published each year is treated almost like a constant, though it would be more accurate to say that it is a variable that is only changed once per year. This is the only value that is visible to the tennis consumers.
When a player self-rates, it is essentially setting both the published and dynamic rating variables. That is why self rating is both so impactful and controversial. As I have touched on previously, the self-rating process and guidelines is a major weakness of the NTRP system.
Manual Origins of the NTRP System
As is my custom, let’s take a meandering walk down a rabbit trail. When the NTRP system was first introduced, initial player ratings were usually initially determined by the tennis professionals at the club or facility that the player belonged to. If the pro was unfamiliar with the player, this would typically be done through a playing observation process.
The professional who assigned the initial rating had other factors influencing the determination other than the actual playing level of the player. There is a benefit to the rating professional to underestimate the initial rating.
One reason for this is because the player might be motivated to take additional lessons to move up to the next level. Another factor is that when applied across a large playing population of a club, the club teams would generally do well competitively against other teams. Winning players are happy players.
Players and enterprising captains also quickly came to realize the benefits of being slotted into lower tiers. Rumors and stories of players “tanking” the observational evaluation process were rampant.
Grievances, Appeals, and Disqualifications
The USTA League Program first appeared in the USTA Yearbook in 1980, following creation of the program in 1979. The first set of regulations contained five rules totaling six sentences. There was no mention of grievances or appeals in that inaugural rule set, though those amendments came quickly in the subsequent years. As mentioned yesterday, the USTA objective to have a progression of league playoffs culminating in nationals was included from the inception. We will get back to that shortly.
The 1984 USTA yearbook coined the term NTRP “verifier” that was cited as an innovation by one of the USTA sections. The verifiers were said to be instrumental to provide consistent level of competitive play. Elsewhere in that same year, the Midwest section was touting a team of 8 “clinicians” who had observed all the players at sectionals the previous year and developed more detailed guidelines for performing initial ratings determinations.
The word “grievance” currently appears 196 times in the 2020 USTA league regulations. In contrast, there were only 105 words in the original 1980 league rule specification. This is a sign that something has gone horribly wrong. The issue is not with the NTRP system, but rather around a culture of NTRP ratings manipulation that has sprung up in what I want to say is scattered places. Unfortunately, the reality is that this has become commonplace.
Early manual NTRP rating abuse enforcement mechanisms by the USTA were an unmitigated disaster. Roving ratings officials had the authority to disqualify players at each level of playoffs. I heard several first person accounts from an area playoff in the late eighties in Texas where an official associated with one of the participating CTAs systematically disqualified the best players from all the teams outside of his area.
Computers Solve Everything, Right?
While computerization of the NTRP system was probably inevitable, I will always remain convinced that a large part of the initial push for the conversion was to remove humans bias from the decision making process. Unfortunately the culture of grievances and appeals was already well ingrained in the system by the time the conversion occurred.
The grievance and appeal process is one of the weaknesses in the framework that surrounds the NTRP system. In my experience the players who are most often flagged with strikes and dynamic disqualification are recently self rated players that quite possibly had pure intentions on initial self ratings. On the other hand, the notorious habitual offenders in my area who have to manipulate their ratings through dubious match play never seem to suffer any consequences.
Another interesting aspect of migrating NTRP system to a computer and away from human observation is that consideration of the published the playing characteristics is systemically excluded from consideration. The computer only has the scores of the matches. That fundamentally changes the essence of what the NTRP system is said to be calculating.
The computer cannot possibly be considering playing characteristics when making the NTRP determination. While the descriptions of the playing characteristics are still an roughly valid rule of thumb, it is no longer directly tied to the ratings outcomes. I am convinced that there is a manual “calibration” process still involved that determines the cut off point for each NTRP level each year.
While I believe that manual “calibration” is being performed, I would have no way of knowing that for sure. This is because the USTA NTRP algorithm and other associated factors are shrouded in secrecy. I understand the lack of transparency, which is concern over ratings manipulation abuse. The problem is that some enterprising players and captains are still quite adept at doing that even given the lack of visibility into the process.
One of my local CTAs hosted a NTRP informational session a couple of years ago where a representative from USTA National claimed that some suspicious matches are systematically deleted from the NTRP computer. The Texas section NTRP coordinator recently told me that this has never happened to the best of his knowledge. What bothers me most about those contradictory statements is that I don’t think anyone really knows for sure.
USTA League Nationals and the Tennis Product
As the USTA is working through the restructuring process, one thing that I hope would be considered is the nature of the tennis product that leagues and tournaments are providing to the tennis consumer. Fundamentally I would frame the debate over whether the product is the opportunity to engage in competitive play or the opportunity to advance to nationals.
At the moment “advancing to nationals” seems to be the de facto product. The salient question is whether that should be the case and if that is fundamentally good for tennis. League progression to nationals is a large part of what incentivizes enterprising captains and players to under perform (i.e., tank) to enable team play at an artificially low competitive level.
Having played on a team that went to nationals, I fully appreciate how fun that experience can be. At the same time, I am not sure individualized fun and glory is worth some of the broader negative consequences.
Specifically I am wondering what tennis would look like if the variable ratings calculations were used on a dynamic basis to determine eligibility for level based play. The primary impediment to doing that is the self-imposed annual nature of the NTRP published rating… which is necessary for only one reason. Nationals.
A Year is a Very Long Time
A year is a very long time to lock players into a numeric playing level value in what is in reality a variable and dynamic ratings system. When the yearly ratings updates come out, a player’s fate is sealed to that duration. If ratings floated dynamically, this would create the untenable situation of having most of a team ineligible to play nationals by the time it rolled around.
Are League Nationals Fundamentally Good for Tennis?
There is an easy solution, but it would be very painful and widely unpopular to implement. Eliminate nationals. Alternatively, keep nationals, but only for 5.0. I can hear the wailing and gnashing of teeth. Hear me out.
NTRP nationals at all levels incentivizes the tanking and sandbagging which is both unsportsmanlike and bad for tennis. NTRP nationals is what inspires enterprising captains to encourage individual players to abuse the NTRP self-rating process for a competitive advantage. These abuses create a negative player experience for those around them. Elimination of nationals would also take a slew of systemic problems along with it.
Without nationals forcing a year long definition of playing level, errors in initial self ratings would be quickly corrected and no big deal. Things might be out of kilter for a few matches, but the self correction of the data would happen very quickly. That facilitates more competitive play.
When the USTA restructuring initiative was announced, USTA Executive Officer and Executive Director Michael Dowse was quoted as “We have an opportunity to re-imagine the structure of the organisation to better serve the tennis community in the United States.” Additionally he added “This new structure allows the USTA to be more agile and more cost effective, while getting closer to tennis players at the local level.”
At some point the USTA will have to make some hard decisions about the primary product that the organization provides to the tennis playing community. I would hope and assume that facilitating competitive local play would be high on the priority list. Nobody needs to go to NTRP leveled nationals for that objective to be achieved.
- Kimball, Warren F. (2017) The United States Tennis Association: Raising the Game, University of Nebraska Press, Lincoln, Nebraska.
- 1999 USTA Yearbook, United States Tennis Association, White Plains, NY.
- 1984 USTA Yearbook, United States Tennis Association, H.O. Zimman Inc, Lynn, Massachusetts
- 1980 USTA Yearbook, H.O. Zimman Inc, Lynn, Massachusetts
- 2020 USTA League Regulations, April 24, 2019
- Tennis: USTA Cuts 110 Jobs Due to COVID-19 Fallout, Steve Keating, Reuters/New York Times, June 8, 2020.
- USTA Announces Sweeping Plan to Reorganize and Prioritize Its Structure, Events, and Activities to Grow the Game and Service the Broader Tennis Industry, USTA Official Press Release, hosted on Open Court website, viewed 8/2/2020.
Fiend At Court participates in the amazon associates program and receives a paid commission on any purchases made via the links in this article. Additional details on the disposition of proceeds from this source are available in the “About Fiend at Court” page.
