Figure Makers: Hard-Core Computing

by | 10.18.2018 | 12:26pm
An example of a Brisnet past performance

So far in our series on speed figures, we've seen how numbers get adjusted using the Beyer method, how pace is incorporated into TimeformUS ratings and how so-called “designer” figure makers like Thoro-Graph and The Sheets take into account factors such as ground loss and wind speed.

In our final installment, we're looking at two companies that unapologetically turn everything over to a computer. Everything.

“Ninety-nine point nine nine nine percent of the time there's no human intervention,” said David Siegel, president of Trackmaster, which produces Equibase Speed Figures. “I believe we may have changed, with human input, one speed rating in the last five years. So I want to say it's virtually zero.”

Brisnet's Ed DeRosa doesn't quite take it that far — human minds do occasionally step in — but it's still pretty rare.

“I would definitely say our philosophy is strongest toward 'the number is the number',” said DeRosa. “We very rarely split variants, look for reasons to upgrade or downgrade and we don't not do that because we don't think there is a benefit. It's more: other people do it, number one, and number two, we think there's some strength to a solid 'number is the number' approach where we believe in our pars and variants and feel like on a race-to-race basis, that's what gives the most accurate presentation of the quality of the race.”

The Brisnet and Equibase algorithms are based on traditional formulas of final times, pars and variants with adjustments made for beaten lengths. They're using a projection method; data suggests how a race should be run, then the actual results are compared with that notion.

Briefly breaking down the Equibase computer model will give a better idea of how this works. Equibase uses two variants. One is called the Inter-Track Variant, which measures the fundamental difference in speed between racetracks, not day-to-day changes but things like size, sharpness of turns, banking, composition of the surface — elements that usually don't vary over time. Equibase has an Inter-Track Variant for each distance/track combination.

“Ideally, what we'd want to know is if you had a robotic horse and you set it to expend a certain amount of energy in, let's say, a six-furlong race at Golden Gate Fields, and that robotic horse ended up running it in a minute and ten seconds,” said Siegel. “Then, leaving those settings exactly the same, the robotic horse ran a six-furlong race at Aqueduct in a minute and 11 seconds. We would deem the Aqueduct six-furlong dirt combination to be one second slower than the Golden Gate distance-surface combination.”

Of course, we don't have robotic horses for this purpose, so Equibase replicates the idea by doing a massive study of runners shipping from track to track every year.

The second variant, the Race Variant, is akin to the variants we've already discussed, designed to account for surface changes on a particular day. The Race Variant is split between turf and dirt as well as condition — so, fast versus sloppy or good versus firm. But again, the computer does all of that. Even in a case where evidence suggests the track sped up throughout a card, the computer model should pick up on it, assuming there's an identifiable pattern, and it will automatically correct.

Equibase uses something called a Class Rating, which is akin to the “pars” we've already discussed, but there is one major difference. While other pars are based on class levels using a huge set of previous races, the Equibase Class Rating is determined by the actual horses in the field for a particular race – it's an averaging of their recent speed figures.

“You could have two $20,000 claiming races at Aqueduct, each at a mile on the same day. Most people would say those two races have the same par, using that word, because it's usually class-based,” said Siegel. “That's not what our number is. Our number is the expected winning speed rating of each of those races, and it could be that if one group of horses is a little bit stronger just by random than the other, the class rating for one of those races could be an 82 and the other one could be an 80.”

The computer will adjust those ratings using the variants and actual results to create a final figure.

DeRosa said when he handicaps, he considers the Brisnet number a starting point to compare horses and then he'll come back to it at the end of the process to use it as a deciding factor between horses he likes and/or ones he might toss.

“We definitely try on a day-to-day basis to stand by the number and it does lead to differences in opinion with Beyer or other figure makers,” DeRosa said. “But it's a parimutuel game and I think it's worth asking which one you trust and what price you're getting to play a certain number if there's a big difference.”

DeRosa said there may be rare cases where human intervention is required so the results aren't skewed — for example, there are very few races at 1 3/4 miles on the turf, and they're almost always stakes horses. The sample size and the caliber of the winners can inflate pars.

Another example: Student Council winning the 2007 Pacific Classic at a mile and a quarter in an incredibly slow time of 2:07.29. The BRIS Speed Rating that came back was absurdly low for a Grade 1 race, and it was obvious to the team that Del Mar's newly installed Polytrack surface was the culprit for the slow time. At the time, the computer wouldn't know anything about that.

As Siegel said, Equibase doesn't alter individual ratings, but ratings still change. At the beginning of every year, Trackmaster redoes the Inter-Track Variant for the previous year and the speed figures from that year are “recast,” so they change once, based on the new variant. The team also re-writes code to, for example, improve speed ratings for unusual distance races with small data sets.

But overall, it's human hands off.

“We view that the human touching it is very problematic because it is inconsistent,” Siegel said. “Just like the differences with baseball umpires; they don't all have the same strike zones. I understand there's some gain in (tweaking numbers) but we don't think philosophically the gain in doing that is worth the inconsistency.

“Here's what I can tell you: every single rating is wrong, 100 percent of them,” Siegel continued. “They're all wrong but on average they're right. There are too many variables to have them exactly precise especially race to race to race.”

Siegel gives the theoretical example of a tree out in the yard. If you ask one person how many leaves are on it, they'll say “I have no idea.” But if you asked them, is it more than one? Yes. Is it less than a billion? Yes. And if you kept going like that, you'd get down to one number.

“And if I had another hundred people and asked them all the same, every one of them would be wrong,” said Siegel. “But you'd be shocked when I averaged those hundred, how accurate that number will turn out to be. And that's kind of the way it is in speed ratings.”

Still, if we revisit the case we've been using, Carrick versus Robert Bruce on Arlington Million day, guess what? The two computer modeling companies, Brisnet and Equibase, came back with opposite results.

Brisnet gave Carrick, who ran a faster time in a faster-paced race at 1 1/4 miles, a 100 rating. Robert Bruce, whose race was slower with a slower pace at the same distance, received a 95. But in the Equibase ratings, Carrick earned a 103 while Robert Bruce got a much higher 122. When the two horses faced each other the following month, Carrick, who finished far back in the field, got a 100. Robert Bruce, who finished second, received a 125 Equibase rating.

Siegel said Trackmaster didn't have an Inter-Track Variant for 10 furlongs on the turf at Arlington, and Carrick's Secretariat performance came over a “good” grass course, the only “good” turf race that day at the distance. The turf changed to “firm” by the Arlington Million. It's not clear exactly how those issues might affect the computer's determinations.

“The computer would look largely at the composition of the field and then how the horses finished relative to one another to draw a reasonable statistical conclusion about the ratings for each race,” said Siegel. “They could be wrong because you're dealing with really small data sets but as it turns out, I know we only have one more race and I would say that's not a fair conclusion, but based on the next race that each of those horses ran, I would say give us a pat on the back. Both ran within three points of the (Arlington Million day) ratings their next race back.”

From the Brisnet perspective, DeRosa said he's comfortable knowing what does and doesn't go into a BRIS Speed Rating. The BRIS figure is final time-based, so the pace isn't part of the equation, while other figure makers, either a computer or a person, might draw different conclusions because other factors are incorporated.

“That's what you pay for and if you trust the figure makers' abilities, then you want that information,” he said. “It's no different than trusting someone's opinion looking at a workout or picking a horse at the sale but that's not what we do. We're definitely final time-based and we trust our math to get the best number for that race.”

Click here to read Figure Makers, Part 1, on Beyer Speed Figures

Click here to read Figure Makers, Part 2, on TimeformUS

Click here to read Figure Makers, Part 3, on Thoro-Graph and The Sheets

Twitter Twitter
Paulick Report on Instagram