From time to time we report on a new scientific study and see some of the same comments asking questions or posing suppositions about how the research was done — or how it should have been done. For people outside the research world, it can be confusing to see an expensive study done to test what seems to be a tried-and-true principle for most farm managers, or to understand why test groups are often small in equine research.
Dr. Martin Nielsen is an assistant professor and parasitology researcher at the Gluck Equine Research Center at the University of Kentucky. Nielsen has worked both as a practicing veterinarian and an academic researcher in his native Denmark and is a boarded veterinary microbiologist and parasitologist in Europe and the United States.
We posed common questions or statements from our comment section to Nielsen, who shed some light on the world of research.
“Anyone who works with horses already knows about the conclusions found here. Why study something that's treated as fact?”
Because history is full of examples of “facts” that turned out to be completely wrong. As scientists, we need to constantly challenge our current understanding and knowledge. Once we conclude something to be a fact that never has to be investigated again, we have committed a huge mistake.
Can you provide examples of previously-accepted “facts” in equine or veterinary science that turned out to be totally wrong?
From the world of parasitology, I can mention a couple:
– For many decades, the wisdom was that a good “killing frost” would take care of parasite stages on pasture. That turns out to be completely wrong, as those parasite larvae do really well in freezing temperatures.
-All the veterinary textbooks describe one equine large roundworm: Parascaris equorum. It turns out that there are two species, and P. equorum is so extremely rare that it goes completely undetected.
– For a long time, leading scientists claimed that drug resistance was very unlikely to happen in the large roundworm — for a lot of reasons. Then one day, resistance was described and now it is widespread.
“Why are sample sizes so small in horse studies?”
A sample size really only has to be large enough to evaluate the question. If the study design is good and solid, a large sample size is not necessary. A power calculation will determine the required sample size. Furthermore, we have a huge ethical responsibility when conducting animal research studies. Our ethical codex mandates us to keep the sample size as small as possible. And finally, it could be a funding question. There are very limited funding sources available for equine research and horse studies are expensive.
How can we draw any sort of useful conclusion from studies with small sample sizes?
A small sample size does not make it a bad study. Most often a study with a smaller sample size allows for better quality data (more sample points, more different types of measures and analyses) and it allows for controlling other factors that may affect the study outcome. Thus a study with a small sample size often gives more clear-cut results.
“This study was supposed to shed light on a problem in racehorses, but it was done on another breed/idle horses/horses working on a treadmill. How seriously should I take the results?”
Evaluating horses on a treadmill allows for a tighter study design where many extraneous factors can be controlled. Studying horses on the racetrack or in training can be very complicated because of the countless factors interfering with the study outcome. A treadmill study can be used to understand mechanisms/issues relevant to race horses, but it is always important to consider the study population. In other words, how similar are the horses in the study to horses in full training on the race track? And then exercise caution when interpreting the results.
Why don't researchers ask on-track trainers to allow horses to be part of a nutrition or drug study?
My area of research is parasitology, which does not have a direct relation to race track matters so I cannot speak for track researchers with certainty but I can provide my perspective here. There could be many reasons for this, but one is often that a good study design requires an untreated control group. Very few trainers and owners are willing to let researchers do that. Another reason is that it is impossible for researchers to control for other possible factors affecting the outcome (medication, training methods, other supplements offered to the horse etc.)
“So, this study found a correlation between X and Y. That means it's scientific fact, right?”
In that study, yes. But we don't know how often that same correlation is found under different circumstances. So a good scientific principle is always to reproduce your findings. Many times.
What's the difference between a peer-reviewed study, a study done by a university along with a product company, and a “study” whose results are published on a company's website?
Peer-reviewed means the study has been published in a scientific journal that has a peer-review process as part of its approval procedure. The manuscript gets sent to two anonymous experts in the given scientific field and they read it and provide a written report to the editor. The editor then decides to either reject the manuscript of request a revision addressing the comments raised by the reviewers. After one or more iterations, the manuscript will finally be published.
Now, all three examples provided above could be peer-reviewed, even something posted on a company's website. But one has to look for a scientific reference (names of authors, title of paper, name of journal with volume and page numbers). The overwhelming number of studies done at a university are published in peer-reviewed journals (regardless of funding source), but the reference should be provided.
A study claimed to be scientific but not published in a peer-reviewed journal should always be taken with a great degree of caution. Why isn't it published? There is usually a reason, since everyone (including companies) wants their papers published. Also, it is important to keep in mind that something presented at a scientific conference is not necessarily peer-reviewed.
New to the Paulick Report? Click here to sign up for our daily email newsletter to keep up on this and other stories happening in the Thoroughbred industry.
Copyright © 2018 Paulick Report.