Close-up of a bay horse's head and neck in three-quarter view, standing calmly in soft stable shade.

Why data integrity matters more than volume

Most data providers talk about size. Fewer talk about whether the rows can be trusted.

If you build models, run a production pipeline or power a tipping product, you already know the problem. A duplicate row quietly skews a feature. A missing trainer ID breaks a join. Two horses with the same name merge into one career. None of these errors announces itself. They sit in the data until a result looks odd, and then you spend days tracing it back.

This article explains how HorseRaceDatabase (HRDB) approaches horse racing data integrity. It covers what we check, when we check it and what that means for the way you build on top of our data.

We do not publish tips or predictions. Our job is to give you a source you do not have to second-guess.

What one racing day looks like in the data

Take one recent day. On that day, the UK and Ireland card ran to 47 races and 536 runners across six courses: Chepstow, Dundalk, Kempton, Musselburgh, Nottingham and Wexford. The race types include chase, hurdle, flat turf and flat all-weather.

In Hong Kong, the card was 9 races and 108 runners at Happy Valley.

That was 644 runner entries on a single day, across two jurisdictions and several surfaces. Every one of them has to become a clean row with a horse, a jockey and a trainer attached to it.

Now scale that up. HRDB holds 2.2M result records across 226K races and 173K horses, covering more than 85 racecourses. Hong Kong history runs from 1979. UK and Ireland history runs from 2011. At that size, manual spot checks are not a strategy. The checks have to be automated and they have to run every time.

The four integrity gates

Every nightly update passes automated integrity gates before it is published. There are four.

Gate 1: no horse assigned to two races on the same day

A horse cannot run at Kempton in the afternoon and at Dundalk in the evening. On a day with six UK and Irish meetings, that sounds obvious. In raw source data it is not always obvious, because entries, declarations and results can be recorded in different places and at different times.

If a horse appears twice on one date, something has gone wrong. It may be a mis-assigned identity, a stale entry or a data entry error. The gate catches it before the update reaches you.

Why it matters to you:

  • Rest-days and days-since-last-run features depend on one run per horse per day.
  • Form and workload calculations quietly double-count if a horse appears twice.
  • Backtests assume a runner exists in one place at one time.

Gate 2: no duplicate runner rows

A runner should appear once in a race. Not twice, not three times with slightly different fields.

Duplicates are among the most common and most damaging problems in racing datasets. They inflate field sizes. They distort strike-rate style aggregates for jockeys and trainers. They also make joins unpredictable, because one key can now match two rows.

Our gate checks that each runner row is unique before publication. When you load a race, the number of rows is the number of runners.

Why it matters to you:

  • Field size is a real feature, so it has to be right.
  • Aggregations by jockey, trainer or course stay honest.
  • Your ingestion code does not need defensive de-duplication logic.

Gate 3: no missing horse, jockey or trainer IDs

A result row with no horse ID is an orphan. A row with no jockey or trainer ID cannot be attributed to the people involved.

Stable identifiers are what turn a pile of results into a relational dataset. If one in a few thousand rows has a null key, every join you write needs an exception, and every dashboard needs a footnote.

Our gate checks that horse, jockey and trainer IDs are present on every row before the update goes out.

Why it matters to you:

  • Foreign keys behave like foreign keys.
  • Trainer and jockey dashboards do not drop runs silently.
  • Schema constraints in your own database do not fail on load.

Gate 4: separation of different horses that share a name

Horse names are not unique. Different horses can share a name, sometimes in different countries and different eras. If a dataset keys on name, their records merge into one impossible career.

HRDB separates them. Each horse has its own stable identifier, and the name is a label rather than a key. The gate checks that horses sharing a name are kept apart.

Why it matters to you:

  • Career histories belong to one animal.
  • Model features built on past performance are not contaminated by a namesake.
  • You can join on ID and never on a name string.

Why the gates run before publication

The order matters. Checks run on the nightly update, before the data is published, not after a customer reports a problem.

That is a deliberate design choice. For a model builder, a bad row discovered after training is expensive. For an analytics team with a production pipeline, a bad row discovered after a dashboard has refreshed is worse. Catching issues first means the data you pull is the data that has already passed the checks.

Racecards and results are updated daily. The data is updated after racing, so it is not a live or real-time feed, and there are no webhooks. That is by design. A daily, post-racing update gives the integrity gates a clear window to do their work before anything reaches you.

What stable IDs give your pipeline

The gates and the identifiers work together. The gates protect the rules. The IDs let you rely on them.

With stable identifiers for horses, jockeys, trainers and courses, you can:

  • Build an ingestion pipeline that upserts on ID rather than matching strings.
  • Store a local copy and refresh it daily without remapping keys.
  • Join races, racecards and results with confidence.
  • Combine datasets over time without rebuilding your reference tables.

The public API documentation lists 26 UK and 30 Hong Kong data endpoints. They cover races, racecards, records (results), horses, jockeys, trainers and courses, plus Hong Kong veterinary records. UK and Hong Kong data live in separate, isolated schemas, so the two jurisdictions do not bleed into each other.

That separation is itself an integrity feature. A Hong Kong runner cannot be mistaken for a UK one, and each schema can be reasoned about on its own terms.

What this does and does not claim

It is worth being precise.

What we say:

  • Every nightly update passes the four automated gates before it is published.
  • Identifiers are stable, so you can key on them.
  • Data is updated daily, after racing.

What we do not say:

  • We do not say the data is live or real-time.
  • We do not say any dataset is a substitute for your own validation. Good teams validate their inputs, and we expect you to.
  • We do not say the data predicts anything. It describes what happened.

The gates reduce a specific set of structural errors. They are not a promise that nothing will ever need correcting. They are a promise that these four classes of problem are checked every night, every time.

How to check this yourself

You should not have to take our word for it. If you are evaluating HRDB, here is a practical test plan:

  1. Pull a recent date of results and count runner rows per race. Compare with the declared field.
  2. Group by horse ID and date. Confirm no horse appears more than once on a day.
  3. Scan for nulls in the horse, jockey and trainer ID columns.
  4. Pick a horse name you know is shared by more than one animal and confirm the IDs differ.
  5. Repeat the checks across a few days, including a busy mixed card with several courses.

The same approach works for the historical datasets in CSV, SQL and JSON. Run your own integrity tests on a sample before you commit.

Access is straightforward. You exchange an API key for a token valid for 24 hours using POST /auth/token. The GET /user/usage call reports quota used and remaining, and it does not consume a call, so you can monitor usage while you test. API access is provisioned automatically after purchase, with usage quotas per plan and add-on packs for extra calls.

Trust is built from boring checks

Data integrity is not glamorous. It is a set of unremarkable rules applied every night: one horse, one race per day; one runner, one row; every row with its IDs; every horse kept distinct.

Those rules are what let a model builder train without worrying about hidden duplicates. They are what let an analytics team ship a dashboard without a footnote. And they are what let a product team publish daily with confidence in the underlying records.

If you are comparing providers, ask each one the same question: what checks run before the data reaches me? We have answered it above. The plans page shows which option fits your volume.

Next step: Explore the datasets

Categories:

Tags:

Comments are closed

0
    0
    Your Cart
    Your cart is emptyReturn to Shop