On 21 October a firm called Wedgewood Polls posted a statement admitting that most of the polls it had published were fabricated. It described the whole thing as an "experimental project to demonstrate the vulnerabilities, lack of oversight, and overall low barrier to entry in the modern political polling industry". The admission came only after the aggregator FiftyPlusOne tried to replicate Wedgewood's Mississippi poll and could not. Decision Desk HQ and others pulled the numbers from their averages the same day.

We had eleven of them.

Where they were

The polls covered six Senate races and five governor's races between 31 July and 14 October: Mississippi, Georgia, Maine, Kansas, Ohio and Texas for the Senate; Georgia, Kansas, Iowa, Ohio and Texas for governor. All eleven reached us through one of our two poll feeds. The other, the New York Times poll database, had already removed them by the time we looked.

They did not lean consistently towards either party. In some races Wedgewood's number sat on the Democratic side of everyone else's polling, in others on the Republican side, and in a few it landed almost exactly on the average. That is roughly what you would expect from someone inventing plausible figures: the easiest way to avoid being noticed is to look like the consensus.

What removing them did

In ten of the eleven races, almost nothing. Each moved by less than a quarter of a point. Those races have between a dozen and more than forty polls each in our averages, and one fabricated survey among them is diluted by the rest.

Mississippi was different. Its Senate race has two polls in total, so one of them was half of everything we knew. Wedgewood's showed the race at four points. Once it was gone, our average moved from about seven points to about twelve, and the trailing candidate's chance fell from around 13% to around 4%. Across the whole Senate forecast the change was nine-tenths of a point.

The lesson is not that fake polls are harmless. It is that their damage depends almost entirely on how much real polling surrounds them. A fabricated survey is most dangerous exactly where there is the least to check it against, and those are often races that get little attention precisely because they are expected to be one-sided. Nobody is cross-checking Mississippi every week.

Why we removed them rather than down-weighting them

Every poll in our averages is weighted by how reliable its firm has been in the past, among other things. It would have been easy to give Wedgewood the lowest weight we have and move on.

That would be the wrong tool. A low rating assumes the numbers are real and merely noisy: a firm with a poor track record is still measuring something. A fabricated poll measures nothing. Its agreement with other polls is no evidence that it is genuine, and its disagreement is no evidence of a real shift. So the only defensible weight is zero, which is also the practice FiveThirtyEight and Silver Bulletin have followed for years with firms found to have invented data.

What changed here

Until this week, our warehouse had a column for banned pollsters and nothing that read it. Now there is a short, public list of banned firms, each with the date and a link to the evidence. Every six-hourly update sets aside every poll from a listed firm, in every race and in the national generic ballot, and the automated check that runs before anything is published fails if a single one survives.

A ban is a reviewed decision, not something a feed can switch on. That matters for a list like this: being added to it is serious, and it should take evidence rather than suspicion.

The part worth worrying about

Wedgewood's claim that it was running an experiment is hard to take at face value, given that it published the polls as real surveys for months. But the weakness it says it was exposing is real. Anyone can publish a poll. The aggregators, including this one, rely on reputation, track records and the occasional replication attempt to catch the ones that are not real, and this time it took a replication attempt.

The best protection a forecast has is the one that limited the damage here: many independent polls from many firms. Where that exists, one bad actor barely registers. Where it does not, readers should hold a single poll, from anyone, more loosely than the headline number suggests.