Sign in. It’s quick, free and it’s up to you.
An account is an optional way to support the work we do. Find out more.
OPINION POLLS ARE one of the main talking points of the recent local and European Parliament elections. Tánaiste Micheál Martin and Jack Chambers, Fianna Fáil Director of Elections, publicly criticised pollsters using online panels.
Fianna Fáil has good reason to be frustrated. General election polls have significantly underestimated support for Fianna Fáil and vastly exaggerated support for Sinn Féin. For them, this creates a mistaken media narrative about popular support for Sinn Féin.
But does this underestimation of support for FF in general election polls mean that we should completely ignore opinion polls? Do they still have value for politicians, journalists and voters? How can the media, politicians and voters engage more constructively with public opinion polls? The starting point is to recognise that general election polls do not measure support for parties in local elections. Irish general election polls have always been a poor predictor of local elections, but are good predictors of general election results.
One criticism raised by politicians and commentators after these elections relates to the huge difference between support in current opinion polls for the upcoming general election and the local election results.
Let’s untangle local and general election polls. Are opinion polls for general elections accurate? This is the first question I will address.
The Irish Polling Indicator project, which Tom Louwerse (Associate Professor, Leiden University) and I started 10 years ago, aggregates all opinion polls into a single estimate of public support.
We have collected all opinion polls published since the mid-1980s, resulting in a sample of 717 surveys.
This tedious exercise involved going through newspaper archives, checking various websites and contacting academics. The resulting dataset not only allows us to present aggregated estimates but also helps us understand how well polls predict elections.
To assess the accuracy of polls, I extracted the Polling Indicator estimate on the day of the election and compared the figures to the actual first preference vote shares. Next, I calculate the absolute mean error, which is the difference between the first-preference vote share at the election and the aggregated estimate. If a party obtained 25 per cent of first-preference votes, but the poll of polls estimated support at 21 per cent, the absolute error for this party amounts to 4 percentage points. This analysis is performed for all parties to calculate the average absolute difference.
The graph below reveals that the absolute mean error results to be just 1.32 percentage points across the eight general elections between 1989 and 2020. In other words, on average, the poll of polls is just 1.3 points off the election result. This is very accurate.
From a political science perspective, it is also very encouraging: most estimates fall within the margin of error. There are, of course, exceptions to the rule. In 2002, the polls overestimated support for Fianna Fáil by 4.6 percentage points but underestimated the party’s support by 4.4 points in 2007 and 3.4 points in 2016.
In 2020, the aggregated polls overestimated support for Fianna Fáil by 2.4 points. In 2016, the polls overestimated support for Sinn Féin and Fine Gael by around 3 points each. Yet, overall, polls have done a pretty good job in predicting party support in Irish general elections over the past 25 years.
What about Local Elections? Pollsters received a lot of criticism for underestimating Fianna Fáil support and vastly overestimating the Sinn Féin vote last week. Yet, opinion polls do not intend to predict local election results.
Looking at the second figure here supports this argument. The absolute mean differences range from 2.46 (2004) to 3.77 (1991) points. Last weekend, the mean difference was 3.18 points. At almost every election, the estimates for at least one party are completely different to the outcome.
Why are the polls so far off? Irish local elections are unique. Incumbency plays a huge role in Local Elections. Our research shows that even incumbents who got elected on the last count face a huge advantage in rerunning and getting re-elected compared to the candidate who lost on the last count. Canvassing, name recognition and community ties are highly relevant. Door-to-door campaigning is a unique feature of Irish politics and it is a crucial condition for getting elected.
An Ireland Thinks poll fielded on the day of the local and European Parliament elections mirrors these claims. 75 per cent of Fianna Fáil voters cast their first preference for a candidate rather than the party the candidate represented. 66 per cent of Fine Gael voters ranked the party over the candidate.
The focus on candidates is lower for smaller parties – possibly because they ran fewer incumbents and their candidates were simply not as well known.
Fianna Fáil and Fine Gael also canvassed very successfully. 35 per cent of respondents said they met a Fianna Fáil candidate at their doorstep. 34 per cent of respondents had direct contact with a Fine Gael candidate. Only 18 per cent met a Sinn Féin representative. This might be part of the explanation why Sinn Féin performed even more poorly than anticipated: they are not only losing support in opinion polls, but they had few incumbents and their candidates were simply not as active on the ground as Fianna Fáil and Fine Gael. To sum up, incumbency and local ties are crucial, and Fianna Fáil and Fine Gael outperformed Sinn Féin in this hugely important aspect of campaigning.
We need to keep in mind that the electorate is becoming more volatile. According to Exit Polls from the past 25 years, around half of the voters make up their minds about the first preference vote in the last month before an election. Since the financial crash, around four in 10 voters have switched between parties from one general election to the next. Campaigns and canvassing are crucial in general elections and can reshuffle the Irish political landscape.
The debate between the efficacy of online and face-to-face polls continues to be contentious. While some, like Fianna Fáil’s Micheál Martin and Jack Chambers, argue that online polls tend to underestimate their support, face-to-face polls also present challenges, such as low response rates and social desirability biases. Red C Research, despite being criticised at the moment, accurately predicted Fianna Fáil’s performance in the 2020 general election.
Ideally, we rely on a mix of methodological approaches to better understand Irish party politics. Telephone polls, face-to-face surveys, and online surveys all have advantages and shortcomings. Michael A. Bailey, Professor at Georgetown University, describes these issues in an excellent recent book “Polling at a Crossroads”. Having an open mind towards public opinion will ultimately improve our understanding of public opinion.
What can we learn from this debate on opinion polls and how can we move forward the debate? I have four recommendations:
1. Media outlets should adjust their reporting. We must consider structural differences between polling companies, focus on trends rather than individual polls and not overplay small “changes” lying within the margin of error as “bounces”, “rebounds”, and “recoveries”. Sampling variation drives most of these “changes”, which have usually been within the margin of error of ±3 percentage points. Political scientists Erik Gahner Larsen and Zoltán Fazekas show that Danish news reports frequently interpret deviations within the margin of error as substantive changes. The authors conclude that “a horserace coverage of polls about change can rest on a foundation of stability.” Irish newspapers often apply the same rhetoric. A move away from horserace coverage is necessary.
2. In addition to changing the language, polling companies and newspapers could present party support as within windows (i.e., the 95 confidence interval). Reporting single numbers projects a false sense of precision. Other countries are slowly adopting this approach. For example, Italian media reported the results from the exit poll for the European Parliament elections in intervals. Rather than focusing on a single number – and suggesting a wrong sense of certainty – their headline finding was that “Meloni’s FdI to get 26-30% in EU elections – exit poll Centre-left Democratic Party to come second with 21-25%”. Irish media should follow suit.
3. We should focus more on trends. For example, the polling trends clearly showed the loss of support for Sinn Féin in general election polls – across all companies. While pollsters deviate systematically in their survey mode and baseline results, these developments are unambiguous. Voters should approach poll results critically, considering the broader context and acknowledging uncertainty.
4. We need to keep in mind that polls provide a single snapshot and that using polls to predict elections is extremely difficult. Paul Perry, back then Vice Chairman of Gallup Polling, focused on these problems in a Public Opinion Quarterly paper published 45 years ago:
All one has to do is use a properly drawn sample of the electorate large enough to minimise random sampling error, get honest answers from everyone, do the questioning close enough to the time of voting to minimise changes in voting intentions, anticipate how the undecided will vote, and, finally, distinguish between voters and non-voters in the electorate – (Perry, 1979: page 312).
Pollsters should need to be even more transparent in their reporting methods. While most outlets now list the poll’s margin of error and time when the survey was in the field, the exact approach of selecting households for face-to-face polls, selecting respondents from the online panel, non-response rates, and the weighting of respondents is still a black box. This is understandable because polling is highly complex and relies on cutting-edge statistical methods. Media outlets could add a few sentences about the sample selection and weighting.
Finally, it is important to note that polls are extremely useful for understanding voter preferences and behaviour. Journalists and politicians need polls to understand the electorate. I believe the current debate on the usefulness of polls is constructive and ought to lead to important reflections among all actors involved. I hope that mentioning methods and uncertainty will increase trust in public opinion. Ultimately, these changes would help us to focus on policy issues and the solutions that parties offer.
Stefan Müller is Assistant Professor in the School of Politics and International Relations at University College Dublin. He maintains the Irish Polling Indicator and the Irish Politics Data project.
To embed this post, copy the code below on your site
SF were rejected in favour of Ind because SF remained in favour of migration. As some other poster says, they improved on their performance from the last local elections. Both FG and FF didn’t improve while the Greens had a disaster.
That being said, these were local elections. And people vote differently in locals, national and European elections.
Be interesting to see what happens in the by-elections.
@AnthonyK:
I have my doubts that those by-elections will ever happen
@AnthonyK: then why did the “mass migration open borders” parties like FFG and even SF get most seats while anti-immigrant parties got 2 seats, a mandate of 0.2%? Racism was rejected. And all parties support rules based regulated migration.
@9QRixo8H: No, all parties do not support rules based regulated migration.
There is no regulation on immigration from the EU.
Nor do FF, FG, The Greens, Labour, SF, the SDs,… propose any controls on this immigration.
@9QRixo8H: Could it be because FFG councillors, on a local level, work with and for the people… Unlike TDs… Or so it seems.
@ItWasLikeThatWhenIGotHere: we have regulation on immigration and emigration. Have you been to an airport? What happens? And immigration act, permit to stays, visas, aka regulations. Now what we don’t have is complete closed border for both immigration AND emigration which is what the other authoritarian lot want
But laughing at SF never gets old lol
@John D Doe: FF won less seats than they did in the last local elections. FG won less seats than they did in the last local elections. SF won MORE seats than they did in the last local elections. While they performed below expectations, they’re the only major party to improve on their previous outing.
While you’re on about laughing.
@John Ennis: Shinners didn’t capitalise on FF and FG slump, the public rejected them, they choose independents instead, that is why we must laugh at them lol
@John Ennis: So, essentially, it’s a victory for SF in your opinion.
The best way to gauge the accuracy of opinion polls is by having elections. We just did. Opinion polls had SF at 30% and sometimes higher. They had FF FG languishing in the high teens or low 20s, and everyone thought SF would romp home with a landside.
The reality, of course, is different. The government parties combined attracted just shy of 50% of the vote while SF received 12% even a 10% movement from FFG to SF, which is highly unlikely, in a GE would in all probability re-elect this government .
Much has been made on this site of who some members of the Government got elected 5th count etc, SFs seats were mostly won after long counts and if they get 2 MEPs none will have reached a quota.
Some victory.
@John Ennis: SF put 335 candidates forward for locals. They only got 100 elected. Do you think a rise of 2.3% is an improvement after 5 years of supposed SF momentum and poll topping? 5 years of telling anyone anything to get a vote. I appreciate SF are trying to wrestle back control of the narrative, but after 5 very difficult years for government, this is nothing but another disaster for SF.
Who is going to employ Mick Wallace’s children now as assistants in the E.P.? Genuinely curious to know.
@Numinous20111: Who cares.
Did he ever repay the VAT or the money stolen from his employees pension fund?
Himself and Clare are now just ordinary citizens which is still too much of an honour in my view.
@Paul O’Mahoney: It is almost laughable they give people like you a vote in this country. You may dislike Mick and Clare (and it seems more personal than political) but you do realise most have voted for the status quo policies in Europe (the same backside) with the people they’ve elected to represent them. Please stfu when it comes to more of the same, you have nothing to complain about, at least Mick and Clare put a rocket up the EU and like you are glad they’re gone.
@Marcus Maher – Triskellion Films: It’s a rare thing to read such gibberish nowadays, thanks for the giggle and for proof that small minds still exist.
@Marcus Maher – Triskellion Films: Ah now, as bad as our politicians are, and some are very bad, those 2 were especially poor representatives. They were contrarians of the highest order. And that’s before we even get to the nepotism.
@Paul O’Mahoney: I was never a fan of Wallace and Daly but to state that they don’t deserve to be ordinary citizens is taking it too far in my view.
@James Leahy: Well that’s your view they did nothing for this country during the time they were in Europe other than make silly statements that did not have the backing of the majority.
They essentially were disrespectful of the country and the country has responded, a basic tenet of citizenship is to respect the country and defend it .
Can you point out any examples where they did this?
@Numinous20111: did Wallace lose his seat?
@Paul O’Mahoney: at least they put themselves forward for election and I respect that even though I don’t agree with their policies.
@Paul O’Mahoney: Problem for inept people, devoid of critical thinking skills is they reach for ad hominem attacks rather than substantive replies. What part of my post about Mick or Clare was off point Paul? Gibberish, small minds, they certainly exist, just spend 5 minutes in front of a mirror Paul and you’ll see what is staring back at you.
@Marcus Maher – Triskellion Films: all of your posts are gibberish lol
@Marcus Maher – Triskellion Films: Dude you were the one who began a post with ad hominen attacks . I’m certainly not point out where you think your wrong, that would simply deny the pleasure of a bit of self awareness.
I look in the mirror everyday and walk away happy.
As for critical thinking, just because your mourning the result doesn’t require me to do anything.
And there is only one “l” in Triskelion films.
What is O Riordain on about, combining the centre left to tackle the far right. If these local elections showed anything, it’s that there is no far right, or so few as to not even matter. The far right is a law and order problem, not a political one.
@ecrowley ecrowley: Probably the highlight of the Election him not being heard anymore.
It’s nonsense Labour tried this with DL and were decimated in subsequent elections both parties can get together in government if that ever happens….
@ecrowley ecrowley: the far right got over 5% of the vote in Ireland South. If the various factions can organise together and get a charismatic leader, then they may become relevant
@ecrowley ecrowley: Ó Ríordáin is right: we need a proper centre-left force in this country, to combat the far right but also for its own sake. SF has been trying/pretending to fill that role but has now been exposed.
It would be wrong to be complacent about the far-right threat. We are not immune.
@Brendan O’Brien: I’ve done various questionnaires to see where I fall on the political spectrum and they all have me slightly to the left. These “centre left” are evidently further away from the centre. I hesitate to label them far left but they’re also incorrect to label themselves centre left.
As for 5% voting for far right candidates.. 1 in 20 when immigration is such a huge issue? Come off it. To paraphrase one of their apes, “They’ll do nuthin”.
@ecrowley ecrowley: One of the worst far-right people is sitting on 28,000 votes in Ireland South.
@Brendan O’Brien: One candidate…Bravo for him.
@ecrowley ecrowley: If The Journal comment section was a constituency then the we’d have a far right Taoiseach
@ecrowley ecrowley: in addition to the lad on 28k, there’s another on 14k. Organisation and leadership is what they need
@Brendan O’Brien: Labour showed itself to be anti-democratic and authoritarian in that water charges fiasco.
The SDs would do well to steer well clear of them, lest they become contaminated too.
@SYaxJ2Ts: Absolutely correct, and if I learned anything from the locals, it’s that the smaller the minority the louder they are. If the far right were going to make inroads, this was the time it was going to happen. They failed utterly. I don’t have the nationwide figures but in the scheme of things their overall vote was miniscule. As any sensible person would have suspected. In short, I don’t think we need to be worrying about a Reich. These head-the-balls are more likely to be arrested than elected.
@SYaxJ2Ts: Hardly.
The were a few blowhards shouting loudly.
(Perhaps just a couple of accounts.)
They’ve mostly gone very quiet.
@ItWasLikeThatWhenIGotHere: Past performance is no guarantee of future results. The water charges debacle is no longer relevant.
@ecrowley ecrowley: more likely to be arrested than elected, I like that! And I hope you’re right, but I’m not so sure. Migration in general is going to increase significantly and I don’t see how Ireland can remain an outlier in Europe in relation to far right politics
@ecrowley ecrowley: O’Riordan gets himself a nice little earner in Europe because he knows he is going to be kicked out at the next General Election.
@SYaxJ2Ts: The main parties are going to have to get their act together in relation to immigration. In fairness, Harris seems to be looking to address it. Like I’ve said elsewhere, he can spout all he wants but FG have consistently not fulfilled promises but I’m hoping he starts to act. If he does get proper immigration policies in place, the far right will vanish politically almost overnight. Fingers crossed anyway.
@Nodon: That is correct.
Door to door canvassing is the key.Face to face meetings will register with voters and FFG are very good at it. Mind you most of them have a neck like a jockeys bol@lix.
@Willie Marty: no fg or ff came to my door!!
@Joanne Stokes: they are so clever they knew there was no vote there.
@Joanne Stokes: The only party to come to my door was SF.
Thrice.
People didn’t show up to vote how it can be accurate.
@Clau O’Riordan: If people weren’t bothered to vote, then their opinion doesn’t count, literally and figuratively.
Elections 2024?? Should that not read Election #1 2024??
Methinks there could be more in 2024 with by-elections or more than likely a general election! Harris will hit when the Shins and Cabbage Heads are down!!
@Ollie Fitzpatrick: There are arguments both ways.
And here we have another pollster missing the points entirely.
1) The electorate for the Local and EU elections is very different than that for the General (Dail) Election.
2) The turnout for these tends also to be smaller, further skewing the electorate breakdown.
3) Polling only tries to capture 1st preference voting intentions. The Irish system is such that generally does not reflect the final allocation of seats. FG will probably get 2 EU seats this time around, while SF could get the same, and maybe another. Despite the EU elections being more national than any GE constituency, or LE constituency. Until pollsters being to ask the simple question of the 2nd preference intentions, then the polls will rarely be accurate in the final allocation of seats, except by chance.
You see, what this pollster is saying is that the 1st preference vote is not *that* far off what the polls indicated.
While ignoring the difference between 1st preference votes and the final number of seats won. Which is what polling *should* be trying to capture.
But the author is quite correct on one point – the polling presented to us is very poor, and cannot be used as any indicator of local elections in particular.
@ItWasLikeThatWhenIGotHere: Agree with much of that , but the 1st preference is of huge importance and every election outcome will be dependent on it, there’s little point in a candidate getting 1000 first preference and 5000 2nd preference they would probably be eliminated.
Additionally, the larger parties manage their vote, so preferences are passed onto other party candidates.
2nd 3rd preferences are important if the candidate survives the first count.
@Paul O’Mahoney: Well, you cannot get a 2nd preference unless you get a 1st.
So the number of 1st preferences is still needed.
But unlike FPTP systems where 1st and only preferences dictate the outcome, in our system transfers dictate most outcomes.
After the 1st count we have a very good idea who will be elected to the first few seats.
But the later seats will be determined in large part by the 2nd preferences of those closer to the top.
Knowing the likely distribution of Peadar Toibin’s vote when he is eliminated would provide a lot of information, as an example.
Those are the bottom of the pile are not really that important, but knowing the 2nd preferences of that cluster with significant vote counts is.
There’ll be no heavy reliance on the supplementary register at the general election…
So of course it’ll be a different story compared to the local and europeans, take the referendum earlier in the year… They thought they had that in the bag but ultimately it fell flat on it face for them because they couldn’t dip the supplementary register pool to get them over the line.
have your say