Market research company Ipsos has recently released a range of different figures, all reported on as authoritative by the media, on party political support for the big three parties: the ANC, DA and EFF. But the truth is, half of them were guesses, and the other half were severely limited in what they were able to tell us. Nevertheless, you would be none the wiser if you followed media reports on them.
The first of these came on May 28, when Mari Harris, director of public affairs at Ipsos SA, told the Huffington Post that the ANC’s support had risen to 52%; that "the DA’s support is down to 20%"; and that the EFF was treading water, on or around the 6.3% it received in 2014.
Strangely, there was no poll accompanying this figure. On enquiry, Harris said, "I based my view on the things I have seen so far this year in other studies (non-public studies)".
The next set of numbers came on June 21, when Harris told Radio 702 , "Currently, it looks like the ANC support is somewhere between 55% and 60%"; that, "the DA’s support is between 13% and 17%; and the EFF’s support between 7% and 9%."
In an interview with Bongani Bingwa, she said those figures were for all South Africans (in other words, not exclusively registered voters) and "definitely not a prediction". But, once again, there was no accompanying poll published by Ipsos.
For the third time, the actual poll did not accompany the findings, which were reported on by a number of news outlets as definitive, on the day. Only the following day, on July 17, was a full description released along with the accompanying methodology and technical detail
Finally, on July 16, in what seems to have been an exclusive release to eNCA , Ipsos revealed the findings of its bi-annual Pulse of the People survey. It put the ANC on 60%, the DA on 13%, and the EFF on 7%.
For the third time, the actual poll did not accompany the findings, which were reported on by a number of news outlets as definitive, on the day. Only the following day, on July 17, was a full description released along with the accompanying methodology and technical detail.
So, in summary, in the six weeks since May 28, Ipsos has had the ANC on 52%, between 55% and 60%, and, finally, on 60%. It has had the DA on 20%, between 13% and 17%, and, finally, on 13%. And the EFF on 6.3%, between 7% and 9%, and, finally, on 7%.
This represents some wild swings over a six-week period — from 20% to 13% for the DA and from 52% to 60% for the ANC. And that is before you account for the margin of error. This isn’t polling, it’s prognostication.
And, for all of that, Ipsos has only one actual piece of market research to point to, which supports only its final set of numbers, made public on July 16 (more about that later).
For a supposedly reputable market research company, this is a dismal way to go about your business. In fact, it is disgraceful. For the most part, it seems Ipsos was pulling numbers out of thin air.
Here is what likely happened: we know now that The Pulse of the People survey was in the field between April 20 and June 7; while Ipsos was collating the results, it gave a series of media interviews — notably on May 28 to the Huffington Post and on June 21 to Radio 702 — in which it effectively did no more than pre-emptively guess the results; because it has the reputation of a serious polling company, and the media is so profoundly weak at analysing polls, both these sets of numbers were reported on as credible and authoritative, without any accompanying data or a single question about the methodology.
Thus, Ipsos’s guesses generated headlines such as, "DA Support Down To 20 Percent Because Of De Lille, Policy And Leadership Issues, Research Shows" (Huffington Post, May 28).
No, there was no research, just one person’s opinion.
Here are two important questions any journalist should ask before reporting on market research.
1. Can the market research company describe the methodology used (when the poll was in the field, the margin of error, the size of the sample, whether the sample was demographically representative, whether it covered rural and urban areas, whether it was a universal sample or exclusively focused on registered voters and, crucially, how many people were undecided)?
2. Can the market research company provide the raw data?
If it cannot produce those two things, what you are dealing with is a guess, not market research.
Let us look, then, at the one piece of actual market research on which, supposedly, Ipsos has based its previous guesses and which currently has the ANC on 60%, the DA on 13%, and the EFF on 7%.
Because [Ipsos] has the reputation of a serious polling company, and the media is so profoundly weak at analysing polls, both these sets of numbers were reported on as credible and authoritative
A few things are important to observe about the methodology. First, the sample — 3,738 people — was universal. In other words, it was not limited to registered voters alone. As people who are not registered will not vote, their opinion is irrelevant in determining party political support and their inclusion thus tends to skew the result.
In turn, a large proportion of the sample did not answer the question about their political support: 3% "did not know" (although significant, actually a remarkably low number in and of itself); 7% "refused to answer"; and 5% said they "would not vote". That’s 15% of your sample. On election day, only votes cast matter. People who don’t vote are irrelevant, undecided voters make a choice (there is no undecided option on an election ballot,) and those people who refuse to answer, reveal their choice.
One must factor this into your assessment of support levels. If not, then explain that the final percentage is not the complete picture. The onus for this sort of thing falls on the press. You can assign undecided voters by asking a range of other questions about their preferences, but Ipsos never does that. So, the media needs to explain this limitation.
The long and the short of it is, what Ipsos has revealed is how a portion of the general public feel about the big three parties, a finding limited by the fact it was not exclusively registered voters and a significant number of people polled (typically, this far out from an election) did not reveal their preference.
This sort of thing can have big implications. In the Western Cape, for example, Ipsos has the ANC on 26% and the DA on 28% (in 2014, the DA secured 59% and the ANC 32%). But a whopping 31% of those polled would not reveal their preference (9% "will not vote;, 17% "refused to answer"; and 5% "did not know"). This is a very high number of undecided voters, although again, not unusual for those two provinces historically, as they are two of the most fluid. But it makes the final number almost farcical. One in three people polled would not reveal their choice.
It is highly likely the vast majority of those people are inclined towards the DA, given the previous election results. No doubt the party has a support problem in the province but, even if you take that into account, factor in the undecideds and there is simply no way the DA is on 28%. If it is, it has suffered the most catastrophic collapse in support any party has experienced between two elections since 1994 — almost 30%. It would be without precedent.
The same is true of KwaZulu-Natal, where 21% of those polled would not reveal their preference. For eNCA to present Ipsos’s findings in a graph such as the one below, without including the undecideds, is just dishonest.
But no more dishonest than the prediction made on Radio 702 by an Ipsos representative, who claimed on the back of these numbers, "There is a very high possibility the ANC could take the Western Cape". That is a very silly statement to make. With 31% undecided and no way of knowing which way they will break, Ipsos is basically guessing, again, at what will happen — in a year’s time! It should know better.
But these sorts of shortcomings in the Ipsos poll all went by without comment. The eNCA story, shockingly, did not even report on the methodology. No scrutiny at all. Radio 702, in a subsequent story titled, "ANC and DA tracking close to each other in the Western Cape — Ipsos Report" (July 17) , had no less than a "political analyst" on, who did not even mention or interrogate the methodology of the poll, rather took it on face value. All of this does the public a great disservice.
Now, just to be clear, none of this is to downplay the DA’s problems. It has them to be sure, and it is highly unlikely that it is currently anywhere near the 59% it got in 2014. But to present it as on 28% in the Western Cape is just bonkers. And you can be sure it is not on 13% nationally either.
In June, Business Day reported DA internal polling had the party on 24.5%. Also in June, Financial Mail reported that the DA has set an internal target of 27% for the upcoming elections. One should not take those numbers as inherently true either (mainly because the DA does not share its methodology or data), but the DA has no reason to lie to itself and, whatever your reservations about those numbers, they cut a stark contrast with what Ipsos is reporting.
In a recent article for the Daily Maverick (Pollsters, the DA has a surprise in store for you, July 10), DA chief strategist Jonathan Moakes took issue with Ipsos polling generally, which he described as "almost consistently wrong, and their accuracy appears unrelated to how long in advance of the election their poll is taken." He said, "I am confident that we will see the continuing of the most clear trend in South African politics in the past decade — and that is the trend of the DA growing and increasingly becoming a party of government that will challenge for control of the Union Buildings."
One thing you can be sure of, if the media and Ipsos continue to play games with market research in the way they have been doing, they will look like fools come election day — not the political parties. Both Ipsos and the media have a duty to set out the limitations of their findings. Without that, polling becomes guess work, misrepresented and misunderstood. All you achieve is confusion. Over the last six weeks, both Ipsos and the media have done their fair share to achieve exactly that.
• Van Onselen is the head of politics and government at the South African Institute of Race Relations.
Would you like to comment on this article?
Sign up (it's quick and free) or sign in now.
Please read our Comment Policy before commenting.