Opinion polls and municipal polls: error vs. reality
Two daily newspapers, Prothom Alo and Daily Star, published the results of two opinion polls on 6 January 2011. Both the newspapers published summary of the polls on the front page. Full reports were published with graphics in tabloid sized special supplements.¬† Municipal polls to elect mayors and councillors were to be held on 11 January. Naturally, two opinion polls published only five days prior to municipal polls were due to attract interests of political parties, candidates, workers, supporters and of course, voters. The municipal elections were held between 11 January and 18 January. The results have been quite different from the forecasts made by those two opinion polls. That is why concerned people are now trying to analyse and understand why those two opinion polls went wrong and what the real objectives of that exercise were.
First in Bangladesh
I would like to inform very humbly that probably weekly Jaijaidin was first to introduce political opinion poll in Bangladesh on March 18 1986. President Lt Gen H M Ershad had declared a general election will be held on 7 May 1986. In this context, Jaijaidin sent out 70,000 questionnaire forms to the countrywide members of the 700 Reader‚Äôs Rights Committees (RRC). At that time Jaijaidin was selling nearly 150,000 copies per week. Questionnaire forms were printed in two colours, white and green. Same 15 questions were asked and RRC volunteers were requested to fill in white forms and get the green forms filled up by persons unknown to them and then mail back to Jaijaidin the completed forms. The results collated from both sets of returned forms were published in Jaijaidin on 18 March 1986.¬† Regarding the two different results Jaijaidin made the following editorial comment:
‚ÄúWe are aware that no opinion poll can be fully correct. Therefore, we shall not claim that the two results of these opinion polls are fully correct. But, we would like to point out that the survey work has been carried out all over the country by the members of RRC and all kinds of adults and people belonging to different kinds of profession and vocation have taken part in the poll. However, a big limitation was that very few peasants and day-labourers took part in this poll.¬† We hope that it will be possible to overcome this weakness in future.‚ÄĚ
But four months later Jaijaidin was banned and I was sent to exile in London. As a result, it was not possible to conduct any other opinion poll in Bangladesh.
In 1986, computer technology in Bangladesh was limited. The country‚Äôs leading firm of Chartered Accountants, Rahman Rahman Huq (RRH) did not have a single computer. I was then a senior partner of that firm and I had friendly relationship with my politically conscious Articled Clerks. Some 50 of them voluntarily worked at my residence, first to sort out the returned forms in white and green and then to tabulate by hand the answers. They worked in groups in different rooms at a stretch for nearly 48 hours. They accomplished a wonderful job. They were the pioneering opinion poll workers in Bangladesh. All of them are well settled in their life now and I wish them well. I would also like to thank all the postal workers who in spite of a fear of persecution by Ershad regime, cooperated with Jaijaidin, to deliver thousands of envelopes before the dead line. We had put rubber stamps on the returnable envelopes marking: ‚ÄúOpinion Poll ‚Äď Please Deliver The Mail Soon.‚ÄĚ I would like to thank those democracy-minded unknown postal workers who responded so well. There was no courier service then.
I mention this incident, because, although people of Bangladesh are mostly homogenous in respect of language, religion, food habit, dress, etc., their life pattern differs because of huge income difference. That is why, for a correct opinion poll, a very large sample is required and it was then possible by the help of many poll takers, tabulators and postal workers. But, we could not know how correct the opinion poll was because, BNP led by Khaleda Zia boycotted that election whilst Awami League led by Sheikh Hasina after a last minute deal with Ershad, participated. However, the interesting point to note is that the question number 15 that is the last question was:
How do you think the present government (Ershad Government) will be dislodged from power?
Three alternative answers were given and the people replied:
Readers– Non Readers
Election– 15% — 20%
Revolution (Popular Uprising)– 59%– 62%
Others 26%– 18%
That opinion poll of March 1986 was proved to be correct eventually in December 1990 when General Ershad had to resign after a popular uprising.
I should also mention that Jaijaidin was first published on 8 October 1984 and it immediately requested its readers to take part in monthly opinion poll to elect the best of television programmes, artistes, newscasters, etc. They elected the best of 1984, the result of which was announced in March 1985 at a public function at Sheraton Hotel which was attended by dignitaries including Sheikh Hasina. In fact many of JJD readers who were young and students, had become familiar in opinion poll process since 1984. That is why, in spite of the absence of computers and financial constraint, we could conduct the opinion poll of March 1986.
Weekly opinion poll for Spectrum Radio
In August 1986 I was sent to exile in London. At that time UK had state-owned BBC radio and television and also privately owned television companies but not privately owned radio companies.¬† From 1987, I campaigned for licences to be given to private radio stations and joined with other persons who had been pursuing the same goal. Mrs Margaret Thatcher was the then prime minister of UK and she was MP from my area that is Finchley, London. I contacted her. Our goal was achieved on 25 June 1990 when UK‚Äôs first privately owned multi-lingual radio station, Spectrum Radio, began broadcasting from Endeavour House, Brent Cross, North London. As the founder finance director and later CEO of Spectrum Radio, I became involved in opinion poll. But, this time surveys were conducted professionally by The Joint Industry Committee for Radio Audience Research (JICRAR). Spectrum Radio was then broadcasting in more than 15 languages including, Bengali, Hindi/Urdu, Chinese, Arabic, Italian, Hebrew, etc. We wanted to find out which language and which programme was commanding how many listeners. Polls were taken every week by JICRAR.¬† Advertisement income of Spectrum depended on their report. At that time, I began to know about latest opinion poll technique and its history
It all started in US
Opinion poll began in US in 1924. In that year Hamsburg Pennsylvanian after conducting a poll locally, reported that of the two presidential candidates, Andrew Jackson was ahead compared to John Quincy Adams. According to the survey Jackson had received 335 votes and Adams 1169. Such local polls based on small samples were conducted in cities and were known as Straw Poll. In 1916 in an attempt to increase circulation, Lierary Digest conducted a nation-wide opinion poll and correctly forecast that Woodrow Wilson would be elected president. To carry out this survey, Literary Digest had mailed thousands of postcards around the country and hand counted the postcards which were retuned. Note that, some 70 years later Jaijaidin had carried out the first ever opinion poll in Bangladesh almost exactly in the same way. Also note that, prior to this, successive military rule since 1958 in this land had made people fearful in expressing their opinion publicly. That is why probably nobody had made any attempt to conduct a nationwide opinion poll.
Anyway, in US, Literary Digest had correctly predicted the winners in the next four presidential elections. In 1936, Literary Digest went wrong.¬† 2.3 million ‚Äúvoters‚ÄĚ had taken¬† part in their survey and it was a very big sample. But all these voters belonged to affluent class and were Republican Party supporters. Just one week before the election, Literary Digest reported that the Republican candidate Alf London was more popular than the Democrat Franklin Roosevelt. At the same time, George Gallup had carried out a survey in a scientific manner based on a much smaller sample but more representatives of various classes of people. Gallup had predicted Roosevelt would win by a landslide. After this, survey work by Literary Digest stopped and the paper ceased publication.
You may note that, after 75 years, Prothom Alo and Daily Star have committed the same error.
Another American, Elmo Roper, came forward to conduct opinion polls on scientific basis. Elmo Roper correctly forecast that President Roosevelt would be re-elected successively three times in 1936, 1940 and 1944. Louis Harris was another American who was also conducting opinion poll on scientific basis since 1947. Later on he joined Roper‚Äôs firm.
In the meantime George Gallup opened a subsidiary office in UK.¬† In 1945, Gallup correctly predicted that Labour Party would win the election. All others had predicted that Tory Party led by Winston Churchill, World War II hero, would win. The correct prediction by Gallup established his credibility in UK Opinion poll became a regular feature of many daily newspapers in UK.
It is also true that both in US and UK, opinion polls went wrong at some times. In 1948 it was forecast that Republican candidate Thomas Dewey would win by a big margin against Democrat Harry Truman. Both Roper and Gallup had made that forecast. But, Truman won.
Likewise, in UK, polltakers could not correctly predict that in 1970 Tory Party and in 1974 Labour Party would win. In 1992, all polltakers forecast that Labour Party led by Neil Kinnock would win. But that election was won by Tory Party led by John Major. Apart from these mishaps, generally speaking, polltakers have correctly forecast in US and UK.
After the fall of Ershad regime, I returned to Bangladesh in early 1992 and republished weekly Jaijaidin. Again, I conducted some surveys on political and non-political issues with the participation of readers. I was fully aware, such surveys were biased. So, I detached the survey work from my magazine and formed an organisation called Social Survey to conduct bias-free opinion polls. To put Social Survey on a scientific basis I went to London and met Ms Merrill James who was then the Chief Officer of Gallup International Ltd.
Ms Merrill frankly told me that Gallup International‚Äôs 90 per cent income generates from non-political surveys, such as, which is more popular, Coke or Pepsi? If Coke is more popular, then what are the reasons? Is it its taste, price, can‚Äôs design and colour or the ease of availability? Is KFC more popular than Pizza Hut? If so, why? Is it because consumers are non-vegetarian ?
Ms Merrill informed survey woks are carried out not only for such products, but also for large projects, such as, building extensions of Heathrow Airport. At that time people were objecting to expansion of Heathrow Airport which would have caused more noise pollution.¬† UK government wanted to know the opinion of the local people.¬† Bangladesh government is now planning to build a mega airport in Munshiganj. But would the government carry out an opinion poll of the local population there?
Ms Merrill said that, Gallup‚Äôs main income generates from such government and multi-national contracts. But chances of getting such contracts in Bangladesh are dim and any firm which has Gallup‚Äôs standard, cannot possibly be commercially viable in Bangladesh.
That is why I could not build a formal relationship with Gallup International. At a later stage, Social Survey stopped polltaking.
However, in mid-‚Äė90s, some newspapers began publishing the results of opinion polls. They were readers based and consequently biased.¬† Hence, they were not reliable.
But, an NGO called Democracywatch continued to conduct opinion polls since 1995. This NGO appointed some statisticians, formed a team and trained them. By following Gallup principles, they conducted some opinion polls. Particularly important opinion polls were in June 2001 (conducted in association with another NGO called PPRC) and in September 2001, i.e., just a one month before the impending general election of 1 October 2001.
September 2001 opinion poll used 5,000 samples and was based on face-to-face interviews. This opinion poll reported that 42 percent voters will cast their votes for BNP alliance. In the election, 47 percent votes were cast for BNP alliance and they were the winners. Democracywatch had forecast correctly.
It was known at that time, Awami League had carried out at least three opinion polls engaging different organisations. They had reported that Awami League would win.
The reasons why Democracywatch proved to be correct were: sample was big (5,000), it reflected proper representation of population group, easy questionnaire and trained staff. It may be mentioned here, Democracywatch submitted this report to the then caretaker government, embassies, donor agencies and to some political parties. But for unknown reasons Democracywatch did not make this report public.
After this, Democracywatch conducted another survey on election campaign coverage in electronic media. The result of this survey was published.
Meanwhile, use of computer had increased in the country and some newspapers began to publish results of online polls where readers were asked only one or two simple questions on current topics. But, they were reader biased. It is also reported that two security agencies of Bangladesh government, namely DGFI and NSI, do also secretly conduct opinion polls, the results of which are known only to government.
Opinion poll’s influence on election
Two opinion polls were published on 6 January 2001. They have been conducted by ORG-Quest Research (Prothom Alo) and AC-Nielsen (Daily Star).
Peter Hitchens, a columnist for Daily Mail, London, has written in his book, The Broken Compass, opinion polls can themselves influence opinion in three ways:
1. Opinion poll creates a bandwagon effect. Voters begin to gather behind the candidate who is shown to be leading. Many voters love to be with the winner and hence this effect.
2. Opinion poll creates an underdog effect. Some voters begin to sympathise with the possible loser and they cast their votes for him. As a result candidate trailing may at the last moment get more votes and win.
3. Opinion poll may influence voters to go for tactical voting. For example, in UK’s general election of 1997, sitting Tory minister Michael Portillo was contesting from Enfield-Southgate, London constituency. This was regarded a safe seat for him. But, opinion polls reported that Labour candidate Stephen Twig was gradually becoming more acceptable to voters. At this point, voters who were strongly against Portillo went all out to get Twig win. Twig won. Portillo, who could have been Tory Party leader, left politics and is now a columnist and TV show presenter.
Such incidents prove that opinion polls published before the election can influence opinion. This is why political parties and their supportive media publish opinion polls conducted from their perspective to serve their own cause.
Opinion polls published on 6 January 2011 by Prothom Alo and The Daily Star, just five days before the municipal elections had precisely the same goal. But they could not score. Why did they fail?
Twins, but who is who?
It is up to the readers to decide whether Alo and Star are just Twins or Twins of Evil and who is who. But for the time being, we see that in spite of their being twins, two opinion polls published by them are radically different.
Polls conducted by ORG-Quest AC-Nielsen
Awami League 46% 39%
BNP 39% 22%
JP 6% 4%
Jamaat– NA 2%
Others NA 1%
Cast No Vote NA 3%
Will not vote NA 3%
No response NA 26%
Note that Star has reported that 26 percent did not respond. It has also reported the intention of four other category of voters. But Alo has not given corresponding information and their tabulation is incomplete.
In the sub-total, Alo has shown that 46 percent voters preferred Awami League and 39 percent voters preferred BNP. On the other hand, Star has showed that 39 percent voters preferred Awami League whilst only 22 percent voters preferred BNP.
It may be assumed that both the newspapers knew this difference in voting intentions. But, both decided to report on the same day two different results. This they probably thought would act as safety valves, i.e. if one poll went wrong, the other poll would possibly be correct. In the end neither was correct. Alo had said Awami League was ahead by 7 percent and Star had said Awami League was ahead by 17 percent. Such a big difference, 10 percent, between the two newspapers belonging to the same house is strange and may not be acceptable to their readers.
Two other differences should be mentioned. Star in its supplement’s last page has published only two questions and answers with bar charts. The questions are:
1. If there is an election tomorrow, which party would you vote for?
2. Finally, I want to know, for which party did you cast your vote in last election?
I have already given the answers to the first question. Answers to the second question was:
Wasn’t a voter–1%
You will see Star’s total is wrong here. This small error should be forgiven.
Alo did not ask the second question regarding the last election. Even if they did, for unknown reasons, they have not published it.
Now, if you look at Star’s report, you will note that although 53 percent voters cast for Awami League in last election, 39 percent now intend to cast for Awami League in next election. In other words, in last two years, Awami League has lost 14 percent support.
Whilst Star has published the answers to the most important two questions, Alo has not done so. Probably, Alo did not wish to show how much support Awami League has lost. Alo has published answers to the first question (whom would you vote for in the next election?) in the main section of the newspaper in a rather non-conspicuous manner.
Alo has been very careful not to annoy the prime minister Sheikh Hasina.
But, reality could not be suppressed. Reality is, municipal elections have proved that both Alo and Star opinion polls were grossly wrong.
To compare final results with Alo-Star’s forecast, I tried to get a full list of votes cast in municipal elections. I was informed that, compilation by Election Commission of all votes cast will take some time. So, I am depending on results published by various newspapers. It shows, in seven divisions’ 236 municipal elections votes cast were 70 percent and out of this BNP candidates have secured more votes than AL candidates. According to a newspaper report, total votes cast for BNP was 1,451,337 (41 percent) and for AL it was 1,740,575 (39 percent). The declared results show that out of 236 elected mayors, BNP has 97, whilst AL has 93.
If Alo’s poll was correct then the corresponding numbers would have been, BNP 92 (236 x 39 percent) and AL 109 (236 x 46 percent) mayors.
On the other hand, if Star’s poll was correct, then the corresponding numbers would have been BNP 52 (236 x 22 percent) and AL 92 (236 x 39 percent) mayors.
From this simplified calculation, I hope the readers will understand how gross were the errors made by both Alo-Star. According to both, BNP should have been defeated by a large margin. In reality, BNP won.
Lost in violence
On 8 January 2011 Sheikh Hasina and Jatiyo Party (JP) leader General H M Ershad campaigned from the same platform in North Bengal. In 1986, Hasina had taken part in Ershad’s election. This time she went with him to North Bengal hoping, a bandwagon effect would start from North Bengal where some voters were known to be favourable to Ershad. When it did not happen and BNP won there with large majority, orders were probably passed on to AL leaders-workers to make sure AL candidates win in other five divisions, particularly in Noakhali where voters were known to be favourable to BNP. Violence followed (particularly in Luxmipur, Feni and Noakhali) and 20 municipality elections were abandoned.
If such incidents could have been avoided, BNP would have gotten more votes and more mayors. And Alo-Star polls would be in a worse situation.
However, at the moment of writing this article the results of 236 municipal elections show the following mayors:
AL Rebels– 16
BNP Rebels– 8
Nagorik Committee– 1
Who will gain from transit?
Alo’s supplement published 33 questions and answers and Star’s supplement published 61 questions and answers. Regarding transit facility to be given to India, their question was:
Q. Do you think transit facility given to India would benefit Bangladesh?
The answers were:
Yes 53% 46%
No 38% 9%
Will benefit India - 13%
Will benefit both - 3%
Don’t know 9% 29%
Total 100% 100%
Again differences in the two polls are quite big. Alo’s poll says more than half of respondents think that it will benefit Bangladesh. On the other hand, less than half of Star’s respondent feels that it will benefit Bangladesh.
Everybody knows that transit issue is very important for all Bangladeshis. So, in view of suspect polls of 6 January, Alo-Star may now consider to conduct a more reliable poll exclusively on this issue.
Margin of error
Opinion polls can, of course, be wrong. But, after the actual results are known, one should consider the margin of error in the preceding poll.
In developed countries a 3 percent margin of error in a sample of 1,000 may be acceptable.
Alo has said in its report, their published poll may have 1.79 percent margin of error. But reality has proved how wrong they were.
Star took precaution not to mention its margin of error. Well done.
Why such big errors?
Answers to this question will take a long time and I do not have any wish to act as a consultant for Alo-Star. Still, for the sake of the country’s politics, media credibility and Alo-Star’s acceptability, I shall give some reasons very briefly:
1. Weekly Jaijaidin and Democracywatch based their polls on a large sample (minimum 5,000). Their questions were fewer and simpler. People of Bangladesh are homogeneous in respect of language, food habit (rice, lentils, vegatables and fish) and religion (Muslim 88%). Otherwise, people are not so homogeneous. For most of the people, fast food like KFC and Pizza Hut is luxury, Coke-Pepsi is the ultimate sign of sincerity in guest entertainment, they have not seen beer, wine and whisky, they have never taken holidays to spend time abroad or on a sea beach, they have seen aeroplanes in the sky but have never flown, they have rode vehicles (bus) but have never dreamt of owning a car, most of them do not have power and gas connections. The difference between this class of people (who are in large majority) and those who are affluent has been increasing. In view of this, random sampling should be coupled with selective sampling and the total should be at least 10,000 to arrive at a meaningful result.
Managing Director of ORG-quest Mr Manzurul Huq has said that whilst in US 1,000 sample is regarded enough, his organisation has taken 3,000 sample and that should be enough for Bangladesh.
Some people may disagree with him. In US, people enjoy a minimum standard of living and hence a minimum homogeneity is assured there. We know, most Americans most have tasted KFC-Pizza Hut-McDonald’s, Coke-Pepsi, beer-wine-whisky (at least once possibly), they have taken annual holiday, have probably flown and probably knows how to drive and if they wish can buy a new or used car. Life without electricity – gas is unimaginable. They have this homogeneity. Moreover, even houses and flats in a particular area or street may look homogeneous where people of more or less same range of income reside.
In Bangladesh, next to buildings you will see slums. I live in Eskaton Gardens where at one end we have Holy Crescent (formerly Holy Cross) Hospital and the state guest house Padma. Then we have Ladies Club, foreign minister’s official residence, a school and first class officers’ quarters. At the other, end of the road, we have Bangladesh Navy Chief’s official residence. In between, even in front of the main gate of my residence, you will see growing number of blue polythene camps where rootless people live. Where is the homogeneity here? How would you reflect the opinion of these different kinds of people without selective samples?
2. Number of questions in Alo-Star polls was too many. Some Bengali questions were probably hard to understand. And regarding some Bengali questions, respondents may have been totally ignorant. Take for example two questions by Alo:
a. Do you think steps taken by the present government in the field of education have been successful?
How would an illiterate farmer answer to this question? He would probably reply, I don’t know. These answers would be in majority. And yet, we find in Alo’s pie chart, only 1% respondent has said, I don’t know!
b. The next question is, do you think steps taken by the present government in agriculture sector have benefited the farmers?
How this question can be answered by the D-Juice (mobile phone) generation of young people or by the city dwellers? Yet, Alo’s bar chart shows only 2% respondents replied, I don’t know.
Alo-Star should have focussed on fewer questions on politics. On other issues special polls could have taken e.g., targeting educationists, students, guardians regarding questions on education, lawyers and politicians on constitutional changes, farmers on agriculture, etc.
Both Alo and Star’s questionnaires have a degree (mixture) of diverse issues, e.g., education to agriculture, constitution to transit, employment to price rise, war crimes trial to extrajudicial killings, law and order to living standard, etc, etc, etc.
c. Both polls have set some questions to arrive at comparative pictures covering last two years. I am not sure, whether this has been correctly handled. When I was conducting weekly polls in London for Spectrum Radio, I used Rolling Sample method. Every week I would change 100 respondents out of, say, 1,000 sample. In this way a continuity of respondents was maintained which was necessary to get their comparative judgement. Was this method followed by ORG-Quest / Ac-Nielsen?
Now you can ask the question, why Alo-Star spent so much time, energy and money to publish such unreal, unbelievable and unacceptable polls?
The answer may lie in the two editor’s political ambition. Both of them propagated Minus-2 Formula.
One can say, three forces are now operating in the political arena of Bangladesh:
1. BNP and its like minded parties.
2. AL and its like minded parties.
3. One-eleven propagators which include a section of media, a section of civil society and some ambitious army officers.
Since 1991, BNP had the opportunity of forming government three times and AL had the opportunity twice. The third force, may quite naturally harbour the intention of forming a government again. If the present AL government fails, they would not like BNP to come to power. That is why this third force has continuously been telling, AL has failed and people are not happy with BNP. In support of this theory, they are concocting people’s opinion polls. That is why Prothom Alo’s headline in the poll supplement was, “People’s disappointment in politics has increased”.
Star has informed that this last poll was fourth in a series. It may be assumed Alo-Star will publish more opinion polls. But, I hope good sense will prevail upon them.
In 2007, they thought their trump card was Nobel winner Dr Yunus.
When the deciding point came, Dr Yunus refused to head an army-backed government. Instead, he recommended Dr Fakhruddin Ahmed who became the powerless head of a caretaker government which was backed by some powerful army generals. After two years, that government failed, Dr Fakhruddin went to US and the generals retired. Now who will head the next one-eleven government? Who will be the civilian head of a military backed government?
I would request the ‚Äėtwins‚Äô to consider this matter very deeply and remind them Churchill’s famous saying, “No one pretends that democracy is perfect or all-wise. Indeed it has been said that democracy is the worst form of government except all those other forms that have been tried from time to time”. (House of Commons, 11 November 1947). I hope, they will be patient and give democracy a chance to advance and flourish.
After the first phase of municipal elections ended, Star’s headline in front page (15 January 2011) was: “AL surprised at BNP’s gain”.
If we consider the poll published by Star just nine days ago, then we must say the appropriate headline should have been: “Star surprised at BNP’s gain”.
Star was not surprised.
I remember a story:
American lexicographer Noah Webster has been famous for writing dictionaries. His, A Compendium Dictionary of the English Language (1806), English Grammar (1807), American Dictionary of the English Language, 2 volumes (1828) which is now known as Webster’s New International Dictionary of the English Language are essential books for any writer in English language.
Noah Webster’s wife was beautiful and men fell in love with her.
On one afternoon, Noah Webster returned to his home unexpectedly.
He discovered his close friend was making love with is wife in the bedroom.
Noah’s friend hurriedly covered himself and Mrs Webster by bedsheet and exclaimed, “I bet, you are surprised”.
Noah, the lexicographer, replied, “No, I am not. You are surprised, I am astounded”.
Probably Star was astounded.
Shafik Rehman is a writer, editor, TV programme presenter, chartered accountant