Forecasting Results: Purpose And Practice

In her article in The Hindu today, Vidya Subrahmaniam is proud that it is the journalists on the field, rather than the psephologists with their sophisticated statistical tools, who could exactly feel the pulse of the U.P. electorate before the elections. She says: “Yet in recent years, political forecasting has proved a treacherous area for pollsters, and more so in the caste minefield called U.P.” While examining why it has proved to be treacherous, she casts aspersions on the pollsters suggesting that they were probably influenced by the BJP’s PR machinery. Tracing the opinion and exit poll results since the early 1990s, she claims that the BJP has the satisfaction of doing well in these polls, even though the results of the actual elections proved otherwise.
She may well be correct. But my question is why we need these forecasts at all? What is their purpose? As she put it, the journalists and the pollsters alike are intrigued by the question which they pose to themselves: which way would the vote go? She says: “Across TV channels, pollsters described U.P. as a nightmare, difficult to read, impossible to call. The pollsters were confused, not the voters.
Opinion polling is a valuable tool, whether in market research, in the social sciences or in politics. The Centre for the Study of Developing Societies has incisively analysed changing political trends and shifting voter loyalties, drawing from a database compiled during the course of successive elections.” But she did not answer the question why we need the forecasts at all – whether it is a nightmare, or an arm-chair theorizing is secondary. Whether it is a print journalist or a pollster, it makes no sense to suggest that they should be obsessed with forecasting, rather than with the issues, the voters’ concerns, and the parties’ strategies. There is no evidence to suggest that forecasts help the political parties to make mid-course correctives, to improve their actual chances at the hustings. There is no evidence to suggest that the viewers/readers believe forecasts would help them vote better. (on the contrary, forecasts are suspected to influence the voter decision, and hence there are well-reasoned demands that they should be banned/ restricted). Do the forecasts leave the viewer/reader in any sense better informed about their parties and candidates? Even here, as Vidya has pointed out the scope for misleading is immense, considering the disproportionate advantage the BJP secured in these polls all these years. A print journalist like Vidya, even if she is reasonably sure that BSP would get absolute majority, would be reluctant to say so in her reports before the elections, precisely because a contrary result would cause sufficient professional embarrassment. The HT, for instance, depending on the forecasts that a hung assembly is in the offing, carried a front page story on the possible scenarios, based on various permutations and combinations, a day before the actual results were out. In retrospect, such stories were absolutely unnecessary, and served no reader interest. Similar was a story in The Hindu, on possible options before the Governor in the event of a hung assembly based on the views of so-called legal experts. In retrospect, we could have well had no such hypothetical analyses, if the forecasts were not there – whether coming from sophisticated pollsters, or from ‘ear-to-the-ground’ print journalists making some qualified forecasts in the midst of their copies. It serves no purpose absolutely. The data the pollsters may generate would be useful, but not the forecasts, which in their own assessment, may turn out to be inaccurate, or may even be quite off the mark. In my view, even if the forecasts turn out to be correct, they would have served no purpose, from the point of view of most stake-holders, except the pollsters and the channels, and the print journalists who could have the satisfaction of saying “I told you so”. But I agree that there will be some interest in knowing why and how the psephologists failed to forecast the U.P.results – not necessarily because correct forecast would have served any good purpose, but just to satisfy our curiosity.

Update (May 16): As expected, Yogendra Yadav has written his post-mortem piece (Part I) in today’s Indian Express, wherein he clears many of the misgivings against psephologists in the wake of the U.P. results. He identifies three factors: sampling error, response bias and vote-seat distortion which led to under-estimating the BSP and over-estimating the BJP. Pollsters usually hope that the different errors they make will cancel each other; this time they reinforced each other. He says: “there is a big gap between what polls promise and are expected to deliver on the one hand and what they are capable of doing on the other. This allows us to acknowledge that the art of polling and forecasting has not seen any major methodological innovations since the path-breaking research by Prannoy Roy and the India Today-MARG team in the 1980s. Above all, this allows us to respond to Behenji’s call to the media for ‘atmachintan’. It’s time pollsters got together in some Atmachintan Workshops.” Hopefully, his Part II piece will throw more light on this issue. But I find the question what polls promise and are expected to deliver (forget what they are capable of doing) intriguing. I wish YY dwelt more on this, so that I could have cleared my own misgivings about the purpose of these polls. YY has congratulated the print journalists for capturing the hawa before the actual polls in their despatches. My answer is: “so what?” I agree there is some amount of reader-interest which such reports cater to. But most often, they give misleading picture of the hawa. The value of such reports is short-lived. While trying to capture the hawa, the journalists get obsessed with the likely outcome, and try to indulge in prophesy, which I think is unjournalistic. In this endeavour, some times, they turn out to be right, and sometimes not – the rate of failure often exceeding the rate of success. And it is not a profound prophesy about long-term implications of something which they try to explain – but something which is going to be obvious in a few weeks’ time. So, why not describe the campaign, and let the readers judge for themselves? After all, there is so much in the campaign, which needs to be reported, but goes unreported for lack of interest among the journalists. And the question what section of readers is interested in forecasts also needs to be addressed. Certainly, the BSP voters are not. Is there also a class bias in the business of forecasting?