Changing Policy Discourse in India – e Big Idea with Big Data ?

Big Data
Big Data

Everyone talks about Big Data Analytics (BDA) primarily because it  identifies uncover hidden patterns, unknown correlations, uncovered trends  and other useful business information. Thus every organization employs analytics and it is commonly believed that those who do not are unlikely to survive.  However, what is often neglected is that the biggest beneficiaries of BDA will be Governments. Especially for emerging economies like India, this is perhaps the only way to set our house in order. For too long, policy interventions in India have been rendered ineffective owing to structural reasons.  Given the heterogeneity we possess, most policy interventions needs to be targeted towards specific groups. Such specific targeting meant two things- one, the interventions have to be carried afresh every time one faces  different stakeholders and two, it heavily depends on human interventions thereby encouraging corruption through  moral hazard  BDA is set to change all that if we could grab the opportunity that lies ahead.

Let us consider the case of agriculture-the backbone of Indian economy absorbing about 50% of the labour force and contributing under 14% of GDP. This mismatch is often contributed to two major factors- overtly dependent on monsoon and the lack of access to rural credit. Effective use of BDA can address both the problems.

Traditional models of agricultural credit has failed us. Credit offtake in any society depends upon the availability of credit as well as the credit worthiness of the borrowers. Traditional models of agricultural credit  have treated agricultural loans more akin to personal loans than business loans. While assessing business loans, the decisions are based mostly on the profitability of the business and the quality of the underlying assets while personal loans are solely determined by borrower’s solvency. As a result, the business loans have always attracted lower interest rates than personal loans. Unfortunately, for agricultural loans, lot of weights are attached to the solvency of the borrower than the quality of the assets making it expensive as well as relatively rare to avail. The reason is simple to see. The main asset that a farmer possesses is the land, whose quality is difficult to measure accurately and in real time. One can however, use yields as proxies. However, there are two distinct problems banks face if they used yield as a proxy- one, the absence of any reliable information about yield and most importantly, the ability to predict the future yield. In the absence of using such proxies, most banks resort to assessing the solvency of the borrower while deciding agricultural credit. 

How does BDA solve the problem?

The strength of BDA lies in using unusual data sources and thereafter generating invaluable insights from it.  However, what transforms bid data techniques to effective policy making is how well one can combine the analytical techniques with human intelligence, experience and intuition and finally making it scale-able and implementable. In spite of agriculture in India being largely determined by weather related uncertainties, it made and still makes significant contributions because of the collective experience of the farming communities. When it comes to crop assessment, the visual assessments of an experienced farmer is invariably the best estimate. However, the major challenge is the inability to scale and leverage them all across. If one wants to assess the conditions covering wide spread farms, it would mean such experienced farmers physically verifying each and every single plot to give his or her assessment which is prohibitively expensive in terms of time, costs making it unfeasible. This is where, big data combines with human intelligence by providing the experienced farmer’s eye estimate across wide areas and that too in real time. Let us understand how.  

Most of us can visually distinguish between leaves that are healthy and those that are not by looking at their color and texture. Healthier leaves will be in general greener than the rest. Experienced eyes can further distinguish between shades of green and attribute the reasons for it. All that is needed therefore, is an algorithm that classifies crops based on its colour! BDA steps in by  using data from Geo satellite images and thereafter using artificial intelligence that replicates how we look at things. Over the last few decades the resolution of the satellite images have improved significantly with resolution as high as 0.2 meters per pixel. Thus, we may often be able to see what even the most experienced eye misses out. Once plots are classified based on colour, it uses various analytics techniques in tracking  the historical yield as well as predict the future yields with considerable degree of accuracy.  Given that the data is extracted satellite images, it is not self-reported and hence does not suffer from the usual moral hazard problems that plague most of  policy making in India. Secondly, the data is dynamic, can be updated every week implying that there is enough rigor to track any harvest cycle real time. Finally, analysis based on such data can expand the entire cultivated area-down to each square meters! The advantages of such analysis are tremendous.  The data is reliable, comes with high frequency and is gathered at negligible cost. Such analysis when properly employed not only aids the farmer in accessing cheaper credit (owing to lower risk premium) but most importantly aids him real time decision making that will increase the yield. Simply put the expertise of a seasoned farmer is now scale able and accessible real time to all! This is the next big revolution waiting to happen and hopefully India will show the way. The encouraging news is that already some Indian entities are doing that.

India is now the hub of Analytics, an industry that earns annual revenue of around 2 billion USD,  growing at over 25% CAGR and is poised to make major breakthroughs in the way we will look at policy initiatives. However, very little of the current analytics in India aids our own policy making. That should change with the Government leading the way. A separate cell to identify and encourage such BDA initiatives,  connecting with Universities and Academic Institutions who are involved in Big Data Analytics would be a natural first step towards embracing  the next big idea-Big Data Analytics!

Profile photo of Bappaditya Mukhopadhyaya and Awadhesh Kumar Jha
Bappaditya Mukhopadhyay (Director, PGPBA, Great Lakes Institute of Management) and Awdhesh Kumar Jha, (Director, Quenext)