Figure credit: Yphtach Lelkes. From joint work on selective exposure.
Media Bias is in the Eyes of the Beholder
20
Nov
Author Archives: editor
Why Were the Polls so Accurate?
16
Nov
The Quant. Interwebs have overflowed with joy since the election. Poll aggregation works. And so indeed does polling, though you won’t hear as much about it on the news, which is likely biased towards celebrity intellects than the hardworking many. But why were the polls so accurate?
One potential explanation: because they do some things badly. For instance, most fail at collecting “random samples” these days, because of a fair bit of nonresponse bias. This nonresponse bias, if correlated with the propensity to vote, may actually push up the accuracy of the vote choice means. There are a few ways to check this theory.
One way to check this hypothesis: were the results from polls using Likely Voter screens different from those not using them? If not, why not? From the Political Science literature, we know that people who vote (not just those who say they vote) do vary a bit from those who do not vote, even on things like vote choice. For instance, there is just a larger proportion of `independents’ among them.
Other kinds of evidence will be in the form of failure to match population or other benchmarks. For instance, election polls would likely fare poorly when predicting how many people voted in each state. Or tallying up Spanish language households or number of registered. Another way of saying this is that the bias will vary by what parameter we aggregate from these polling data.
So let me reframe the question: how do polls get election numbers right even when they undercount Spanish speakers? One explanation is that there is a positive correlation between selection into polling, and propensity to vote, which makes vote choice means much more reflective of what we will see come election day.
The other possible explanation to all this – post-stratification or other posthoc adjustment to numbers, or innovations in how sampling is done: matching, stratification etc. Doing so uses additional knowledge about the population and can shrink s.e.s and improve accuracy. One way to test such non-randomness: over tight confidence bounds. Many polls tend to do wonderfully on multiple uncorrelated variables, for instance, census region proportions, gender, … etc., something random samples cannot regularly produce.
Raising Money for Causes
10
Nov
Four teenagers, on the cusp of adulthood, and eminently well to do, were out on the pavement raising money for children struck with cancer. They had been out raising money for a couple of hours, and from a glance at their tin pot, I estimated that they had raised about $30 odd dollars, likely less. Assuming donation rate stays below $30/hr, or more than what they would earn if they were all working minimum wage jobs, I couldn’t help but wonder if their way of raising money for charity was rational; they could have easily raised more by donating their earnings from doing minimum wage job. Of course, these teenagers aren’t alone. Think of the people out in the cold raising money for the poor on New York pavements. My sense is that many people do not think as often about raising money by working at a “regular job”, even when it is more efficient (money/hour) (and perhaps even more pleasant). It is not clear why.
The same argument applies to those who run in marathons etc. to raise money. Preparing and running in marathon generally costs at least hundreds of dollars for an average ‘Joe’ (think about the sneakers, the personal trainers that people hire, the amount of time they `donate’ to train, which could have been spent working and donating that money to charity etc.). Ostensibly, as I conclude in an earlier piece, they must have motives beyond charity. These latter non-charitable considerations, at least at first glance, do not seem to apply to the case of teenagers, or to those raising money out in the cold in New York.
Randomly Redistricting More Efficiently
25
Sep
In a forthcoming article, Chen and Rodden estimate the effect of ‘Unintentional gerrymandering’ on number of seats that go to a particular party. To do so they pick a precinct at random, and then add (randomly chosen) adjacent precincts to it till the district is of a certain size (decided by the total number of districts one wants to create). Then they go about creating a new district in the same manner, randomly selecting a precinct bordering the first district. This goes on till all the precincts are assigned to a district. There are some additional details but they are immaterial to the point of the note. A smarter way to do the same thing would be to just create one district over and over again (starting with a randomly chosen precinct). This would reduce the computational burden (memory for storing edges, differencing shapefiles, etc.) while leaving estimates unchanged.
A Potential Source of Bias in Estimating the Impact of Televised Campaign Ads
16
Aug
Or When Treatment is Strategic, No-Intent-to-Treat Intent-to-Treat Effects can be biased
One popular strategy for estimating the impact of televised campaign ads is by exploiting ‘accidental spillover’ (see Huber and Arceneaux 2007). The identification strategy builds on the following facts: Ads on local television can only be targeted at the DMA level. DMAs sometimes span multiple states. Where DMAs span battleground and non-battleground states, ads targeted for residents of battleground states are seen by those in non-battleground states. In short, people in non-battleground states are ‘inadvertently’ exposed to the ‘treatment’. Behavior/Attitudes etc. of the residents who were inadvertently exposed are then compared to those of other (unexposed) residents in those states. The benefit of this identification strategy is that it allows television ads to be decoupled from the ground campaign and other campaign activities, such as presidential visits (though people in the spillover region are exposed to television coverage of the visits). It also decouples ad exposure etc. from strategic targeting of the people based on characteristics of the battleground DMA etc. There is evidence that content, style, the volume, etc. of television ads is ‘context aware’ – varies depending on what ‘DMA’ they run in etc. (After accounting for cost of running ads in the DMA, some variation in volume/content etc. across DMAs within states can be explained by partisan profile of the DMA, etc.)
By decoupling strategic targeting from message volume and content, we only get an estimate of the ‘treatment’ targeted dumbly. If one wants an estimate of ‘strategic treatment’, such quasi-experimental designs relying on accidental spillover may be inappropriate. How to estimate then the impact of strategically targeted televised campaign ads: first estimate how ads are targeted depending on area and people (Political interest moderates the impact of political ads [see for e.g. Ansolabehere and Iyengar 1995]) characteristics, next estimate effect of messages using the H/A strategy, and then re-weight the effect using estimates of how the ad is targeted.
One can also try to estimate the effect of ‘strategy’ by comparing adjusted treatment effect estimates in DMAs where treatment was targeted vis-a-vis (captured by regressing out other campaign activity) and where it wasn’t.
Sample This
1
Aug
What do single shot evaluations of MT (replace it with anything else) samples (vis-a-vis census figures) tell us? I am afraid very little. Inference rests upon knowledge of the data (here – respondent) generating process. Without a model of the data generating process, all such research reverts to modest tautology – sample A was closer to census figures than sample B on parameters X,Y, and Z. This kind of comparison has a limited utility: as a companion for substantive research. However, it is not particularly useful if we want to understand the characteristics of the data generating process. For even if respondent generation process is random, any one draw (here – sample) can be far from the true population parameter(s).
Even with lots of samples (respondents), we may not be able to say much if the data generation process is variable. Where there is little expectation that the data generation process will be constant, and it is hard to understand why MT respondent generation process for political surveys will be a constant one (it likely depends on the pool of respondents, which in turn perhaps depends on the economy etc., the incentives offered, the changing lure of incentives, the content of the survey, etc.), we cannot generalize. Of course one way to correct for all of that is to model this variation in the data generating process, but that will require longer observational spans, and more attention to potential sources of variation etc.
Moving Away From the Main Opposing Party
1
Jun
Two things are often stated about American politics: political elites are increasingly polarized, and that the issue positions of the masses haven’t budged much. Assuming such to be the case, one expects the average distance between where partisans place themselves and where they place the ‘in-party’ (or the ‘out-party’) to increase. However, it appears that the distance to the in-party has remained roughly constant, while the distance to the out-party has grown, in line with what one expects from the theory of ‘affective polarization’ and group-based perception. (Read More: Still Close: Perceived Ideological Distance to Own and Main Opposing Party)
By Party:
Representativeness Heuristic, Base Rates, and Bayes
23
Apr
From the Introduction of their edited volume:
Tversky and Kahneman used the following experiment for testing ‘representativeness heuristic’ –
Subjects are shown a brief personality description of several individuals, sampled at random from 100 professionals – engineers and lawyers.
Subjects are asked to assess whether the description is of an engineer or a lawyer.
In one condition, subjects are told group = 70 engineers/30 lawyers. Another the reverse = 70 lawyers/30 engineers.
Results –
Both conditions produced same mean probability judgments.
Discussion:
Tversky and Kahneman call this result a ‘sharp violation’ of Bayes Rule.
Counterpoint:
I am not sure the experiment shows any such thing. Mathematical formulation of the objection is simple and boring so an example. Imagine, there are red and black balls in an urn. Subjects are asked if the ball is black or red under two alternate descriptions of the urn composition. When people are completely sure of the color, the urn composition obviously should have no effect. Just because there is one black ball in the urn (out of say a 100), it doesn’t mean that the person will start thinking that the black ball in her hand is actually red. So on and so forth. One wants to apply Bayes by accounting for uncertainty. People are typically more certain (lots of evidence it seems – even in their edited volume) so that automatically discounts urn composition. People may not be violating Bayes Rule. They may just be feeding the formula incorrect data.
State of the Union
22
Mar
Over the past forty years, the proportion of respondents reporting at least one union member in the household has declined precipitously (Source: American National Election Studies).
Interviewer Assessed Political Information
15
Mar
In the National Election Studies (NES), interviewers have been asked to rate respondent’s level of political information: “Respondent’s general level of information about politics and public affairs seemed — Very high, Fairly high, Average, Fairly low, Very low.” John Zaller, among others, has argued that these ratings measure political knowledge reasonably well. However, there is some evidence that challenges the claim. For instance, there is considerable unexplained inter- and intra-interviewer heterogeneity in ratings – people with similar levels of knowledge (as measured via closed-ended items) are rated very differently (Levendusky and Jackman 2003 (pdf)). It also appears that mean interviewer ratings have been rising over the years, compared to the relatively flat trend observed in more traditional measures (see Delli Carpini, and Keeter 1996 and Gilens, Vavreck, and Cohen 2004, etc).
Part of the increase is explained by higher ratings of respondents with less than a college degree; ratings of respondents with BS or more have remained somewhat more flat. As a result, the difference in ratings of people with a Bachelor’s Degree or more and those with less than a college degree is decreasing over time. Correlation between interviewer ratings and other criteria like political interest are also trending downward (though the decline is less sharp). This conflicts with evidence for increasing ‘knowledge gap’ (Prior 2005).
The other notable trend is the sharp negative correlation (over .85) between intercept and slope of within-year regressions of interviewer ratings and political interest, education, etc. This sharp negative correlation hints at possible ceiling effects. And indeed there is some evidence for that.
Interviewer Measure – The measure is sometimes from the pre-election wave only, other times in the post-election wave only, and still other times in both waves. Where both pre and post measures were available, they were averaged. The correlation between pre-election and post-election rating was .69. The average post-election ratings are lower than pre-election ratings.
Comparing Datasets and Reporting Only Non-Duplicated Rows
27
Feb
The following is in response to a question on the R-Help list.
Consider two datasets:
reported <- structure(list(Product = structure(c(1L, 1L, 1L, 1L, 2L, 2L, 3L, 4L, 5L, 5L), .Label = c("Cocoa", "Coffee C", "GC", "Sugar No 11", "ZS"), class = "factor"), Price = c(2331, 2356, 2440, 2450, 204.55, 205.45, 17792, 24.81, 1273.5, 1276.25), Nbr.Lots = c(-61L, -61L, 5L, 1L, 40L, 40L, -1L, -1L, -1L, 1L)), .Names = c("Product", "Price", "Nbr.Lots"), row.names = c(1L, 2L, 3L, 4L, 6L, 7L, 5L, 10L, 8L, 9L), class = "data.frame") exportfile <- structure(list(Product = c("Cocoa", "Cocoa", "Cocoa", "Coffee C", "Coffee C", "GC", "Sugar No 11", "ZS", "ZS"), Price = c(2331, 2356, 2440, 204.55, 205.45, 17792, 24.81, 1273.5, 1276.25), Nbr.Lots = c(-61, -61, 6, 40, 40, -1, -1, -1, 1)), .Names = c("Product", "Price", "Nbr.Lots"), row.names = c(NA, 9L), class = "data.frame")Two possible solutions: A.m <- rbind(reported, exportfile) m1 <- m[duplicated(m),] res <- m[is.na(match(m$key, m1$key)),]
B.
exportfile$key <- do.call(paste, exportfile)
reported$key <- do.call(paste, reported)
a <- reported[is.na(match(reported$key, exportfile$key)),]
b <- exportfile[is.na(match(exportfile$key, reported$key)),]
res <- rbind(a, b)
Correcting for Differential Measurement Error in Experiments
14
Feb
Differential measurement error across control and treatment groups or in a within-subjects experiment, pre- and post-treatment measurement waves, can vitiate estimates of treatment effect. One reason for differential measurement error in surveys is differential motivation. For instance, if participants in the control group (pre-treatment survey) are less motivated to respond accurately than participants in the treatment group (post-treatment survey), the difference in means estimator will be a biased estimator of the treatment effect. For example, in Deliberative Polls, participants acquiesce more during the pre-treatment survey than the post-treatment survey (Weiksner, 2008). To correct for it, one may want to replace agree/disagree responses with construct specific questions (Weiksner, 2008). Perhaps a better solution would be to incentivize all (or a random subset of) responses to the pre-treatment survey. Possible incentives include – monetary rewards, adding a preface to the screens telling people how important accurate responses are to research, etc. This is the same strategy that I advocate for dealing with satisficing more generally (see here) – which translates to minimizing errors, than the more common, more suboptimal strategy of “balancing errors” by randomizing the response order.
Against Proxy Variables
23
Dec
Lacking direct measures of the theoretical variable of interest, some rely on “proxy variables.” For instance, some have used years of education as a proxy for cognitive ability. However, using “proxy variables” can be problematic for the following reasons — (1) proxy variables may not track the theoretical variable of interest very well, (2) they may track other confounding variables, outside the theoretical variable of interest. For instance, in the case of years of education as a proxy for cognitive ability, the concerns manifest themselves as follows:
- Cognitive ability causes, and is a consequence of, what courses you take, and what school you go to, in addition to, of course, years of education. GSS, for instance, contains more granular measures of education, for instance, did the respondent take a science course in college. And nearly always the variable proves significant when predicting knowledge, etc. This all is somewhat surmountable as it can be seen as measurement error.
- More problematically, years of education may tally other confounding variables – diligence, education of parents, economic strata, etc. And then education endows people with more than cognitive ability; it also causes potentially confounding variables such as civic engagement, knowledge, etc.
Conservatively we can only attribute the effect of the variable to the variable itself. That is – we only have variables we enter. If one does rely on proxy variables then one may want to address the two points mentioned above.
Education and Economic Inequality
7
Dec
People seem to believe that increasing levels of education will reduce economic inequality. However, it isn’t clear if the policy is empirically supported. Here are some potential ways increasing levels of education can impact economic inequality:
- As Grusky argues, the current high wage earners, whose high wages depend on education and lack of competition from similarly educated men and women (High Education Low Competition or HELCO) from similarly highly educated, will start earning a lower wage because of increased competition (thereby reducing inequality). This is assuming that HELCO won’t respond by trying to burnish their education credentials, etc. This is also assuming that HELCO exists as a large class. What likely exists is success attributable to networks, etc. That kind of advantage cannot be blunted by increasing education of those not in the network.
- Another possibility is that education increases the number of high paying jobs available in the economy and it raises the boats of non-HELCO more than HELCO.
- Another plausible scenario is that additional education produces only a modest effect with non-HELCO still mostly doing low paying jobs. This may due to only a modest increase in overall availability of ‘good jobs.’ Already easy access to education has meant that many a janitor and store clerk walk around with college degrees (see Why Did 17 Million Students Go to College?, and The Underemployed College Graduate).
Without an increase in ‘good jobs,’ the result of an increase in education is an increased heterogeneity in who succeeds (random draw at the extreme) but no change in the proportion of successful people. Or, increasing equality of opportunity (a commendable goal) but not reduction in economic inequality (though in a multi-generation game, it may even out). Increasing access to education also has the positive externality of producing a more educated society, another worthy goal.
How plentiful ‘good’ jobs are depends partly on how the economic activity is constructed. For instance, there may have once have been a case for only hiring one ‘super-talented person’ (say ‘superstar’) for a top-shelf job (say CEO). Now we have systems that can harness the wisdom of many. It is also plausible that that wisdom is greater than that of the superstar. It reasons then that the superstar is replaced; economic activity will be more efficient. Or else let other smart people who can contribute equally (if educated) be recompensed alternately for doing work that is ‘beneath them.’
Recoding Variables Reliably and Systematically
12
Nov
Survey datasets typically require a fair bit of repetitive recoding of variables. Reducing errors in recoding can be done by writing functions carefully (see some tips here) and automating and systematizing naming, and application of the recode function (which can be customized) –
fromlist <- c("var1", "var2", "var3", "var4", "var5")
tolist <- paste(c("var1", "var2", "var3", "var4", "var5"), "recoded", sep = "")
data[, tolist] <- sapply(data[, fromlist], function(x) car::recode(x , "recode.directions"))
Simple functions can also be directly applied to each column of a data.frame. For instance,
data[, tolist] <- !is.na(data[, fromlist])
data[, tolist] <- abs(data[, fromlist] - .5)
Measuring the Impact of Media
10
Nov
Measuring the impact of media accurately is challenging. Findings of minimal effects abound when intuition tells us that an activity that an average American engages in over forty hours a week is likely to have a larger impact. These insignificant findings have been typically attributed to the frailty of survey self-reports of media exposure, though debilitating error in dependent variables has also been noted as a culprit. Others have noted weaknesses in research design, inadequate awareness of analytic techniques that allow one to compensate for the error in measures, etc. as stumbling blocks.
Here are a few of the methods that have been used to overcome some of the problems in media research, along with some modest new proposals of my own:
- Measurement
Since measures are error-prone, one strategy has been to combine multiple measures. Multiple measures of a single latent concept can be combined using latent variable models, factor analysis, or even simple averaging. Precaution must be taken to check that errors across measures aren’t heavily correlated, for under such conditions improvements from combining multiple measures are likely to be weak or non-existent. In fact, deleterious effects are possible.Another point of worry is that measurement error can be correlated with irrelevant respondent characteristics. For instance, women guess less than men on knowledge questions. Hence responses to knowledge questions are a function of ability and propensity to guess when one doesn’t know (tallied here by gender). By conditioning on gender, we can recover better estimates of ability. Another application would be in handling satisficing.
- Measurement of exposure
Rather than use self-assessments of exposure, which have been shown to be correlated to confounding variables, one may want to track incidental consequences of exposure as a measure of exposure. For example, knowledge of words of a campaign jingle, attributes of a character in a campaign commercial, source (~channel) on which the campaign was shown, program, etc. These measures factor in attention, in addition to exposure, which is useful. Unobtrusive monitoring of consumption is, of course, likely to be even more effective. - Measurement of Impact
- Increased exposure to positive images ought to change procedural memory and implicit associations. One can use IAT or AMP to assess the effect.
- Tracking Twitter and Facebook feeds for relevant information. These measures can be calibrated to opinion poll data to get a sense of what they mean.
- Data Collection
- Data collection efforts need to reflect half-life of the effect. Recent research indicates that some of the impacts of the media may be short-lived. Short-term effects may be increasingly consequential as people increasingly have the ability to act on their impulses – be it buying something, or donating to a campaign, or finding more information about the product. Behavioral measures (e.g. website hits) corresponding to ads may thus be one way to track impact.
- Future ‘panels’ may contain solely passive monitoring of media use (both input and output) and consumption behavior.
- Estimating recipient characteristics via secondary data
- Geocoded IP addresses can be used to harvest secondary demographic data (race, income, etc.) from census
- Para-data like what browser and operating system the customer uses etc. are reasonable indicators of tech. savvy. And these data are readily harvested.
- Datasets can be merged via matching or by exploiting correlation across items and by calibrating.
Working With Modestly Large Datasets in R
2
Nov
Even modestly large (< 1 GB) datasets can quickly overwhelm modern personal computers. Working with such datasets in R can be still more frustrating because of how R uses memory. Here are a few tips on how to work with modestly large datasets in R.
Setting Memory Limits
On Windows, right click R and in the Target field set maximum vector size and memory size as follows:
"path\to\Rgui.exe" --max-vsize=4800M (Deprecated as of 2.14).
Alternately, use
utils::memory.limit(size = 4800) in .Rprofile.Type in mem.limits() to check maximum vector size
Learn more
Reading in CSVs
Either specify column classes manually or get the data type for each column by reading in the first few rows – enough so that data type can be inferred correctly – and using the class that R is using.
# Read the first 10 rows to get the classes
ads5 <- read.csv("data.csv", header = T, nrows = 10)
classes <- sapply(ads5, class)
Specifying the number of rows in the dataset (even a modestly greater number than what is there) can be useful.
read.csv("data.csv", header = T, nrows = N, colClasses = classes)
Improvements in performance are not always stupendous but given the low cost of implementation, likely worthwhile.
Selective Reading
You can selectively read columns by specifying colClasses=NULL for the columns you don't want read.
Alternately, you can rely on cut. For instance,
data <- read.table(pipe("cut -f 2,5 -d, data.csv"))
Opening Connections
Trying to directly read CSV can end in disaster. Open a connection first to reduce memory demands.
abc <- file("data.csv")
bbc <- read.csv(abc)
Using SQLDF
library(sqldf)
f <- file("data.csv")
Df <- sqldf("select * from f", dbname = tempfile(), file.format = list(header = T, row.names = F))
Problems include inability to deal with fields which have commas etc.
Using Filehash
Filehash package stores files on the hard drive. You can access the data using either with() if dealing with env variable, or directly via dbLoad() that mimics the functionality of attach. Downside: it is tremendously slow.
library(filehash)
dumpDF(read.csv("data.csv", header = T, nrows = N, colClasses = classes), dbName = "db01")
ads <- db2env(db = "db01")
Selecting Columns
Use
subset(data, select = columnList) rather than data[, columnList].Impact of Menu on Choices: Choosing What You Want Or Deciding What You Should Want
24
Sep
In Predictably Irrational, Dan Ariely discusses the clever (ex)-subscription menu of The Economist that purportedly manipulates people to subscribe to a pricier plan. In an experiment based on the menu, Ariely shows that addition of an item to the menu (that very few choose) can cause preference reversal over other items in the menu.
Let’s consider a minor variation of Ariely’s experiment. Assume there are two different menus that look as follows:
1. 400 cal, 500 cal.
2. 400 cal, 500 cal, 800 cal.
Assume that all items cost and taste the same. When given the first menu, say 20% choose the 500 calorie item. When selecting from the second menu, percent of respondents selecting the 500 calorie choice is likely to be significantly greater.
Now, why may that be? One reason may be that people do not have absolute preferences; here for a specific number of calories. And that people make judgments about what is the reasonable number of calories based on the menu. For instance, they decide that they do not want the item with the maximum calorie count. And when presented with a menu with more than two distinct calorie choices, another consideration comes to mind — they do not too little food either. More generally, they may let the options on the menu anchor for them what is ‘too much’ and what is ‘too little.’
If this is true, it can have potentially negative consequences. For instance, McDonald’s has on the menu a Bacon Angus Burger that is about 1360 calories (calories are now being displayed on McDonald’s menus courtesy Richard Thaler). It is possible that people choose higher calorie items when they see this menu option, than when they do not.
More generally, people’s reliance on the menu to discover their own preferences means that marketers can manipulate what is seen as the middle (and hence ‘reasonable’). This also translates to some degree to politics where what is considered the middle (in both social and economic policy) is sometimes exogenously shifted by the elites.
That is but one way a choice on the menu can impact preference order over other choices. Separately, sometimes a choice can prime people about how to judge other choices. For instance, in a paper exploring effect of Nader on preferences over Bush and Kerry, researchers find that “[W]hen Nader is in the choice set all voters’ choices are more sharply aligned with their spatial placements of the candidates.”
This all means, assumptions of IIA need to be rethought. Adverse conclusions about human rationality are best withheld (see Sen).
Further Reading:
1. R. Duncan Luce and Howard Raiffa. Games and Decision. John Wiley and Sons, Inc., 1957.
2. Amartya Sen. Internal consistency of choice. Econometrica, 61(3):495– -521, May 1993.
3. Amartya Sen. Is the idea of purely internal consistency of choice bizarre? In J.E.J. Altham and Ross Harrison, editors, World, Mind, and Ethics. Essays on the ethical philosophy of Bernard Williams. Cambridge University Press, 1995.
Reconceptualizing the Effect of the Deliberative Poll
6
Sep
Deliberative Poll proceeds as follows — Respondents are surveyed, provided ‘balanced’ briefing materials, randomly assigned to moderated small group discussions, allowed the opportunity to quiz experts or politicians in plenary sessions, and re-interviewed at the end. The “effect” is conceptualized as average Post–Pre across all participants.
The effect of the Deliberative Poll is contingent upon a particular random assignment to small groups. This isn’t an issue if small group composition doesn’t matter. If it does, then the counterfactual imagination of the ‘informed public’ is somewhat particularistic. Under those circumstances, one may want to come up with a distribution of what opinion change may look like if the assignment of participants to small groups was different. One can do this by estimating the impact of small group composition on the dependent variable of interest and then predicting the dependent variable of interest under simulated alternate assignments.
See also: Adjusting for covariate imbalance in experiments with SUTVA violations
The Worry About Anna (Hazare)
29
Aug
The following piece is in response to Arundhati Roy’s opinion published in The Hindu.
That Anna’s proposal for Lokpal is deeply flawed is inarguable. Whether Anna is also a bigoted RSS sympathizer, if not their agent, propelled by foreign money, as Roy would have us believe, is more in doubt. Since the debate about the latter point is rendered moot by the overwhelming support that Anna seems to enjoy, I focus on some important, though very well-tread and long understood, questions around corruption raised by Roy in her polemical screed.
Corruption is ubiquitous in India. Ration shops (considerable adulteration, the skim sold off), government employment schemes (ghost employees), admission to government schools (bribes must be paid to the principal), allocation of telecom and mining licenses (bribes paid for getting licenses for cheaper than what a fair auction would fetch), ultrasound clinics providing prenatal gender identification (bribes paid to police to keep these running) etc. are but a few examples of this widespread practice.
That corruption has serious negative consequences is also not in doubt. The poor get lower quality produce, if anything at all, as a result of corruption in ration shops. Inadequate public goods (e.g. canals) result from public’s money, and some intended beneficiaries denied the benefit, as a result of ghost employees in government employment schemes. Sex-selective abortions result from continued operation of prenatal ultrasound clinics. And a considerable loss in government revenue (which can be used to provide public goods) results from corruption in granting of licenses.
On occasion, corruption may increase the welfare of those most in need. For example, if some laws are arrayed against the poor, and if the poor can pay a nominal bribe to circumvent the law, corruption may benefit the poor. The overall impact of corruption on the poor is still likely to be heavily negative, if only because the loss to the public exchequer via the widely suspected significantly greater corruption among the rich is expected to be far greater. There also exists some empirical evidence to support the claim that corruption causes poverty (Gupta et al., 2002). However, an argument can be made to not enforce anti-corruption laws in some spheres, if successful attempts to amend the law that warrants circumvention can’t be mounted.
In all, the case for reducing corruption is strong. However, schemes of solving corruption by creating a bureaucracy to go after the corrupt may be upended by bureaucrats going rogue. Stories of the almost limitless power of a ‘Vigilance Commissioner’ to harass and extort are almost legend.
“Who shall mind the minders?” is one of the central questions in institutional design. The traditional solution to the problem has been to institute a system of checks and balances to supplement accountability via “free and fair” elections (which themselves need a functioning institutional framework). The system only works within limits, through innovative institutional designs to solve the problem can be thought off. The only other fruitful direction for reducing corruption has been to increase transparency (via RTI, post-facto disclosures of all bids in an auction, etc.), and via increased automation (cutting out the middlemen, keeping bids blind from the committee so as to prevent certain kinds of collusion, etc.) — something the government is slowly and unevenly (depending on vested interests) working towards.
Corruption in enforcement is harder to tackle. Agents sent to enforce pollution laws have been known to extort from factory owners by threatening them with falsely implicating them with deliberately adulterated samples. There automating testing, and scrambling identity of the source during analysis, may prove useful.
Bibliography:
Gupta, Sanjeev, Hamid Davoodi and Rosa Alonso-Terme. 2002. Does corruption affect income inequality and poverty? Economics of Governance. 3: 23–45