Qualitative Vs. Quantitative Methods

9 Jun

Epistemology of Causality

How do we know that something is the ’cause’ of something and how do we impute ‘causality’ through data?

To impute causality in quantitative models, we rely on the argument that it is unlikely that the change in Y could be explained by anything else other than X since we have ‘statistically controlled for other variables’. We ‘control’ for variables via experiments or we can do it via regression equations. This allows us to isolate the effect of say variable x on y. There are of course some caveats and some assumptions that go along with using these methods but robust experimental designs still allow us to impute causality in a fairly robust way. Generally, the causal claim is buffeted with a description of a plausible causal pathway. All of the analysis and the resulting benefits of reliably imputing causality are predicated on our ability to ‘correctly’ assign numbers to ‘constructs’ (the real variables of interest).

Let’s analyze now how qualitative methods can impute causality. While it seems reasonable to assume that ‘systematic’ ‘qualitative’ analysis of a problem can provide us with a variety of causal explanations and under most circumstances provide us with a reasonably good idea of how much each of the explanatory variables affects the dependent variable, there are crucial problems and limitations that may induce bias in the analyses. Additionally, we must define what constitutes as ‘systematic’ analysis.

Another thing to keep in mind is that ethics and rigor are not enough to impute causality. What one needs are the right epistemic tools.

A lot of qualitative research is marred by the fact that it ‘selects on the dependent variable’. In other words, it sees a dependent variable and then goes sleuthing for the possible causal mechanisms. It is hard in that case to impute wider causality between variables because the relationship hasn’t been tested for varying levels of X and Y. It is useful to keep in mind that sometimes it is all that we can hope to achieve. Additional problems can emerge from things like “selection bias” and logical fallacies like “Post hoc ergo propter hoc”. Partly the way qualitative research is written can also impose its own demands and biases including demands for narrative consistency.

It is unclear to me whether a system exists to impute causality reliably using qualitative methods. There are however some techniques that qualitative methods can borrow from quantitative methods to improve any causal claims that they may be inclined to make – one is to use a representative set of variables, the other is to look for ‘natural experiments’, and pay attention to larger sociological issues and iterate through why alternative explanations don’t apply as well here – a sort of a verbal regression equation.

There are of course instances where deeper more in-depth analysis of few cases allows one to get a deeper understanding of the issue but that shouldn’t be mistaken as coming up with causes.

Epistemology of generalization in empirical methods

There is very little space that we get edgeways when we think about a systematic theory of generalization for empirical theories unless. To generalize we must either ‘know’ fundamental causal mechanisms and how they work under a variety of contextual factors or use probability sampling. Probability sampling theories are built on the belief that we know nothing about the world. Hence we need to take care to collect data (which ideally transposes to the constructs) in a way that makes it generalizable to the entire population of interest.

Causal arguments in Qualitative research

For making ‘well grounded’ causal arguments in qualitative research – say with a small n – the case must be made for generalizability of the selected cases, use deduction to articulate possible causal pathways, and then bring them together in a ‘verbal regression equation’ and analyze which of the causal pathways are important – as in likely or have a large effect size- and which are not.

Epistemic standards in interpretation and methodology

Quantitative methods share a broad repertoire of skills that is shared across the disciplines while comparatively no such common epistemic standards exist across a variety of qualitative sub-streams that differ radically in terms of what data to look at and how to interpret the data. Common epistemic standards allow for research to be challenged in a variety of ways. From Gay and Lesbian studies to Feminist Scholarship to others – there is little in common in terms of epistemic standards and how best to interpret things. What we then have is merely incommensurability. Partly, of course, different questions are being asked but even when same questions are being asked – there appears to be little consensus as to what explanation is preferred over the other. While each new way to “interpret” facts in some ways does expand our understanding of the social phenomena, given the incommensurability in epistemic standards –we cannot bring all of them to a qualitative ‘verbal regression equation’ (my term) through which we can reliably infer the size of the effect of each.

Caveat Lector
The above article deals with the debate between qualitative methods and quantitative methods on a small select sample of issues – generalizability and causality – that are explicitly more tractable through quantitative models. It would be unwise to construe larger points about the relevance of qualitative methods from the article.