Category Archives: Book Reviews

Noise: A Flaw in Book Writing

10 Jul
Posted on Author editor

This is a review of Noise, A Flaw in Human Judgment by Kahneman, Sibony, and Sunstein.

The phrase “noise in decision making” brings to mind “random” error. Scientists, however, shy away from random error. Science is mostly about systematic error, except, perhaps, quantum physics. So Kahneman et al. conceive of noise as seemingly random error that is a result of unmeasured biases. For instance, research suggests that heat causes bad mood. And bad mood may, in turn, cause people to judge more harshly. If this were to hold, the variability in judging stemming from the weather can end up being interpreted as noise. But, as is clear, there is no “random” error, merely bias. Kahneman et al. make a hash of this point. Early on, they give the conventional formula of total expected error as the sum of bias and variance (they don’t further decompose variance into irreducible error and ‘random’ error) with the aim of talking about the two separately, and naturally, never succeed in doing that.

The conceptual issues ought not detract us from the important point of the book. It is useful to think about human judgment systems as mathematical functions. We should expect the same inputs to map to the same output. It turns out that it isn’t even remotely true in most human decision-making systems. Take insurance underwriting, for instance. Given the same data (realistic but made-up information about cases), the median percentage difference between quotes between any pair of underwriters is an eye-watering 55% (which means that for half of the cases, it is worse than 55%), about five times as large as expected by the executives. There are a few interesting points that flow from this data. First, if you are a customer, your optimal strategy is to get multiple quotes. Second, what explains ignorance about the disagreement? There could be a few reasons. First, when people come across a quote from another underwriter, they may ‘anchor’ their estimate on the number they see, reducing the gap between the number and the counterfactual. Second, colleagues plausibly read to agree—less effort and optimizing for collegiality, asking, “Could this make sense?”, than read to evaluate, “Does this make sense?” (see my notes for a fuller set of potential explanations.)

Data from asylum reviews is yet starker. “A study of cases that were randomly allotted to different judges found that one judge admitted 5% of applicants, while another admitted 88%.” (Paper.)

Variability can stem from only two things. It could be that the data doesn’t allow for a unique judgment (irreducible error). (But even here, the final judgment should reflect the uncertainty in the data.) Or that at least one person is ‘wrong’ (has a different answer than others). Among other things, this can stem from:

  1. variation in skill, e.g., how to assess patent applications
  2. variation in effort, e.g., some people put more effort than others
  3. agency and preferences, e.g., I am a conservative judge, and I can deny an asylum application because I have the power to do so
  4. biases like using irrelevant information, e.g., weather, hypoglycemia, etc.

(Note: a lack of variability doesn’t mean we are on to the right answer.)

The list of proposed solutions is extensive—from selecting better judges to the wisdom of the crowds to using models to training people better to more elaborate schemes like dividing the decision task and asking people to make relative than absolute judgments. The evidence backing the solutions is not always hefty, which meshes with the ideolog-like approach to evidence present everywhere in the book. When I did a small audit of the citations, three things stood out (the overarching theme is adherence to the “No Congenial Result Scrutinized or Left Uncited Act”):

  1. Extremely small n studies cited without qualification. Software engineers.
    Quote from the book: “when the same software developers were asked on two separate days to estimate the completion time for the same task, the hours they projected differed by 71%, on average.”
    The underlying paper: “In this paper, we report from an experiment where seven experienced software professionals estimated the same sixty software development tasks over a period of three months. Six of the sixty tasks were estimated twice.
  2. Extremely small n studies cited without qualification. Israeli Judges.
    Hypoglycemia and judgment: “Our data consist of 1,112 judicial rulings, collected over 50 d in a 10-mo period, by eight Jewish-Israeli judges (two females) who preside over two different parole boards that serve four major prisons in Israel.”
  3. Surprising but likely unreplicable results. “When calories are on the left, consumers receive that information first and evidently think “a lot of calories!” or “not so many calories!” before they see the item. Their initial positive or negative reaction greatly affects their choices. By contrast, when people see the food item first, they apparently think “delicious!” or “not so great!” before they see the calorie label. Here again, their initial reaction greatly affects their choices. This hypothesis is supported by the authors’ finding that for Hebrew speakers, who read right to left, the calorie label has a significantly larger impact..” (Paper.)
    “We show that if the effect sizes in Dallas et al. (2019) are representative of the populations, a replication of the six studies (with the same sample sizes) has a probability of only 0.014 of producing uniformly significant outcomes.” (Paper.)
  4. Citations to HBR. Citations to think pieces in Harvard Business Review (10 citations in total based on a keyword search) and books like ‘Work Rules!’ for a fair many claims.

Here are my notes for the book.

Expertise as a Service

3 Mar
Posted on Author editor

The best thing you can say about Prediction Machines, a new book by a trio of economists, is that it is not barren. Most of the green patches you see are about the obvious: the big gain from ML is our ability to predict better, and better predictions will change some businesses. For instance, Amazon will be able to move from shopping-and-then-shipping to shipping-and-then-shopping—you return what you don’t want—if it can forecast what its customers want well enough. Or, airport lounges will see reduced business if we can more accurately predict the time it takes to reach the airport.

Aside from the obvious, the book has some untended shrubs. The most promising of them is that supervised algorithms can have human judgment as a label. We have long known about the point. For instance, self-driving cars use human decisions as labels—we learn braking, steering, speed as a function of road conditions. But what if we could use expert human judgment as a label for other complex cognitive tasks? There is already software that exploits that point. Grammarly, for instance, uses editorial judgments to give advice about grammar and style. But there are so many other places where we could exploit this. You could use it to build educational tools that give guidance on better ways of doing something in real-time. You could also use it to reduce the need for experts.

p.s. The point about exploiting the intellectual property of experts deserves more attention.

Sidhwa’s Lahore, A Lovingly Embroidered Family Heirloom

21 May
Posted on Author editor

Every great city deserves a worthy admirer. Lahore has just found one. Bapsi Sidhwa’s edited volume is a tribute to the city, a celebration of its landmarks, its cuisine, its gourmets, its brutalizing summers, its people, and its stories.

The book strikes an immediate rapport. It is akin to being invited to a Punjabi family gathering. Reading it, I felt, alternately, like a kid sitting on the lap of his maternal uncle and being told stories about the city, a young adult guiltily listening to adult conversations about brutal episodes from the city’s history, and an objective adult reflecting on the city’s history and politics.

There is a warm intimacy that suffuses each of the stories in City of Sin and Splendour: Writings on Lahore. The emotional immediacy comes from familiarity with subjects and surroundings. And from the naturalistic storytelling. Authors rarely go beyond what is known. It is an important talent. For authors are often tempted by superfluous cleverness. Here, they practice the Jane Austen method of writing — they write honestly, perspicaciously, and often with great wit about what is known, without flirting with the unnecessary or the arcane. It is grounded writing. The authors use words that are well worn and apt, not those with peripatetic grandiloquent pretensions. The resulting atmosphere is educated and homely.

I have never been to Lahore. Yet the city stands alive in front of me. Though I don’t eat meat, I savor the morning Nihari with Irfan Hussein. I share the pain of partition with Ved Mehta and Sadat Manto. I celebrate the indomitable spirit of Ismat Chugtai. I stand ringside as Bina Shah describes the long-standing tussle between Karachi and Lahore. And I wear my heart on the sleeve when I read Urvashi Butalia’s Ranamama. (Butalia’s phrase, “cracked pistachio green walls” perfectly describes the color of the walls of some subcontinent homes.) I admire the honest revolutionary spirit of Habib Jalib’s Dastoor. How did he know the story of Pakistan before it was ever written?

Third World
Many of the big cities in South Asia are shabby and poor and slung in unending mediocrity. The heat is often brutalizing and the atmosphere, dusty and arid. Trees and grass struggle to take root in face of hot summers, scarce resources, and petty corruption. Globalization, self-serving politicians, immigration, sprawl, and poverty presses from all sides. Yet the cities thrive in crevices, in neighborhoods and families, in visits to each other’s houses, in stories exchanged, in chai, and love. People exchange stories with their doodhwallahs (milkmen) and their kaamwaalis (maids). Everything is held together by talking. It is these relations, these conversations, the unsaid courtesies, that Sidhwa celebrates in her book.

Colonial Rule
The British Raj left its mark on Lahore. Kim’s gun haunts the hollow haunches of the emaciated old city. The gardens and separate civil line quarters for the English are a vital part of the city’s social topography. But more importantly, the Raj has scarred Lahore psychologically. Chastened by West that races ahead, and surrounded by pockmarked skeletons of pre-English architecture, Lahoris are unsure of what to make of their heritage.

Delhi and Lahore
Delhi is seen as Lahore’s twin. The cities have similar climates, both are (or, used to be) Punjabi dominated, have similar histories, similar old-new city Raj-inspired distinctions, and similar heartaches of partition. One can easily find flavors of Delhi in the book—the ‘gates’ of the old city, the civil lines area, the colonial bungalows, the partition stories, and the oncoming McDonald’s culture. In getting to know Lahore, you learn about Delhi.

Contemporary Conditions and History
He whose light shines only in palaces
Who seeks only to please the few
Who moves in the shadow of compromise
Such a debased tradition, such a dark dawn
I do not know, I will not own

Dastoor, Hajib Jalib
Lahore has suffered from the vicissitudes of the people in Islamabad and Washington. The onslaught of globalization and technology, unleashed without prior thought, continues unabated. People try to craft their lives around one technology while being led by their noses to the next. It is unsettling when you stop and take stock of all that will be lost to time.

The Elite Lahore
The remembrances of a city and the love of a city only come naturally to those with time for leisure. To that extent, this book is about the padshahs of Lahore. The book is an ode of the ruling class to itself, to its culture, and to its landmarks. Yet, often, the book is much more than that. The everyday street is never far in this book. The everyday street may not have the kaamwaali in it, but it does have the patang baaz, the halwais, the rickshaw wallahs, and more. It is that everyday street that I carry in my heart.

Movie Review: The Namesake

9 May
Posted on Author editor

The Film

The Namesake is a mediocre film based on an equally middling eponymous novel by Jhumpa Lahiri, the Pulitzer Prize-winning London born author of Indian descent. It is a coming of age story of an ABCD by “another badly confused Deshi” (ABCD – Lahiri) [Washington Post]

The novel traces the story of Gogol Ganguli, son of first-generation Indian immigrants – Ashoke and Ashima – presented in the movie as cardboard characters, whose one-dimensional struggles superfluously adorn the movie –and his struggle to come to terms with his cross-cultural identity. Gogol goes through various expected phases of someone shooing away a psychological ghost – unexpressed anger, rebelliousness, and then rapprochement that comes at the behest of his father’s unexpected death and later through his wife’s infidelity. While the issues are real, they seem to have been frozen and then perfunctorily staked over by an inane screenplay by Nair’s usual collaborator – her Harvard peer Sooni Taporevala. It appears that by trying to cram in too much – a bi-generational story – it fails to do justice to any of the stories.

Samosas, Rasogullas, and Indian Relatives

Nair captures the perversities of an immigrant’s life with great humor and a great eye for detail. We get to sit in the endless uncle-aunty parties full of Bengali food and watch as our little ABCDs squirm when talked to by the way ‘uncool’ uncle and aunties. We get to see how the American raised children take in the soot-laden, chaotic Indian cities and the clinging relatives on their visits to India. Of course, the Indian relatives themselves remain caricatures of humans.

Gogol wants his overcoat back

Gogol’s overcoat has been done a disservice. Much like the name of Virginia Woolf was expropriated by the mediocre and unrelated eponymous play, “Who is afraid of Virgina Woolf?”, Lahiri leans on the exoticness of Gogol to rescue her. Lahiri doesn’t have the intellectual depth to even throw in a line about why Russian authors were popular in India. Gogol’s deeply ironical and existentialist short story Overcoat becomes a peg on which Lahiri tries to hang ‘the namesake’, Gogol Ganguli’s pretentious superfluous problems.

Visual Metaphor and Nair

The Atlantic Ocean shimmers exhibiting a grey luminescence; the humid chaos of Calcutta streets is viscerally alive; and the forlorn winter landscape of New York, marked by decay, stoically real. Mira Nair is a master auteur. She has an astute eye for capturing the elemental affective truth of a place. Nair is also edacious. While she has a wonderful aesthetic eye, she uses it with the indulgence of a nouveau aesthete. Nair unhesitatingly and unfailingly puts her camera in front of every scar, every photogenic shot, and includes it.

Editing: Weaving a tapestry with unusual neighbors

The movie has been edited in a way that provides for abrupt transitions between different environments. It appears to be a deliberate strategy to highlight the often times almost schizophrenic existence of an immigrant in multiple environments, and continuation and disruption that characters feel as they straddle (or travel between) different microcosms.