Sunday, August 22, 2010

Dead Cert

The Leaving Cert results were greeted with the by-now-annual chorus of 'this isn't working/this can't go on'. It seems that everybody agrees the system needs to change... and nobody is doing anything about it. It is an uncanny reminder of the growing chorus of commentators who were warning that our credit-fuelled binge in the mid-2000s would end in tears. Everybody knew it couldn't go on... but it did until it came to a crashing halt.

And oddly enough there is a connection between the two. It struck me when reading Amar Bhidé eloquent essay on The Judgement Deficit in which he identifies the over-reliance on statistical models and algorithms - alongside the removal of human judgement and experience - for the global financial crisis. For example, he observes that:
In recent times, though, a new form of centralized control has taken root—one that is the work not of old-fashioned autocrats, committees, or rule books but of statistical models and algorithms. These mechanistic decision-making technologies have value under certain circumstances, but when misused or overused they can be every bit as dysfunctional as a Muscovite politburo...

Because natural laws and mathematical inferences cannot predict behavior, algorithms are built upon statistical models. But for all their econometric sophistication, statistical models are ultimately a simplified form of history, a terse numerical narrative of what happened in the past. (The simplifying assumptions of most statistical models are in fact so great that they can almost never be used successfully to reconstruct the very historical data used to construct the models.) They reveal broad tendencies and recurring patterns, but in a dynamic society shot through with willful and imaginative people making conscious choices, they cannot make reliable predictions.

It is common to mock Pollyannas who hope “things will be different this time,” but in a venturesome economy, things actually are different every time. Statistical models disregard the uniqueness of events, treating them like balls in a jar that vary only by diameter or color. What’s more, statistical models of human behavior tend to focus on a small number of variables. Ignoring the one-off characteristics and richness of individual situations, though, is fundamentally incongruent with what makes a decentralized economy tick. As Hayek pointed out in 1945, the inability to cope with context-specific information makes centralized organizations inflexible.
He's talking about lending decisions and risk assessments (and innovation and a lot of other things - do read the full essay) but think about it in the context of the Leaving Certificate. We have reduced the process of determining the ability of an 18 year old boy or girl to succeed in a particular course or career to an algorithm that allocates a score out of 600. Shown in the table above. And like all such simplifications it is vulnerable to error and to failure. Think Triple-A rated Mortgage Backed Securities. Sure what could possibly go wrong?

There are a few candidates for the Leaving Cert equivalent of 'sub-prime lending'. One obvious candidate is the continuing high failure rates in maths and science subjects. With the lead role of 'Fannie and Freddie' played by the teachers unions and the universities (though not all). But this is a problem that can be fixed by learning lessons from others who have faced similar problems (Finland, for example).

A problem that will be much more difficult to solve - the Leaving Cert equivalent of Anglo Irish Bank - is that of the higher failure rate among boys than girls. Yes it is another hardy perennial in the Leaving Cert news cycle (as noted by the Geary Institute blog and virtually nobody else). The consequences are already obvious in terms of the worsening gender inequality in Ireland's third level sector (the majority share of female undergraduates is steadily increasing). Not only does this bode ill for male employment prospects, it also bodes ill for their long-term well-being as graduates live longer than non-graduates. The gender death gap is about to get bigger. Still, at least it'll shrink the gender pay gap...

The growing gender education gap is a problem for which there are no easy fixes - practically or politically. There is though one obvious and very practical solution to the poorer performance of boys than girls in the Leaving Cert. Lower the CAO requirements for boys versus girls. It used to be done in the UK for the 11-Plus exam. The rationale is clear enough: boys and girls experience different rates of cognitive development. And before you object, tell me where you stand on quotas for female TDs?

But the main barriers (as they were with responding to our own and to the global financial crisis) are political ones. We live in an increasingly misandryst culture which either dismisses any disadvantages experienced by boys and men ('manority statistics') or that simply ignores gender altogether when it might focus attention on the specific problems of men (for example in relation to suicide in Ireland). Another case of men becoming 'invisible':

But ultimately we need to move away from the increasingly dangerous 'computer-model' that is the Leaving Certificate as currently structured. I'll leave the last word to Amar Bhidé:
As we rebuild from the economic crisis, we must renew the search for the appropriate balance—in finance and in other endeavors—not just between centralization and decentralization but also between case-by-case judgment and standardized rules. The right level of control is an elusive and moving target: Economic dynamism is best maintained by minimizing centralized control, but the very dynamism that individual initiative unleashes tends to increase the degree of control needed...

Ultimately, however, the “right-sizing” of judgment is itself a matter of judgment. Long experience has taught us how to strike a sensible balance in the area of centralized human control. We no longer blindly trust scientific management, time-and motion experts, or the wisdom of corporate executives and committees; but we do rely on bosses and rules to provide some order. Black-box econometric models are harder to blend with human judgment. Their dictates, which can be as rigid and stultifying as those of time-and-motion experts, are invisible and disembodied and thus more difficult to confront. Yet if we are to preserve the primacy of human judgment, we must learn to harness and control these models, not submit to them.
If we are to avoid the coming 'Credentials Crash' then the time to act is now. No doubt I will be reminding you again next year...

No comments:

Post a Comment

Related Posts Plugin for WordPress, Blogger...