EdTech's Biggest Challenge

I've just had a pretty unnerving but invigorating experience reading The Case Against Education by Bryan Caplan, a professor of economics at George Mason University. It's a book which espouses a bold and controversial point of view, presenting its argument forcefully based on a thorough review of the literature on education and the reams of published data therein. When I read the introduction, I thought Caplan was nuts (I sort of still think this, but now in a good way...), and I came close to jotting down a big list of points I would like to rebut and ridicule in a post after I finished reading it.

As you can tell from the title, Caplan is (deeply) sceptical about the value of almost all of the education in our society, at least in the developed West. The book's subtitle is: "Why the education system is a big waste of time and money". Although he focuses mostly on laying out the current state of affairs and presenting a theory of education, he does occassionally give his own best opinion on how things ought to be instead. On this, he's pretty bold: Caplan argues for full separation of state and education and believes that government funding of education should be dramatically reduced, fees increased, subsidies removed and loans charged at market rates. It's pretty heretical stuff in a society which routinely lauds education as the key to unlocking everything else we care about: social mobility, economic growth, technological progress and ultimately happy and fulfilled lives. When I wrote an earlier post about curiosity, I was firmly in the camp of buying into these standard ideas.

Anyway, the punchline is that by the end of the book I was pretty thoroughly convinced by the main thrust of the arguments (notwithstanding some of the ultra-libertarian 'total separation of education and state' stuff). So I can no longer write about what a load of old tosh Caplan's book is, because it is not. It is certainly challenging stuff, especially for those of us who have always firmly believed in the power of education and its intrinsic value. Instead, I want to summarise some of the key arguments here, and then linger on what it all means for one of the big poster children of socially responsible technological innovation and investment: EdTech. It might already be slightly old news, but with the hindsight of Caplan's analysis, EdTech's efforts in the last few years to disrupt higher education and burst the "education bubble" seem like they were always doomed to failure. But beyond that, it seems that the main thesis of the book provides a powerful lens to assess the merits of any putative disruptor in the space.

Human Capital vs. Signaling

The first half of the book is spent establishing a thesis about what exactly causes education to be so highly valued by the job market and the students themselves. It is well known that a sizeable education premium exists in the job market - by some estimates it's 70% for college grads over non-grads, and 30% for high-school graduates over drop-outs. Caplan spends time covering the copious data to prove this point, though he rigourously factors in many benefits and costs which are blithely neglected by most researchers, and which lead to noticeably lower headline figures.

Caplan makes no secret that his exaplanation for the education premium phenomenon is frequently treated as a fringe theory, especially among other economists (despite the fact that it's a theory which won its inventor a Nobel Prize and seems to fit the facts perfectly). The prevailing wisdom on education is almost universally assumed to be true. It's the idea we all implictly have in mind when extolling its benefits: education builds human capital, producing economically more productive workers who therefore contribute to an enlarged pie for the whole of society and in so doing, earn themselves a commensurately chunkier slice of pie without making anyone else worse off in the process. It's the classic win-win-win of economic growth and prosperity.

The alternative perspective - which Caplan spends four full chapters carefully developing - is that a very significant portion of the explanation for the education premium comes from signaling. Signaling is an economic theory introduced by Michael Spence in a 1973 paper called Job Market Signaling (though applications have been far broader than the job market). The idea is that one party can credibly convey information about itself to another party through various manipulable signals, thus giving the counterparty an enhanced ability to make decisions (such as whom to hire) under conditions of uncertainty (e.g. the difficulty of knowing which candidate will be best in a role ahead of time). Spence was awarded the Nobel Memorial Prize in Economics, along with Akerlof and Stiglitz, for work on analysing markets with asymmetric information. The Nobel committee wrote that "an important example is education as a signal of high individual productivity in the labor market. It is not necessary for education to have intrinsic value."

What does education signal exactly? The three key elements are: 1) intelligence, 2) work ethic, and 3) conformity. The final one is easily missed but quite important - it demonstrates workers' grasp of and submission to social expectations. Caplan elucidates on the importance of demonstrating all three: "the road to academic success and the road to job success are paved with the same materials. An intelligent worker learns quickly and deeply. A conscientious worker labors until the job’s done right. A conformist worker obeys superiors and cooperates with teammates." Later, he points out that:

Heterodox signals of your strengths automatically suggest offsetting weaknesses. Suppose you scored well on the SAT but never went to college. Employers will readily believe you’re smart. But if you’re so smart, why didn’t you go to college? As long as your conscientiousness and conformity were in the normal range, finishing college would have been a snap. Once employers see your SATs, they naturally infer you’re below average in conscientiousness and conformity. The higher your scores, the more suspicious your missing diploma becomes.

It's hard to transmit the full force of Caplan's arguments in favour of the signaling theory here - I can only strongly recommend reading the book. To summarise though, I've copied below the table from pages 118/9, which cover the eight core arguments for signaling. Underneath, I've given a brief overview of each. It's important to note that Caplan doesn't try to claim that Pure Signaling is true, but rather a blend of the two theories is at work in the real world. He thinks the split is 80% signaling, 20% human capital.

Table: Signaling in Sum

IssueWhat Pure Human Capital SaysWhat Pure Signaling SaysAdvantage?
Learning-Earning ConnectionOnly job-relevant learning pays.Irrelevant learning pays too, as long as it's correlated with productivity.Signaling
Collegiate ExclusionColleges prevent unofficial attendance so students actually pay tuition.Colleges ignore unofficial attendance because the market doesn't reward it anyway.Signaling
Failing vs. ForgettingEmployers reward workers only for coursework they still know.Employers also reward workers for coursework they used to know.Signaling
Easy 'A's, Cancelled Classes, and CheatingStudents care about only marketable skills, not graduation requirements or grades.Students care about only graduation requirements and grades, not marketable skills.Signaling
Sheepskin EffectGraduation years won't be especially lucrative.Graduation years may be especially lucrative.Signaling
MalemploymentDegrees required to get a job depend solely on skills required to do a job.Degrees required to get a job rise when those degrees become more common.Signaling
Employer LearningEmployers instantly discover and reward true worker productivity.Employers never discover or reward true worker productivity.Signaling
Personal vs. National ReturnsEducation equally enriches individuals and nations.Education enriches individuals but not nations.Signaling

Learning-Earning Connection

If Pure Human Capital is true, it should only really be lucrative to study subjects which directly contribute to job-relevant skills. In fact, the data (and plenty of anecdotal evidence) suggests that subjects which have barely any direct usefulness on the job also contribute a significant education premium over not earning a degree at all. Most people have known viscerally, since childhood and high school geometry lessons, that the actual content of most education is nearly useless for on-the-job skills or even life in general.

Collegiate Exclusion

While studying for my own degree, I realised that lectures in every faculty were completely unpoliced. There was no hindrance whatsoever to a person unaffiliated with the university but with a passionate love for classical history, say, to simply stroll into a lecture on the subject and partake of some of the best education in the world, no questions asked. (David Mitchell sends up this state of affairs in this memorable episode of the Peep Show.) It is literally true that no one would stop you; in fact the lecturer would probably have been happy to welcome an enthusiast who actually wanted to be there. Caplan points out that this is commonplace at elite institutions around the world, but clearly almost nobody actually takes this course of action - the only people who actually show up are the people who fought through the rigourous selection procedures to formally enroll on the courses, and who stand to earn a lifetime signal. You don't get that special piece of paper if you just show up for the free lectures, even though your human capital might have been enhanced just as much.

Imagine this: you can grant a job interview candidate one of the following wishes - which do you expect most will prefer? 1) A Harvard education with no degree certificate, or 2) a degree certificate from Harvard without having had the education itself?

Failing vs. Forgetting

If you believe the human capital story, it should be problematic for an employee to forget the material they spent years learning at school and university. Under this view, employers are supposed to be rewarding workers for skills alone. They may as well have failed those educational endeavours at the time for all the difference it would make to their skills and knowledge today. Failing and forgetting are identical from a human capital perspective. Under the signaling view, the difference is clear - you undertake the educational endeavours to earn the signals, and the actual content is less important (both to you, and the employer). I think it's clear that we see a lot more of the latter in the real world. How many people freely acknowledge that they can't remember any of the detailed content of their school and university courses? "After the final exam, I'll never have to think about this stupid subject again".

Easy 'A's, Cancelled Classes, and Cheating

If Pure Human Capital were true, students would want nothing but the most marketable skills. They would seek out toughest assignments they could handle to build up the rarest skills, and would be upset to hear of a cancelled class - a missed opportunity to learn. There would never be any point trying to cheat on an exam. In the real world, we see students fight for admission to elite institutions but then hunt down those professors and classes which are most likely to yield easy A-grades. Students are delighted to hear that a class has been cancelled. Some students cheat. All are because the signal is valued higher more than the skills.

Sheepskin Effect

Apparently, diplomas used to be printed on sheepskin. The sheepskin effect is the observed tendency for holders of degrees to earn disproportionately more than people with a similar amount of education but no diploma in hand. For instance, if you study a four-year degree and drop out after three years, you won't get three-quarters of the earnings premium your friend gets who actually reaches graduation. You'll get somewhat less. Signaling is the obvious explanation here, while Human Capital can't account for it.

Malemployment

Malemployment happens when people have too much education for their job - Baristas with astronomy degrees, for instance. Few people could fail to have noticed credential inflation over the last several decades. It's routine to hear members of the boomer generation describing their poor school results and lack of a university degree, followed by vocational, on-the-job training and a decades-long career progression to well-paid, high-powered jobs. These are the same people who, today, would know that they have no choice but to ace every school exam, secure a place at a top university, earn a bachelors degree, possibly continue to a master's and even eventually an MBA. The root cause of this discrepancy is not that the boomers of 40 years ago were less intelligent or that they need fewer skills in the workforce. They were much the same people and they used much the same skills, but they didn't live in an environment where the educational signaling mechanism had reached the proportions it has today. Caplan puts it thus:

malemployment reflects workers' never-ending struggle to outshine each other. Picture the labor market as an arms race. Rising education automatically sparks credential inflation; as credentials proliferate, you must study harder and longer to convince employers to hire you. In an everyone-has-a-B.A. dystopia, an aspiring janitor might need a master's in Janitorial Studies to land a job scubbing toilets. When a B.A. bartender asks, 'Why oh why can't I get a better job?' signaling ruefully answers, 'Because too many competing workers have even more impressive credentials than you do.'

It struck me that this whole situation is reminiscent of the famous Darwinian 'sexual selection' arms race which produces the beautiful-but-pointless-and-costly plumage displays of peacocks and many tropical birds. Signaling arms races explain all.

Employer Learning

One common argument to discredit the signaling theory relates to the fact that, over time, employers learn more about the 'real you'. If this occurs quickly, then bending the truth at hiring time through your signaling would be pointless. Caplan reviews the research on employer learning but demonstrates a few compelling arguments which suggest that such learning is slow, and even when it does occur, it doesn't necessarily hinder an entrenched employee. For one thing, managers rarely like firing people due to the deleterious effect on general morale. And once a subpar employee is finally ejected (perhaps by 'dehiring' rather than outright firing), the old employer will often help the employee move on to an alternative company by providing helpful references or even outright assistance finding a new role. So the signaling nevertheless turned out to be helpful. "The cycle of disappointment, mercy, and deception is reborn."

Personal vs. National Returns

One of the most numerically compelling arguments for the signaling thesis addresses the discrepancy between value-added by education at the individual level and at the national level. The bottom line here is that at an individual level, the education premium works out somewhere in the range of 8-12% for a typical year of education, while the same thing measured at a national level is only 1-3%. Although these numbers are murky and hard to measure with accuracy, these rough ranges seem far enough apart to provide some pretty compelling evidence that the human capital theory of education is far from the whole story. If education does nothing but build human capital, the personal premium should roughly match the national premium. In reality, it seems like extra education doesn't make the overall pie bigger. Instead, earning a diploma gives the individual a way to signal his place higher up the rank-ordering of candidates in the job market. This earns him a bigger slice of a fixed pie - making someone else's slice smaller. So in one sense (selfishly) education worked for this person. At the societal level, it's not doing much.

What about Education as 'Soulcraft'

So perhaps you're now convinced by the signaling concept. I was, at this point. But for most of the book, I was worried that Caplan was missing a critical point about education. Perhaps the real problem is that we need better education rather than less? I kept thinking that all this talk of economic costs and benefits, of incomes and premia and so on misses the blatant fact that education is simply an end in itself - one of the few things which could be described as an end in itself. When people speculate about robot-powered futures where humans have 100% leisure time and no worries about income or the production of resources, I pretty much reflexively assume that a life of "high-culture", learning, reading - in short, education - would be the glorious outcome, welcomed by all.

Perhaps one of the biggest effects reading this book had on me was the sense that it's okay to drop these high-minded and lofty ideals about education. We don't need to achieve a world in which every person is exposed to the joys of poetry and abstract mathematics and actually ends up seriously valuing these things throughout life. Caplan is very clear about his own feelings here. He stresses several times how highly he values education and learning in his own life, but reaches a tougher conclusion more broadly:

I don’t hate education. Rather I love education too much to accept our Orwellian substitute. What’s Orwellian about the status quo? Most fundamentally, the idea of compulsory enlightenment. Educators routinely defend compulsion on the ground that few students want to explore ideas and culture. They’re right about the students’ tastes but forget a deeper truth: intrinsically valuable education requires eager students. Mandatory study of ideas and culture spoils the journey.

I think the hard truth for some education enthusiasts is that high culture just isn't everyone's cup of tea and, crucially, that that is neither a problem nor somehow patronising or elitist to acknowledge. For a great many people, education is about building up concretely useful skills like literacy and numeracy. Many people want to head out into real world vocations and do real stuff and earn real money. They're not especially drawn towards poetry or beautiful geometry theorems. Like any interest or hobby, this sort of high culture appeals to some more than others, and there's little use or value in trying so hard to force it upon those who really find no particular pleasure in it. Particularly when the effort is demonstrably wasted and comes at taxpayers expense.

The challenging thing about this type of thinking is that it can start to feel perilously close to some form of patronising elitism. As a culture, we do seem to have sheltered ourselves in a comforting belief that "anyone can do anything, if given the right opportunities", but the harder truth is that not everyone can be above average. Only one person can be the smartest person on Earth, and only 25% of people can be in the top 25% by intelligence. We should be structuring our society with a sensitivity to this truth. It's fine to admit that being fulfilled in life is quite possible without classical history and Nobel-worthy literature. And once we are willing to accept that, it starts to seem like a huge amount of the societal effort on education is pretty misguided. Think back to the points made earlier about forgetting coursework and earning easy A-grades. What's the point of it all?

Lest you think that Caplan is all doom and gloom with no suggestions or prescriptions for the future, Chapter 8 of the book addresses what solutions he sees. The headline is that we need more vocational education - but I won't expand further on the details of Caplan's arguments.

Where does this leave EdTech?

Now that I've ripped off most of Caplan's hard work trying to convince you of the signaling theory, it's time to return to the original point: what does this all mean for EdTech?

Firstly, I think the signaling theory suggests that in many cases, EdTech might not be the ESG / impact investing panacea we might initially hope. Although education clearly has large measurable benefits at the individual level, perhaps as much as 80% of this seems to really just be coming from enhancing that individual's signaling power in the job market. The overall pie does not increase commensurately - it just gets redistributed. In the worst interpretation, excessive funds flowing into education worsen the extant problems by heightening the credentials arms race, rather than truly expanding opportunity or building human capital in a way that enlarges the pie for all.

To be totally clear, this argument does not say that the investments in these companies don't have meaningful returns potential. As at the individual level, it's quite possible to extract value from this signaling-governed system, but looking at education as a socially impactful investment opportuntity might be misguided if most of the returns are really generated by enhancing your customers' ability to signal to the job market, without actually growing the overall pie for the whole job market.

Furthermore, the signaling theory should warn us that many business models might actually be fundamentally quite flawed. Those businesses which have sought to disrupt higher education (think Udacity, Coursera, Udemy et al.) have already famously had to pivot because of the overwhelming challenge of competing with the signaling power of prestigious higher education institutions. Anecdotally, courses from these institutions garner little respect from employers - likely because, if anything, they signal the opposite of what a candidate wants to signal.

With the benefit of hindsight, the failure of MOOCs (Massive Open Online Courses) to disrupt higher education probably should have been easily predictable, looking at them with the signaling lens. Signaling tells us that more efficient, more easily accessible human capital enhancement has never really been the thing that's actually in demand from higher education institutions. The signal itself is (most of) the product. Leveraging technology to democratise course content and access to the best understanding of that content doesn't disrupt very much after all, because the course content was never the real product anyway. Student A just needs to prove she's better than Student B. They can't both be better than each other, and if they both get smarter or more highly skilled together, that also doesn't matter much to the job market. Caplan's point about conformity is important here, too: a student who tries to go a different route (say by completing cheap online courses instead of college) automatically looks heterodox and non-conformist. It may be difficult for any individual company to effect a society-wide shift in perspective to value, say, vocational education over traditional degrees if the whole system is stuck in a stable equilibrium.

On the other hand, there will no doubt be plenty of opportunities which survive the signaling test. Probably the most valuable EdTech investments which are also truly impactful are those businesses which seek to teach absolutely concrete, technical job skills that are prerequisites for highly in-demand roles. The obvious examples here are companies which rigourously teach coding skills - such as online courses or bootcamps. Software engineering jobs are highly technical and demand continues to outstrip supply. Many employers would genuinely value concrete evidence that you have learned to write quality code, even if that doesn't come from a prestigious computer science degree, specifically because demand is so high. Under these circumstances, enhancing human capital actually matters, and has a positive overall effect both at the national income level and the personal income level. In other words, by learning the skills required to be a software engineer you aren't merely signaling your higher competence, you actually are more economically productive in your new role, and the demand-suppply dynamics mean that the market has the capacity to hoover up almost everyone with the right skills and put them to productive use.

After reading Caplan's book, I'm convinced that no assessment of an EdTech opportunity is complete without thinking carefully about the signaling angle - especially if the intent of EdTech investing is to make impactful, socially important investments. Like many people, I've always firmly believed in the power of education, and I'm also generally a staunch advocate for the power of capitalism and investment to yield favourable outcomes and improve lives. But The Case Against Education has made me considerably more sceptical about exactly how education typically works its magic in our society, and whether ever more of it is really what we need after all.