Barriers to Scientific Progress – Part 4

Over-specialization, continued.

Continuing this essay series on modern barriers to scientific progress, using David Epstein’s book Range: Why Generalists Triumph in the Specialized World, this essay will continue to explore how the modern phenomenon of over-specialization might be posing an additional barrier to scientific progress.

Researchers have probed into whether the sort of specialization pushed by universities tends to diminish critical thinking.  As Epstein writes:

[James] Flynn’s great disappointment is the degree to which society, and particularly higher education, has responded to the broadening of the mind by pushing specialization, rather than focusing early training on conceptual, transferable knowledge. Flynn conducted a study in which he compared the grade point averages of seniors at one of America’s top state universities, from neuroscience to English majors, to their performance on a test of critical thinking. The test gauged students’ ability to apply fundamental abstract concepts from economics, social and physical sciences, and logic to common, real- world scenarios. Flynn was bemused to find that the correlation between the test of broad conceptual thinking and GPA was about zero. In Flynn’s words, “the traits that earn good grades at [the university] do not include critical ability of any broad significance.” … Biology and English majors did poorly on everything that was not directly related to their field. None of the majors, including psychology, understood social science methods. Science students learned the facts of their specific field without understanding how science should work in order to draw true conclusions. Neuroscience majors did not do particularly well on anything. Business majors performed very poorly across the board, including in economics. Econ majors did the best overall. Economics is a broad field by nature, and econ professors have been shown to apply the reasoning principles they’ve learned to problems outside their area. Chemists, on the other hand, are extraordinarily bright, but in several studies struggled to apply scientific reasoning to nonchemistry problems … Flynn’s conclusion: “There is no sign that any department attempts to develop [anything] other than narrow critical competence.” The study he conducted at the state university convinced him that college departments rush to develop students in a narrow specialty area, while failing to sharpen the tools of thinking that can serve them in every area. This must change, he argues, if students are to capitalize on their unprecedented capacity for abstract thought. They must be taught to think before being taught what to think about. Students come prepared with scientific spectacles, but do not leave carrying a scientific-reasoning Swiss Army knife.

Epstein points out that “Three-quarters of American college graduates go on to a career unrelated to their major— a trend that includes math and science majors— after having become competent only with the tools of a single discipline.”

Epstein also points to interesting evidence regarding the much greater effectiveness in the long-run of teachers who press their students to “deep learning,” such that their courses may be more difficult in the short-run (because they stress the underlying reasoning behind the concepts they teach, and not just methods for getting answers) but they tend to produce better critical learners in the long-run:

In return for full scholarships, cadets at the Air Force Academy commit to serve as military officers for a minimum of eight years after graduation. They submit to a highly structured and rigorous academic program heavy on science and engineering. It includes a minimum of three math courses for every student. Every year, an algorithm randomly assigns incoming cadets to sections of Calculus I, each with about twenty students. To examine the impact of professors, two economists compiled data on more than ten thousand cadets who had been randomly assigned to calculus sections taught by nearly a hundred professors over a decade. Every section used the exact same syllabus, the exact same exam, and the exact same post- course professor evaluation form for cadets to fill out. After Calculus I, students were randomized again to Calculus II sections, again with the same syllabus and exam, and then again to more advanced math, science, and engineering courses. The economists confirmed that standardized test scores and high school grades were spread evenly across sections, so the instructors were facing similar challenges. The Academy even standardized test- grading procedures, so every student was evaluated in the same manner. “Potential ‘bleeding heart’ professors,” the economists wrote, “had no discretion to boost grades.” That was important, because they wanted to see what differences individual teachers made. Unsurprisingly, there was a group of Calculus I professors whose instruction most strongly boosted student performance on the Calculus I exam, and who got sterling student evaluation ratings. Another group of professors consistently added less to student performance on the exam, and students judged them more harshly in evaluations. But when the economists looked at another, longer- term measure of teacher value added— how those students did on subsequent math and engineering courses that required Calculus I as a prerequisite— the results were stunning. The Calculus I teachers who were the best at promoting student overachievement in their own class were somehow not great for their students in the long run. “Professors who excel at promoting contemporaneous student achievement,” the economists wrote, “on average, harm the subsequent performance of their students in more advanced classes.” What looked like a head start evaporated. The economists suggested that the professors who caused short-term struggle but long-term gains were facilitating “deep learning” by making connections. They “broaden the curriculum and produce students with a deeper understanding of the material.” It also made their courses more difficult and frustrating, as evidenced by both the students’ lower Calculus I exam scores and their harsher evaluations of their instructors. And vice versa. The calculus professor who ranked dead last in deep learning out of the hundred studied— that is, his students underperformed in subsequent classes— was sixth in student evaluations, and seventh in student performance during his own class. Students evaluated their instructors based on how they performed on tests right now— a poor measure of how well the teachers set them up for later development— so they gave the best marks to professors who provided them with the least long-term benefit.

That discussion reminded me of a passage from Surely You’re Joking, Mr. Feynman? in which the great physicist Richard Feynman describes his frustration with the way school textbooks tend to “dumb down” concepts in an apparent attempt to simplify learning:

What finally clinched it, and made me ultimately resign [from a committee charged with approving public school textbooks], was that the following  year we were going to discuss science books. I thought maybe the science would be different, so I looked at a few of them. The same thing happened: something would look good at first and then turn out to be horrifying. For example, there was a book that started out with four pictures: first there was a wind­up toy; then there was an automobile; then there was a boy riding a bicycle; then there was something else. And underneath each picture it said, "What makes it go?" I thought, "I know what it is: They're going to talk about mechanics, how the springs work inside the toy; about chemistry, how the engine of the automobile works;  and biology, about how the muscles work." It was the kind of thing my father would have talked about: "What makes it go?  Everything goes because the sun is shining." And then we would have fun discussing it: "No, the toy goes because the spring is wound up," I would say. "How did the spring get wound up?" he would ask. "I wound it up." "And how did you get moving?" "From eating." "And food grows only because the sun is shining. So it's because the sun is shining that all these things are moving." That would get the concept across that motion is simply the transformation of the sun's power. I turned the page. The answer was, for the wind­up toy, "Energy makes it go." And for the boy on the bicycle, "Energy makes it go." For everything, "Energy makes it go." Now that doesn't mean anything. Suppose it's "Wakalixes." That's the general  principle: "Wakalixes makes it go." There's no knowledge coming in. The child doesn't learn anything; it's just a word! What they should have done is to look at the wind­up toy, see that there are springs inside, learn about springs, learn about wheels, and never mind "energy." Later on, when the children know something about how the toy actually works, they can  discuss the more general principles of energy. It's also not even true that "energy makes it go," because if it stops, you could say, "energy makes it stop" just as well. What they're talking about is concentrated energy being transformed into more dilute forms, which is a very subtle aspect of energy. Energy is neither increased nor decreased in these examples; it's just changed from one form to another. And when the things stop, the energy is changed into heat, into general chaos. But that's the way all the books were: They said things that were useless, mixed­ up, ambiguous, confusing, and partially incorrect. How anybody can learn science from these books, I don't know, because it's not science.

In my experience substitute teaching at local public schools, educational materials do this with alarming frequency – namely, just labeling things, not explaining them.

Researchers have studied how best to help students make connections between concepts in different fields.  As Epstein writes:

Dedre Gentner wanted to find out if everyone can be a bit more … capable of wielding distant analogies to understand problems. So she helped create the “Ambiguous Sorting Task.” It consists of twenty- five cards, each one describing a real- world phenomenon, like how internet routers or economic bubbles work. Each card falls into two main categories, one for its domain (economics, biology, and so on) and one for its deep structure. Participants are asked to sort the cards into like categories. For a deep structure example, you might put economic bubbles and melting polar ice caps together as positive-feedback loops. (In economic bubbles, consumers buy stocks or property with the idea that the price will increase; that buying causes the price to increase, which leads to more buying. When ice caps melt, they reflect less sunlight back to space, which warms the planet, causing more ice to melt.) Or perhaps you would put the act of sweating and actions of the Federal Reserve together as negative- feedback loops. (Sweating cools the body so that more sweating is no longer required. The Fed lowers interest rates to spur the economy; if the economy grows too quickly, the Fed raises rates to slow down the activity it launched.) The way gas prices lead to an increase in grocery prices and the steps needed for a message to traverse neurons in your brain are both examples of causal chains, where one event leads to another, which leads to another, in linear order. Alternatively, you might group Federal Reserve rate changes, economic bubbles, and gas price changes together because they are all in the same domain: economics. And you might put sweating and neurotransmission together under biology. Gentner and colleagues gave the Ambiguous Sorting Task to Northwestern University students from an array of majors and found that all of the students figured out how to group phenomena by domains. But fewer could come up with groupings based on causal structure. There was a group of students, however, who were particularly good at finding common deep structures: students who had taken classes in a range of domains, like those in the Integrated Science Program. Northwestern’s website for the program features an alum’s description: “Think of the Integrated Science Program as a biology minor, chemistry minor, physics minor, and math minor combined into a single major. The primary intent of this program is to expose students to all fields of the natural and mathematical sciences so that they can see commonalities among different fields of the natural sciences … The ISP major allows you to see connections across different disciplines.” A professor I asked about the Integrated Science Program told me that specific academic departments are generally not big fans. They want students to take more specialized classes in a single department. They are concerned about the students falling behind. They would rather rush them to specialization than equip them with ideas from what Gentner referred to as a “variety of base domains,” which foster analogical thinking and conceptual connections that can help students categorize the type of problem they are facing. That is precisely a skill that sets the most adept problem solvers apart. In one of the most cited studies of expert problem solving ever conducted, an interdisciplinary team of scientists came to a pretty simple conclusion: successful problem solvers are more able to determine the deep structure of a problem before they proceed to match a strategy to it. Less successful problem solvers are more like most students in the Ambiguous Sorting Task: they mentally classify problems only by superficial, overtly stated features, like the domain context. For the best performers, they wrote, problem solving “begins with the typing of the problem.”

Epstein writes about an interesting project from the 1960’s in which a library specialist created a program designed to connect research literature from different fields that might yield cross-fertilizing results:

[Don] Swanson earned a physics PhD in 1952, and then worked as an industry computer systems analyst, where he became fascinated with organizing information. In 1963, the University of Chicago took a chance on him as dean of the Graduate Library School. As a thirty-eight-year- old from private industry, he was an oddball. The hiring announcement declared, “Swanson is the first physical scientist to head a professional library school in this country.” Swanson became concerned about increasing specialization, that it would lead to publications that catered only to a very small group of specialists and inhibit creativity. “The disparity between the total quantity of recorded knowledge . . . and the limited human capacity to assimilate it, is not only enormous now but grows unremittingly,” he once said. How can frontiers be pushed, Swanson wondered, if one day it will take a lifetime just to reach them in each specialized domain? In 1960, the U.S. National Library of Medicine used about one hundred unique pairs of terms to index articles. By 2010, it was nearing one hundred thousand. Swanson felt that if this big bang of public knowledge continued apace, subspecialties would be like galaxies, flying away from one another until each is invisible to every other. Given that he knew interdisciplinary problem solving was important, that was a conundrum. In crisis, Swanson saw opportunity. He realized he could make discoveries by connecting information from scientific articles in subspecialty domains that never cited one another and that had no scientists who worked together. For example, by systematically cross- referencing databases of literature from different disciplines, he uncovered “eleven neglected connections” between magnesium deficiency and migraine research, and proposed that they be tested. All of the information he found was in the public domain; it had just never been connected. “Undiscovered public knowledge,” Swanson called it. In 2012, the American Headache Society and the American Academy of Neurology reviewed all the research on migraine prevention and concluded that magnesium should be considered as a common treatment. The evidence for magnesium was as strong as the evidence for the most common remedies, like ibuprofen. Swanson wanted to show that areas of specialist literature that never normally overlapped were rife with hidden interdisciplinary treasures waiting to be connected.

Epstein also writes how another researcher, Andy Ouderkirk:

became so interested in classifying innovators that he wrote a computer algorithm to analyze ten million patents from the last century and learn to identify and classify different types of inventors. Specialist contributions skyrocketed around and after World War II, but more recently have declined. “Specialists specifically peaked about 1985,” Ouderkirk told me. “And then declined pretty dramatically, leveled off about 2007, and the most recent data show it’s declining again, which I’m trying to understand.”

Today, people aren’t likely to be able to develop a range of experience if they aren’t willing to switch careers at any point.  As Epstein writes:

In 1979, Christopher Connolly [found] that early in their careers, those who later made successful transitions had broader training and kept multiple “career streams” open even as they pursued a primary specialty. They “traveled on an eight-lane highway,” he wrote, rather than down a single- lane one- way street. They had range. The successful adapters were excellent at taking knowledge from one pursuit and applying it creatively to another, and at avoiding cognitive entrenchment. They employed what [Robin] Hogarth called a “circuit breaker.” They drew on outside experiences and analogies to interrupt their inclination toward a previous solution that may no longer work. Their skill was in avoiding the same old patterns … “Match quality” is a term economists use to describe the degree of fit between the work someone does and who they are— their abilities and proclivities. Northwestern University economist Ofer Malamud’s inspiration for studying match quality was personal experience. He was born in Israel, but his father worked for a shipping company, and when Malamud was nine the family moved to Hong Kong, where he attended an English school. The English system required that a student home in on an academic specialization in the last two years of high school. “When you applied to a college in England, you had to apply to a specific major,” Malamud told me. His father was an engineer, so he figured he should do engineering. At the last moment, he chose not to pick a specialty. “I decided to apply to the U.S. because I didn’t know what I wanted to do,” he said. He started with computer science, but quickly learned that wasn’t his thing. So he sampled subjects before settling on economics and then philosophy. The experience left him with an abiding curiosity about how the timing of specialization impacts career choice. In the late 1960s, future Nobel laureate economist Theodore Schultz argued that his field had done well to show that higher education increased worker productivity, but that economists had neglected the role of education in allowing individuals to delay specialization while sampling and finding out who they are and where they fit. Malamud could not randomly assign people to life in order to study specialization timing, but he found a natural experiment in the British school system. For the period he studied, English and Welsh students had to specialize before college so that they could apply to specific, narrow programs. In Scotland, on the other hand, students were actually required to study different fields for their first two years of college, and could keep sampling beyond that. In each country, every college course that a student took provided skills that could be applied in a specific field, as well as information about their match quality with the field itself. If students focused earlier, they compiled more skills that prepared them for gainful employment. If they sampled and focused later, they entered the job market with fewer domain- specific skills, but a greater sense of the type of work that fit their abilities and inclinations. Malamud’s question was: Who usually won the trade-off, early or late specializers? If the benefit of higher education was simply that it provided skills for work, then early-specializing students would be less likely to career switch after college to a field unrelated to their studies: they have amassed more career- specific skills, so they have more to lose by switching. But if a critical benefit of college was that it provided information about match quality, then early specializers should end up switching to unrelated career fields more often, because they did not have time to sample different matches before choosing one that fit their skills and interests. Malamud analyzed data for thousands of former students, and found that college graduates in England and Wales were consistently more likely to leap entirely out of their career fields than their later- specializing Scottish peers. And despite starting out behind in income because they had fewer specific skills, the Scots quickly caught up. Their counterparts in England and Wales were more often switching fields after college and after beginning a career even though they had more disincentive to switch, having focused on that field. With less sampling opportunity, more students headed down a narrow path before figuring out if it was a good one. The English and Welsh students were specializing so early that they were making more mistakes. Malamud’s conclusion: “The benefits to increased match quality … outweigh the greater loss in skills.” Learning stuff was less important than learning about oneself. Exploration is not just a whimsical luxury of education; it is a central benefit. It should come as no surprise that more students in Scotland ultimately majored in subjects that did not exist in their high schools, like engineering. In England and Wales, students were expected to pick a path with knowledge only of the limited menu they had been exposed to early in high school. That is sort of like being forced to choose at sixteen whether you want to marry your high school sweetheart. At the time it might seem like a great idea, but the more you experience, the less great that idea looks in hindsight. In England and Wales, adults were more likely to get divorced from the careers they had invested in because they settled down too early. If we treated careers more like dating, nobody would settle down so quickly. For professionals who did switch, whether they specialized early or late, switching was a good idea. “You lose a good fraction of your skills, so there’s a hit,” Malamud said, “but you do actually have higher growth rates after switching.”  Regardless of when specialization occurred, switchers capitalized on experience to identify better matches.

Epstein describes another experiment on the same subject:

Steven Levitt, the economist who coauthored Freakonomics, cleverly leveraged his readership for a test of switching. On the “Freakonomics Experiments” home page, he invited readers who were considering life changes to flip a digital coin. Heads meant they should go ahead and make the change, tails that they should not. Twenty thousand volunteers responded, agonizing over everything from whether they should get a tattoo, try online dating, or have a child, to the 2,186 people who were pondering a job change. But could they really trust a momentous decision to chance? The answer for the potential job changers who flipped heads was: only if they wanted to be happier. Six months later, those who flipped heads and switched jobs were substantially happier than the stayers. According to Levitt, the study suggested that “admonitions such as ‘winners never quit and quitters never win,’ while well- meaning, may actually be extremely poor advice.” Levitt identified one of his own most important skills as “the willingness to jettison” a project or an entire area of study for a better fit. Winston Churchill’s “never give in, never, never, never, never” is an oft- quoted trope. The end of the sentence is always left out: “except to convictions of honor and good sense.” Labor economist Kirabo Jackson has demonstrated that even the dreaded administrative headache known as “teacher turnover” captures the value of informed switching. He found that teachers are more effective at improving student performance after they switch to a new school, and that the effect is not explained by switching to higher- achieving schools or better students. “Teachers tend to leave schools at which they are poorly matched,” he concluded. “Teacher turnover . . . may in fact move us closer to an optimal allocation of teachers to schools.” Switchers are winners. It seems to fly in the face of hoary adages about quitting, and of far newer concepts in modern psychology.

Other research shows the limits of the slogan “quitters never win.”  As Epstein writes:

Seth Godin, author of some of the most popular career writing in the world, wrote a book disparaging the idea that “quitters never win.” Godin argued that “winners”— he generally meant individuals who reach the apex of their domain— quit fast and often when they detect that a plan is not the best fit, and do not feel bad about it. “We fail,” he wrote, when we stick with “tasks we don’t have the guts to quit.” Godin clearly did not advocate quitting simply because a pursuit is difficult. Persevering through difficulty is a competitive advantage for any traveler of a long road, but he suggested that knowing when to quit is such a big strategic advantage that every single person, before undertaking an endeavor, should enumerate conditions under which they should quit. The important trick, he said, is staying attuned to whether switching is simply a failure of perseverance, or astute recognition that better matches are available. A recent international Gallup survey of more than two hundred thousand workers in 150 countries reported that 85 percent were either “not engaged” with their work or “actively disengaged.” In that condition, according to Seth Godin, quitting takes a lot more guts than continuing to be carried along like debris on an ocean wave. The trouble, Godin noted, is that humans are bedeviled by the “sunk cost fallacy.” Having invested time or money in something, we are loath to leave it, because that would mean we had wasted our time or money, even though it is already gone.

Epstein describes the many advantages that follow from being willing to bob and weave with your interests:

“If you get someone into a context that suits them,” [Ogi] Ogas said, “they’ll more likely work hard and it will look like grit from the outside.” … Herminia Ibarra, a professor of organizational behavior at London Business School … compiled her findings [and found that] the central premise was at once simple and profound: we learn who we are only by living, and not before. Ibarra concluded that we maximize match quality throughout life by sampling activities, social groups, contexts, jobs, careers, and then reflecting and adjusting our personal narratives. And repeat. If that sounds facile, consider that it is precisely the opposite of a vast marketing crusade that assures customers they can alight on their perfect matches via introspection alone … Paul Graham, computer scientist and cofounder of Y Combinator— the start- up funder of Airbnb, Dropbox, Stripe, and Twitch— encapsulated [Herminia] Ibarra’s tenets in a talk he wrote for a high school, but never delivered: It might seem that nothing would be easier than deciding what you like, but it turns out to be hard, partly because it’s hard to get an accurate picture of most jobs. . . . Most of the work I’ve done in the last ten years didn’t exist when I was in high school. . . . In such a world it’s not a good idea to have fixed plans. And yet every May, speakers all over the country fire up the Standard Graduation Speech, the theme of which is: don’t give up on your dreams. I know what they mean, but this is a bad way to put it, because it implies you’re supposed to be bound by some plan you made early on. The computer world has a name for this: premature optimization. . . . . . . Instead of working back from a goal, work forward from promising situations. This is what most successful people actually do anyway. In the graduation- speech approach, you decide where you want to be in twenty years, and then ask: what should I do now to get there? I propose instead that you don’t commit to anything in the future, but just look at the options available now, and choose those that will give you the most promising range of options afterward.” … Ibarra’s new aphorism [is] “I know who I am when I see what I do.”

In the next essay in this series, we’ll explore barriers to scientific progress caused not only by the sheer number of scientific publications, but by how they’re put together.