Wednesday, September 15, 2021

Modeling Creative Bursts

It is a well-known theme of intellectual history that creative work often comes in bursts. A classic case is the major advances Newton made on gravity and calculus during the term Cambridge was closed due to the plague. You often read about "miraculous years" in the life of an artist, when one masterpiece after another poured out; Mozart's last year is a famous example.

A bunch of statisticians have just published a paper in Nature using big data, machine learning, image analysis, and a bunch of very fancy statistics to explore this phenomenon, and they find the following:

Across a range of creative domains, individual careers are characterized by hot streaks, which are bursts of high-impact works clustered together in close succession. Yet it remains unclear if there are any regularities underlying the beginning of hot streaks. Here, we analyze career histories of artists, film directors, and scientists, and develop deep learning and network science methods to build high-dimensional representations of their creative outputs. We find that across all three domains, individuals tend to explore diverse styles or topics before their hot streak, but become notably more focused after the hot streak begins. Crucially, hot streaks appear to be associated with neither exploration nor exploitation behavior in isolation, but a particular sequence of exploration followed by exploitation, where the transition from exploration to exploitation closely traces the onset of a hot streak. Overall, these results may have implications for identifying and nurturing talents across a wide range of creative domains.

This makes intuitive sense. Artists experiment with different styles ("exploration"), then settle on one that works for them and throw themselves into work in that vein ("exploitation"). Chaucer, for example, wrote verse in the Italian style, then in the French style, then settled on his own style and poured forth The Canterbury Tales. Young composers often write pieces in the style of some other composer – Mozart's Variations on a Theme by Salieri is one you may have heard mentioned in a certain film – before finding their own voices. Scientists wander around working on different projects, then find a particular kind of research that feels both vital and right for them, throwing themselves into the work.

I have some caveats, though. For scientists in particular institutional support is crucial; if you read what scientists say about their own careers they often emphasize the situation they were in when they did their most famous work, with the right support and the right colleagues. Darwin's career can be boiled down to: voyage on the Beagle, which opened his eyes to biological diversity in a radical way, followed by fifty years of working and reworking the insights gained on that crucial trip. It is hard to imagine his career without the British government sending him around the world.

You can also think of many artists whose careers have nothing in common with this paradigm, e.g. John Singer Sargent who did great work in pretty much the same style for forty years. Novelists, a kind of artist highly important to me, have radically varying career trajectories. One common type is that they have something to say, usually drawn from their own experience, say it in one or two books, and then never produce anything else nearly as good. Others follow a paradigm something like what these authors describe, for example Hillary Mantel experimenting with various sorts of writing before setting on multi-part historical novels, coming to a glorious peak with Wolf Hall and Bring Out the Bodies. The authors of our studies acknowledge this when they say that while the pattern they see is statistically robust, "the overall effect size seems modest."

I find this interesting as an insight into creative careers but also as a glimpse into what sort of studies may become possible as data sets grow and machine learning deepens. So far, though, what can be produced seems like a rather minor contribution, not very new and explaining only part of what makes for a creative, productive career.

Via Marginal Revolutions

No comments: