Root, by Cohle Wehrle, is my catnip.  It’s a game where factions of adorable woodlands creatures engage in brutal asymmetric warfare to see who will rule the woods.  I grew up reading the Redwall books. Developed an obsession with the history of American empire, counter-insurgency warfare, and the post-modern strategic theories of Colonel John Boyd. I have a doctorate in the Foucauldian analysis of state construction through medicine. I was hooked when I saw the first Kickstarter ad.  When I read a development diary, where Wehrle described one of his goals as developing a biopolitical critique of the state as represented in boardgame, that hook set barbs.

Get. In. This game is mine. It was made for me.  Root is also objectively an incredible game.

The game components of the highest quality, with wonderful art by Kyle Ferrin that conveys both the whimsy and barbarity of woodland combat. The are touches of craft everywhere.  The player boards are masterpieces of clarity, and absolutely necessary since every faction plays differently. Combat is resolved with large d12s marked from 0-3, which have a satisfying heft and roll.  The core game is based around four players, with support for two to six with the expansion, and a co-op mode against a bot faction.  Each player turn in broken into three phases: birdsong, daylight, and evening, and each phase offers potential actions relating to building, crafting, recruiting, and battling.  But in an amazing bit of asymmetric design, no two factions plays remotely similarly. 

The Marquis de Cat is a rising imperial power. The Cat has a powerful economic engine but has to spend cards to get more than three actions per turn and has trouble putting out fires across the map.  The Woodland Alliance are a furry Viet Cong, spreading networks of sympathizers through the woods. They have a pile of supporter cards, which they use to spread sympathy and foster revolts, building towards a conventional military per the third stage of Mao’s People’s War doctrine, but their army will always be small and limited compared to the other factions. The Eyrie Lords seek to reclaim what was once theirs, making multiple moves with a decree stating where they Recruit --> Move --> Battle --> Build, but if they can’t do one of those actions, they fall into disarray, losing their turn, a lot of victory points, and their decree.  Finally, the Vagabond is a lone hero, a trickster figure who aids and fights the other factions, completing quests to win fame in the eyes of the common people.  The expansion factions are Otter Mercenaries, who sell their services to the highest bidder and hope to come out of the war with the most profit, and the Lizard Cult, who use the discard pile to rally the discontented of the woods to their alien ideology.

The fluff is amazing, but Root goes one further and makes the gameplay work. Pacing in boardgames is what separates the good from the great. Root is a brisk 60-90 minutes, played to either 30 victory points or one of four domination wins based on control of the board. This is just enough time to expand, to suffer reverses, and to make a break for victory, without a tedious step of playing out the inevitable. The strategy is driven by intense player interaction. The Cat, Eyrie, and Alliance must expand to score victory points, making conflict inevitable, but any faction that draws the ire of the rest is going to get bloodied. Politics at the table matters as much as Wehrle’s biopolitics of hand management.

Root compares favorably to its nearest neighbors in the design space. The COIN series (A Distant Plain, Andean Abyss, Fire in the Lake, etc) are the gold standard, but while I love a good COIN, they are heavy games, with a lot of tiny pieces, complexities to remember in your turn, and player aids that are more functional than elegant. The card-driven turn order and events also introduce some unfairness.  I recall one game of Fire in the Lake where a card event wiped out every NVA base along the Ho Chi Minh trail, and then due to turn order they never got a chance to rebuild.  Just bad luck.  The COIN games go a little too far in emulating the quagmire of counter-insurgency warfare. To win, you have to knock opposing factions down to low resources so they can’t fight back, establish dominance over your ally, and then hold it until the next scoring phase.  Ending a COIN game with a victory for anyone is hard.

Scythe is also a near neighbor, a thematic blend of Eurogame and wargame.  Scythe is fast, but it often feels like multiplayer solitaire, as each player optimizes the economic engine on their own board.  The battles are more border skirmishes to get victory stars, not decisive actions for control of territory.  The Scythe board is big enough that attacking someone is rarely required, and combat punishing enough that conquest is discouraged. Scythe ends abruptly, usually when one player jumps from four to six victory stars.  And while the theme of mechs and workers battling it out in interwar Eastern Europe is cool, I’ve never felt like it was linked to the mechanics. Popular support is a number on a scale, not a political reality.  When in my game of Root, the Vagabond allied with the Marquis de Cat and destroyed a Woodlands Alliance base, scattering their supporters, I could see the action of this powerful hero-turned-villain striking down rabbits in red berets in their tunnel command post.

I’ve only scratched the surface of Root. I can’t wait to learn how to play properly.  It is genius. Sheer, bloody, genius. This isn’t a world for the weak.  If you want to rule, you have to be the cutest and the deadliest! ​
​

I’d like to sum up the past two weeks of Data Science in Public posts to discuss what I did, and what we learned.  I’m interested in interdisciplinary science, because so much of science is done by interdisciplinary teams. Very few of the problems that we care about: fighting cancer, clean energy systems, ecological resilience and human well-being, have relevant discoveries that can be made by the idealized model of a lone specialist who is very knowledgeable at one type of thing. The National Science Foundation recognizes, and has been aiding in the transformation of science, in part with the IGERT/NRT grant series.  These awards are for about $3,000,000 over five years, with most of the funding going towards graduate student stipends to encourage innovative models in STEM graduate education on high-priority interdisciplinary topics. There have been over 400 of these awards made since 1997, for something like $1.2 billion in total.
​
I think it’s important to know if this program is working.  The 2018 NRT funding solicitation states: “The program is dedicated to effective training of STEM graduate students in high priority interdisciplinary research areas, through the use of a comprehensive traineeship model that is innovative, evidence-based, and aligned with changing workforce and research needs.”  In an ideal world, we’d be able to embed trained ethnographers in every one of these grants, to track the development of students into scientists.  That’s not feasible, and we can’t go back to the earliest grants without a time machine, but one of the nice things about scientists is that they publish what they discover, those publications are indexed by databases like the Thomson Reuters - ISI Web of Knowledge, and we can use the richly structured information in publications to say something about the grants that produced them.

In one sentence, can we use data science to show that the IGERT/NRT solicitation produced more interdisciplinary scholarship?

You can see the source of the data and how it was transformed at each step in the flowchart below. We go from all the IGERTs, to a sample of five, to a list of 202 scientists, a corpus of over 3400 publications, and then a dataframe of feature vectors amenable to data science techniques. An anonymized version of the final dataset (5-scientometrics101-cites.csv) along with some code is available on my GitHub.

I describe how to evaluate interdisciplinarity in DSiP 2, but I have a better metaphor I'd like to share.  Image all of scientific knowledge like the night sky, each star a different fact known to science.  New scientific knowledge is like adding a new star to the heavens. Scientists don’t just throw their ideas up in the air: they construct truth by tying each new claim into what is already known.  Scientific writing is a lot like finding constellations. Constellations aren’t “real” features of the universe, the individual stars may be separated by thousands of lightyears, but they are real features of human experience. The connections that we draw between facts are just as important to understanding as the facts themselves.   

Building on that analogy, we’d expect different kinds of scientists, say a chemist and a botanist, to make different kinds of “constellations” when they’re writing their papers.  And we’d expect a monodisciplinary paper to look different from an interdisciplinary paper.  This concept is operationalized in the Rao-Stirling Diversity Index (SDI), which measures how the citations of a paper are distributed across Web of Science subject categories, and varies between 0 for work entirely within one discipline, and approaches 1 as work becomes increasingly interdisciplinary.
​
The tedious part is going from a list of grants, to a roster of scientists, to a corpus of publications.  Then comes the fun part.  Using the Metaknowledge package by Reid McIlroy-Young, John McLevey, and Jillian Anderson, which is an absolutely fantastic tool for bibliographic work, a directory mapping journal abbreviations to Web of Science subject categories provided by Thompson-Reuters customer service, and a similarity matrix between all Web of Science subject categories by Ismael Rafols, Alan Porter, and Loet Leydesdorff, we’re able to process Web of Science records into a table which looks like this.

​The first thing to check is how the Sterling Diversity Index changes over time.  A boxplot shows that for each of the five groups, the Sterling Diversity Index increased.

A more detailed look at the SDI, with mean and interquartile range plotted, shows the patterns for each group year by year, with the start of each IGERT grant marked by a vertical line. The steep increase of Group B five years after the start of the grant, and the way that SDI stopped declining and actually increased when Group D got their grant are particularly interesting.

I ran t-tests on the change in each of the groups, and the results are clear. The increase is statistically significant, and averages 0.38 standard deviations. If we imagine that SDI is analogous to IQ, where each standard deviation is 15 points, the increase in scores is equivalent to a gain of 5.7 points. That’s pretty good, given how hard it is to make major structural changes in science.

t-test    : p-value  : increase in st.dev
Group A: 0.00833 : +0.33257
Group B: 0.00000 : +0.72993
Group C: 0.00879 : +0.15155
Group D: 0.00250 : +0.31565
Group E: 0.00030 : +0.38804

The IGERT/NRT program has meet its primary stated policy objective. Going forward, I think it'd be interesting to look at the program as a whole, as opposed to just a sample. And we need to look in further detail at the groups that met these goals exceptionally well in order to understand what they did, and how those practices can be translated to other research groups.
​

Everything above is descriptive statistics and data visualization, which are important, but something that I've done before. What's new to me is machine learning.  Machine learning differs from traditional statistics in that while in theory I could have calculated everything above by hand with a lot of tedium, machine learning is about computer programs that optimize the parameters of an algorithm according to a performance measure. There's not a firm line between machine learning and statistics, and indeed most machine learning curriculum start with linear regression, drawing best fit lines through points.

I had some trouble coming up with a good machine learning task for this dataset.  The major types of machine learning operations are regression, predicting a continuous value from a set of inputs; classification, which involves assigning an input to one or more categories; and clustering, finding patterns in data that does not have a prior structure. I toyed with coming up a regression model to predict which scientific papers would have the greatest impact, but discarded it. Important ideas are important because of their content, not because of their citations.

A flaw in my dataset presented the solution. For some reason, 309 of the papers in my sample aren’t associated with any research group.  All five of my IGERTs work in different areas, so maybe I can use patterns of citations to match my misfiled papers with the right research group.  I ran three common classification algorithms, Logistic Regression, a Support Vector Machine, and a Decision Tree, and found that I could associate 70% of papers in the test split with the correct group.  The Support Vector Machine had slightly higher accuracy, but performance was similar across the board.  Looking at the confusion matrices for each algorithm, we can see that most of the errors involve sorting papers in Groups B and E with Group C.  This makes sense, since Group C is the most prolific group, and all three groups work on nanotechnology.  There is overlap in papers about basic topics like the properties of carbon nanotubes.

Looking at the workings of the decision tree can be useful, because it mimics a common human way to sort things.  Pick an attribute where there is a clear difference between Category A and the rest of the data, split, and repeat until each little clump is all of one kind.  To read the decision tree, start at the top, check the condition, and if True go left, and False go right.  Geography shows up immediately as a way to distinguish Group D, medical sciences as a way to separate Group E, and material science tends to lead to Group C.

(click for big)

Knowing that, I can assign the missing papers to their most likely groups with 70% accuracy.  I wrote up a script to do that, and then go back to the articles check which university the authors are at. In the majority of cases, at least one of the authors is at the university that my classifiers assigned the paper to.

Thank you to my instructor, Nathan Grossman, who asked questions that lead a significantly more interesting project, my cohort in the Metis Live Online class, for listening to me explain versions of this work four times, and Sebastian Raschka, for his great book on Python Machine Learning.

Last time, we proved that there was an increase in the Sterling Diversity Index for papers published by a group after an IGERT was awarded. But that is basic statistics, not data science. It turns out that along with the 3139 papers that were published by the five IGERT groups in my sample, I have 307 papers which aren't affiliated with any group. I'm not sure how that happened, but it provides a useful finale to the project. Can I fairly assign each of the 307 error papers to one of my five IGERT groups?
​
Each of the five IGERTs works on its own research topic, so to do this, we need to find a data science method to figure out what the topics of the five groups are, and assign new cases to each group with some degree of certainty. This sounds a lot like good case for logistic regression to me. To do this, I've expanded the data with the citation categories for each paper, which I used to calculate the Stirling Diversity Index. It that looks like this, extended out through the 224 Web of Science subject categories.

We import LogisticRegression and some the usual data wrangling packages, then set up our X to be all the citation categories of papers not in the ERROR group, and y to be the group that they are actually in.  Actually going it is about a dozen lines of code to select the right parts of the data frame, scale the counts, split the data into train and test sets, and then run the logistic regression.

Our accuracy is 0.72, which is acceptably above chance, 0.2 in this case. I tried changing the hyperparameters, and using a support vector machine, and couldn't get higher accuracy.

Let's inspect the confusion matrix, to see where we're guessing wrong. It's clear that in terms of raw numbers, papers from Group C dominate, and we have trouble distinguishing papers in Group B and Group E from those in group C. This makes sense, since all I know from my prior investifation those three groups work in nanotechnology, among other things. It is plausible that there is research on something fundamental, like the properties of carbon nanotubes, which all three groups contribute to.

[ 55 7 16 8 3]
[ 7 99 58 1 12]
[ 4 16 359 4 21]
[ 6 4 20 92 6]
[ 3 8 54 0 79]

And finally, we grab the ERROR papers and run them through the linear regression to see that most of them belong to group C.  

A 8
B 0
C 280
D 13
E 6

This is as good a place as any to put the end of the hard work of the data science in public series.  I'll have one more post wrapping it all up.