From last time, we ran into some problems with the scientists data, which meant it's time to reappraise. Some discussion with my cohort and instructor suggested a few things. First, the scientists data is hard to interprate. Most of the data comes from Americans who are part of an IGERT grant, i.e. a Federal effort to create more interdisciplinary scholars by new funding sources. However about a quarter of the data are Germans who are members of interdisciplinary research centers that formed organically, and then were recognized by the German government. Second, in general scientific careers switch directions very slowly, and it's hard to do much more than categorize them. Third, the interesting science policy question can be summed up as "Did the IGERTs work?" Did these grants foster more interdisciplianary scholarship?
Since I really want to avoid scraping and cleaning more data, I decided to go back to the original data and evaluate them by papers. Each paper is described by several statistics.
We import the data, and run our initial descriptive stats tests.
Since I really want to avoid scraping and cleaning more data, I decided to go back to the original data and evaluate them by papers. Each paper is described by several statistics.
- first_author (bool): is the first author a member of an IGERT grant
- group (categorical): Five IGERTs, A, B, C, D, E
- group_start (int): the year the group was awarded the grant
- number_authors (int) : how many authors on the paper
- sdi (float) : the Stirling Diversity Index for the paper
- times cited (int) : how many times the article has been cited
- journal (string) : the journal that published the paper
- pub_year (int) : date of publication
- velocity (float) : number of citations / years since publication. A measure of impact
- grant_product (categorical) : if the paper was published before or after the grant was awarded
We import the data, and run our initial descriptive stats tests.
A few things jump out. Web of Science began indexing more publications in 2005. The distribution for number of authors and times_cites/velocity are heavily right skewed. There's no clear trend bewtween sdi and any other variable. Papers with many authors, more than 20, don't break into the ranks of highly cited papers, which may indicate 'big science' is rarely 'impactful science.'
A correlation matrix is another standard technique for looking at patterns in data. We see no particularly strong correlations, with the exception of times_cited and velocity, which makes sense since velocity is a derived statistic. The number of authors ia also negatively correlated with the first_author being a member of the research group, which makes sense, since only one scientist can be first author, and that is less likely on large papers.
A correlation matrix is another standard technique for looking at patterns in data. We see no particularly strong correlations, with the exception of times_cited and velocity, which makes sense since velocity is a derived statistic. The number of authors ia also negatively correlated with the first_author being a member of the research group, which makes sense, since only one scientist can be first author, and that is less likely on large papers.
Now we can begin to get into some hypothesis testing. Let's check boxplots for SDI before and after the grants were awarded. Post grant SDI scores are higher. A little data manipulation shows us that post grant SDI scores are higher for every group.
Student's t-test is a standard check to see if two samples were drawn from the same distribution. Running it on each pair of pre and post grant groups, we find with high p-values that SDI increased after the grants were awarded.
T-test : p-value
Group A: 0.008334
Group B: 0.000000
Group C: 0.008278
Group D: 0.002501
Group E: 0.000432
In short, the IGERT grants accomplished their state science policy goals of increasing interdisciplinarity in science, and we can prove it mathematically!
Finally, let's make a little bokeh graph to explore the relationship between SDI and velocity. One thing become immediately clear. Group C (in red) dominates, both in terms of total publications, and in terms of high velocity publications. No other group has a publication with a velocity higher than 50. (edit: can't get the interactive version to upload-static image instead)
T-test : p-value
Group A: 0.008334
Group B: 0.000000
Group C: 0.008278
Group D: 0.002501
Group E: 0.000432
In short, the IGERT grants accomplished their state science policy goals of increasing interdisciplinarity in science, and we can prove it mathematically!
Finally, let's make a little bokeh graph to explore the relationship between SDI and velocity. One thing become immediately clear. Group C (in red) dominates, both in terms of total publications, and in terms of high velocity publications. No other group has a publication with a velocity higher than 50. (edit: can't get the interactive version to upload-static image instead)
And finally, a little light reading for the week. As always, data and code is available at https://github.com/mburnamfink/scientometrics_101_project
RSS Feed
