Cato Tops New CGD Index of Think Tank Profile
This Thursday, the World bank will host the unveiling of the latest edition of the best-known ranking of think tanks, which is produced by the University of Pennsylvania. The public event will reveal whether the Brookings Institution has lost its hold on “Think Tank of the Year,” which tanks made the top 50 worldwide, which are best in Latin America, and so on.
As with the Oscars, the verdicts of the Global Go To Think Tank (GGTTT) Index are rendered on the basis not of performance measurement, but on the perceptions of those in the business, in this case hundreds of journalists, policymakers, and think tank employees. That approach may be one reason expert perceptions of the GGTTT index itself have tended to be highly critical (here, here, here, here, here). Among the concerns: the opacity of the ranking process, the inclusion of institutions that are not think tanks in any usual sense of that term, and fundamental doubts about what it means for a member of such a diverse class to be “the best.” Yet there’s no doubt the results turn heads each year. If the criticisms from think tank experts are right, then the GGTTT may be distorting the behavior of think tanks, as they strive to raise their standing on dubious metrics, as well as misleading think tank funders.
The combination of continuing criticism and continuing interest made us wonder: could we do better, or at least illustrate the possibility of doing better? Last November we posted indicators of think tank “profile”, looking at how often a tank’s work is reported, cited, downloaded, or followed. Here, after tweaking and updating the indicators, we blend them into a single index for ranking, which we easily do by drawing upon methods we honed over 10 years for the Commitment to Development Index.
Let us be clear: CGD does not intend to enter the tank-ranking business long-term. As a think tank, we have a dog in this fight and lack the necessary objectivity. Indeed, we acknowledge that our focus on public profile may bias the results in CGD’s favor, since public outreach is central to our strategy. Moreover, as we wrote last time, web page hits, media mentions, and scholarly citations are just a subset of the characteristics that can make a think tank effective. Thus the “profile” in our title. Some tanks succeed precisely by flying below the radar. Our purpose is to stimulate and improve the discourse around think tank performance.
These caveats notwithstanding, like many in the think tank community, we are data geeks, committed to the use of objective evidence in our research. We believe that any effort to rank the tanks should begin by gathering the best available data. In the spirit of offering some much-needed competition to the GGTTT, let’s take a closer look at the data.
We start with an update of the last post’s first table, covering noted US institutions. Major changes in our data collection include a switch from Google News to Nexis for media mentions counts, the latter having proved more stable from week to week; the use of academic citations of papers published in 2010 instead of 2012, since 2012 is too recent for much mention to have accrued; and the exclusion of Human Rights Watch and NBER as not being think tanks despite being listed as such in the GGTTT: MORE
By :David Roodman and Julia Clark.
No comments:
Post a Comment