Authors:
Carlotta Bonn
Dieter Plehwe
Moritz Neujeffski
Both think tanks and rankings have gained tremendous popularity in the new millennium. It made sense to subject the rapidly growing global community of think tanks to greater scrutiny and to make the results available to the general public. Prof. James McGann of the University of Pennsylvania saw the opportunity and created the Global Go to Think Tank Index back in 2006. Ever since, the report is published on a yearly basis. Based on expert judgement, the index does not rely on data that can be empirically validated. While many experts welcome the greater publicity and hail the index, it has also been subject to a fair amount of disapproval due to methodological shortcomings and visible asymmetries and bias. Notwithstanding the criticism, the index has remained the only comprehensive source of information on think tanks around the world and across policy fields. In a series of three articles we will try to help better understanding the index and its impact. Rankings and ratings are hard to measure and assess in general. Some attest considerable power to global performance indexes of countries like the UN Human Development Index, the World Bank’s Ease to do Business Index or global perception indexes like Transparency International’s Corruption Perception Index. Indexes provide easy to use information for data hungry journalists and create publicity at the event of publication. But attention to indexes is not uniform and the results are subject to more or less controversial debate.
Indexes targeting organizations allow users to observe the status of a university or university department or a business school, for example. Parents, future students and alumni get a chance to observe the relative position of the schools they like. Unlike performance indicators, perception indexes reflect general judgements – or prejudices – experts hold and are not necessarily based on solid evidence. Who can claim to know if a few organizations in different parts of the world are inferior to Brookings, the long standing number one think tank according to the Global Go to Think Tank Index? In the case of think tank indexes and reports, who has an interest in the status of think tanks and in the comparison? In the following three part series we look at the way in which the Global Go to Think Tank Index appears in 1) the newspapers, 2) online media and 3) academic publications over the years to gain a better understanding of the use and relevance of the data. Looking at the impact and repercussion of the index allows us to see the Global Go to Think Tank Index in Perspective, to neither under- nor overestimate the leading – if only – index in the field.
How does the Global Go to Think Tank Index Report appear in the print media?
The Global Go to Think Tank Index Report (GGTTIR) has been published annually since 2006 by the Think Tanks and Civil Societies Program of the Lauder Institute at the University of Pennsylvania in Philadelphia. The director of the program is Prof. James McGann. The report provides comprehensive information on a wide range of think tanks across the world. Think tanks are ranked according to perception of their work in general (top think tank), according to the perception of their work in particular policy fields and according to world regions and individual countries. It is fair to say that the index has advanced to a key reference for think tank pundits, journalists and academic think tank researchers. Although there are a few other efforts to assess think tanks including Transparify’s financial transparency index and the prizes awarded to think tanks by Prospect Magazine and Observatoire des think tanks, the Global Go to Think Tank Index has contributed most to raise awareness about the sprawling think tank community.
The Index is very controversial at the same time for methodological reasons. The principle concern is lack of hard evidence in support of assessments and rankings because it is based on subjective nominations and rankings of so-called experts. A perception index is not evidence based even if Prof. McGann’s team invests some energy to control the results (McGann 2019: 29). McGann’s team is small and U.S. based, however, which makes it difficult to detect problems in the many places around the world the index covers. It is also well known from Transparency International’s corruption perception index, for example, that perceptions do not change much regardless of the real life performance of a country or organization. In the case of organizations we need to consider additional flaws. Since those who are asked to assess the think tanks are leading individuals in think tanks, the system is subject to gaming: If you scratch my back, I scratch yours. Due to considerable inconsistencies and obvious mistakes (think tanks ranked in categories although they did not contribute anything to the area), a strong, albeit weakening U.S. bias and a still visible neoliberal / libertarian bias, the Go to Think Tank Report has to be read with caution (see: Seiler, Wohlrabe 2010; Daniel Florian 2011; Chafuen 2019)
Since the Go to Think Tank Index remains the only global effort to assess and compare think tanks across space, time and policy fields it is nevertheless interesting and informative to subject the reception of the report to further scrutiny. We set out to find answers to simple and straightforward questions: how often and in which ways does the GGTTIR appear in the media? While global performance indicators like the Human Development Index or the Global Competitiveness Index are designed to observe and compare countries in order to push for reforms (technology of persuasion according to Theodor M. Porter), we start the analysis of the media repercussion of the GGTTIR assuming the most important purpose of rankings of think tanks or universities to be the publicity they generate – for some of the organizations in the field at least.
In this first blog post, we take a closer look at the GGTTIR in the traditional media. How many articles mentioned it? Did the number of articles rise over the years? Where are these articles from? And last but not least what are the articles about?
Newspaper Sample
To answer the questions mentioned above, we used data of NexisUni and Factiva and found 273 articles. The number is surprisingly low given the global scope of the index and the number of years it has been published. The majority of the articles is in English (219), complemented by just a few articles in Spanish (10), French (7), German (12), Italian (4), Russian (14), Portuguese (2), Arabic (3), Bulgarian (1) and Malaysian (1). The English articles are from 36 different countries, however. Most of them are from the US (61), India (30) and China (23). The media impact thus seems to resonate with the American origin and the asymmetries characterizing the think tank field at large.
We searched in the period from 2008 until present (08/2019) but the first articles were published in 2014 and even then it was a small number of seven articles only. After that, the number per year grew slightly. In 2019 49 articles were published with four months remaining to the end of the year at the time of publication of this blog post.
50 articles per year can still not be considered a large number. Contrary to our expectations and despite the global data source, the first result is that the GGTTIR is not a frequently discussed topic in the traditional news media.
Geographical distribution
Nevertheless it is still interesting to know where the articles came from. At first we checked the continents and found out that the largest number of articles were from Asia (131) followed by North America (68) and Europe (39). Only 26 articles were from Africa, 5 from Oceania and 2 from South America.
In the next step, we took a closer look at the countries. The articles are from 49 different countries. The countries with the most published articles are the USA with 61, India with 30 and China with 23. 15 articles are from Pakistan and 11 each from Germany and Russia. The remaining countries have all published 10 or less articles. For 19 articles we could not find a country so they were categorized as “no country found”.
The prominent role India plays among the Southern countries is likely due to role India plays as HQ of the Global Development Network, a global umbrella organization of think tanks and experts.[1]
Classification of content of articles
In the next step we skimmed the articles to get a general idea of their content. We classified them in eight categories. The largest category mentions the ranking as a seal of quality for the think tank that appears in the article (91). The second largest category is made up of articles about a specific think tank, which was ranked in the GGTTIR (88). Third largest is the category of articles that report about a country and the think tanks from this country, which were ranked in the GGTTIR (26). The fourth category is about think tanks in general (25). The other categories are pretty small. Only eleven articles report about events on the occasion of the launch of the ranking and only three mention the ranking as a data source for research. The last two categories are one that is made up of articles that express some critique on the index (5) and a residual category of all the articles that do not fit in the other seven categories (24).
Except for the five articles that contain critical information with regard to the index, all the articles report either very positively about the GGTTIR or they are neutral and just mention the ranking positions without any evaluation. A high position in the ranking is often seen as a success of the ranked think tank and many highly ranked think tanks in one country are seen as a success of the country.
“Being recognized by the Think Tank and Civil Societies Program is significant because the rankings provide us with a benchmark to evaluate ourselves in relation to other regional institutions. It helps to highlight both our accomplishments and our opportunities for development.” Dr. Natasha Ridge, the head of research at the Al Qasimi Foundation, a think tank from the United Arabic Emirates said.
Conclusion: summary of findings and general assessment
All in all we can say that the Go to Think Tank Index Report is not frequently mentioned in traditional media between 2008 and 08/2019. It took a long time to find some resonance and with 50 articles per year around the globe (plus those we are missing!) it has a modest presence in newspaper only. There are a few countries inlcuding the United States, India and China where we found a somewhat stronger presence of the GGTTIR in newspaper articles than in the rest of the world, but even there it is evidently not an important topic. We expect to get a different result when we are looking at the online media next.
Nevertheless it is interesting that a large number of articles mentioned the GGTTIR as a seal of quality for the ranked think tank which could be seen as a confirmation of our basic assumption that the main purpose of the ranking is to generate publicity for think tanks, and most importantly for those that are highly ranked.
To find further information whether this is true or not, we will look at the online media coverage of the GGTTIR in the next blog. Apart from online media we will examine if and how tanks refer to the GGTTIR on their own websites.
[1] Further reading: Diane Stone: Banking on Knowledge: The Genesis of the Global Development Network, London, Routledge, 2000, Dieter Plehwe, A global knowledge bank? The World Bank and bottom-up efforts to reinforce neoliberal development perspectives in the post-Washington consensus era, in: Globalizations 4 (4), 514-528
