reports but also books, working papers, and business and government reports.19 Google Scholar also includes non-scholarly citations of articles, which is important since policy reports are not intended for publication in academic journals which populate typical citation databases such as Web of Science. Particularly, since objectives of policy reports include informing governments and the development community, it would not be appropriate to limit the citation count to scholarly databases20 that exclude government and think tank publications. Google Scholar gathers bibliographic information by crawling through websites and fetching HTML and PDF information for the articles, which allows for real-time updates of citations.21 Unlike other databases, Google Scholar does not wait for a paper to be formally published before it fetches its data from working papers and includes it in its database. Using Google Scholar for citation counts is not without problems. First, its automated software sometimes does not detect false positives or double-counting. For example, a working paper that has had different versions uploaded to several repositories could have the citations within each working paper counted separately, inflating the citation count for these articles. Furthermore, Google Scholar did not identify those reports that were ultimately published under a new title, either as a book, a chapter in a book, or as a working paper. Second, it does not have a large coverage of those older, pre- digital publications unless these publications have been processed with optical character recognition (OCR) software. Given that our sample only covers fiscal year 2008 through fiscal year 2012 this last issue should not pose a problem, although there are some policy reports within this period that were scanned documents converted with OCR software. Third, citations data in Google Scholar is susceptible to manipulation as any search algorithm would be (Delgado López-Cózar et al., 2012), but it is the sentiment of the authors that this is not an issue since policy reports are not produced with the main objective of being cited. Fourth, a report may not receive citations simply because it is not included in the Google Scholar database. Using data from the World Bank’s Open Knowledge Repository (OKR), we verified whether the policy reports in our sample were definitely included within the Google Scholar database. Within our dataset, 410 policy reports were not listed within Google Scholar.22 We can assume that these policy reports are not cited, but for the purposes of our analysis we perform regressions with these missing values removed from the data. Citation counts are much lower than download counts (Figure 7). Almost 88 percent of the policy reports (1,054 out of 1,201) in our sample were never cited.23 Few reports received more than 20 19 It does not include newspaper or magazine articles, blogs, editorials, or reviews. 20 Prominent sources for citation counts are the Library of Congress’s database, Elsevier’s Scopus database, and Thomson-Reuter’s Web of Science database. The Library of Congress includes only general bibliographic information for all types of books and academic articles in their database, and does not have any information on the citations within the article nor citations of the article. The Scopus and Web of Science databases covers only serialized publications and conference proceedings, and the journals included in each database differ. Scopus includes 19,809 journals and Web of Science includes 12,311 across all academic disciplines. Between the two there are 11,377 shared journals (Center for Research Libraries, 2013). Both include counts of citations by other articles, but it is limited to citations by articles from within the respective databases. 21 We auto-fetched citation count results using specialized scripts within the open-source program Zotero. There were constraints to fetching the data based on the nature of Policy Reports: Typically Policy Reports are not published under individual author’s names, so we exclusively had to rely on the title of the document. 22 Thirty-three reports were not listed in Google Scholar according to OKR, yet had citations within Google Scholar. These reports were included in the data base. 23 By comparison, Ravallion and Wagstaff (2010) find that nearly 30 percent of World Bank research publications have never been cited. 14

Which World Bank Reports Are Widely Read? - Page 20 Which World Bank Reports Are Widely Read? Page 19 Page 21