How relevant is climate change research for climate change policy? An empirical analysis based on Overton data

or challenge an idea) among others, suggesting the need to understand the semantics of the policy science citations” (p. SI). This study focusses on the connection of climate change research and policy. The study is based on data from the (new) Overton database including policy documents (10,846 climate change policy documents covered in the database) and their citations of scientific publications. With this study, we followed other studies using Overton data investigating links between pol- icy and research (e.g., on COVID-19). Although the Overton database captures a large collec- tion of policy documents, potential biases in coverage and data sample cannot be excluded [11]. For example, the Overton providers will not have access to many governmental archives, and if they have access, it will be restricted to only a part of the existing documents. Other shortcomings of Overton are mentioned by Yang, Huang [9]: “the metadata of such policy documents cannot reveal the semantic information contained in the policy process. At the same time, some policy documents have unstructured features, so attribute identification and labeling may be required”. Overton uses a very broad definition of policy documents, i.e., “documents primarily written by and for policy makers”. The idea behind this is to cover not only text that documents the pol- icy or legislation itself in the corpus, but also documents that were written to inform or influ- ence decisions. Our analyses do not distinguish between those two fundamentally different

classes of policy documents. Documents written for policymakers are often written with the purpose to inform or influence documents authored by policymakers and are as such funda- mentally different from documents authored by scientists. Moreover, under this wide umbrella definition there are very different types of documents: scientific assessments by the scientific community, legislations, policy reports by IGOs and NGOs, policy briefs, speeches etc. The different nature of these documents explains some of the results here. For example, it is the main purpose of scientific assessments as those by IPCC to assess the state of knowledge in climate change research and inform international climate diplomacy and national climate pol- icy with robust evidence. In nature, these assessments are comprehensive reviews of the litera- ture with tens of thousands of references. On the other hand, policy briefs are designed for communications and often deliberately strip out literature sources. The policy impact analysis in this study, therefore to some extent simply highlights different policy document types. Any interpretation of policy impact of research can only be undertaken based on such an important caveat. In this study, we empirically targeted several aspects of the connection between climate change research and policy. Focusing on the time trend of this connection reveals that the dis- cussion of climate change in policy seems to have had its peak some years ago. Although we suspect a lot of uncertainty related to the coverage of policy documents in Overton, there

seems to be an impact of international climate policy cycles on policy document publication. We observe local peaks in climate policy documents around major decisions in international climate diplomacy. For example, we observe temporal peaks in policy documents around the failed Copenhagen Summit in 2009 and the Paris Agreement; there is a growth in policy docu- ments from IPCC’s Fifth Assessment in 2013/2014 with a peak in 2015 when the Paris Agree- ment was made. IPCC reports might play a particular role as they are usually released 2–3 years ahead of major international climate diplomacy events and could trigger substantial co- publication activities. In 2023, the first Global Stocktake on progress with the Paris Agreement is scheduled with IPCC AR6 being released during 2021 and 2022. We might thus expect to see increases in climate change policy documents and citations to the scientific literature in the 2–3 years following. Various types of institutions publish policy documents. Our results point out that IGOs and think tanks–with a focus on climate change–have published more climate change policy PLOS ONE How relevant is climate change research for climate change policy? PLOS ONE | https://doi.org/10.1371/journal.pone.0274693 September 22, 2022 20 / 24