Presidents’ Thelen and Smith response to Women’s Caucus petition on Data Access and Research Transparency (DA-RT)
We write in response to numerous informal queries, as well as a formal petition from the Women’s Caucus in Political Science, requesting that we clarify our positions on data access and research transparency standards in the discipline. We do so to express our own shared judgments, not any official position taken by the APSA. But we do so because we take very seriously the concerns these standards raise, among other things about the potential for DA-RT to work to the disadvantage of younger scholars, colleagues at under-resourced institutions, women political scientists, and scholars of color.
The discussions concerning DA-RT within our association were prompted in large part by a decision in 2016 on the part of several prominent journals to embrace new requirements in the area of data access and research transparency (see the “JETS” statement here). Members who have been following these debates will know that these new policies have caused considerable consternation and concern, particularly but not exclusively within the community of scholars associated with qualitative research, broadly defined. Until October 2014, the principle of transparency in research (under the APSA Ethics guidelines from 2012) had been treated as a matter of professional ethics. It was up to scholars themselves to determine how best to apply this vital principle in their own work. The JETS statement shifted this balance of responsibility considerably — from a regime of self-enforcement by individual scholars and communities of scholars as an ethical matter to a set of requirements that these editors took responsibility for policing.
Many of our members viewed this as a very consequential change and one that had not been adequately discussed among the various constituencies whose work and whose careers would be affected by it. In response to these concerns, the Qualitative and Multi-Method Research section of our association agreed at their 2015 business meeting to “initiate a process of deliberation of transparency in qualitative research” and asked section members Tim Büthe and Alan Jacobs to design and shepherd the process. Büthe and Jacobs assembled a diverse steering committee to advise and accompany their work. The resulting “Qualitative Transparency Deliberations” (QTD) took seriously the ambitions of the original proponents of DA-RT, who rightly insisted on a “‘community standards’ approach, where optimal means of data sharing and research transparency respect and build from the challenges and opportunities that characterize various research traditions” (Lupia and Elman 2014: 20). The core idea of the QTD process was to establish a forum within which the communities of scholars from various research traditions under the broad umbrella of qualitative research could deliberate among themselves to arrive at understandings of research openness that would be appropriate for their own research community.
Now, after nearly two years of open deliberation, the formal QTD process is complete. The draft statements of the various working groups are available here (and with final versions to be posted soon). We thank the QMMR section and Büthe and Jacobs for their leadership in administering such a high quality set of deliberations. The discussions that came out of the QTD process brought to light some potential benefits from the adoption of formal data sharing and transparency requirements. It also surfaced many concerns. Some of these concerns are specific to the various qualitative communities, but many are not. The deliberations have heightened our sensitivity to the uneven effects of new data access requirements, which place greater burdens on scholars in less-well-resourced universities and colleges. They have also alerted us to worries about the ability of younger scholars to benefit fully from the value of the investments they often make in primary field research, including developing their own quantitative data sets. They have educated us about issues surrounding the need to protect human subjects, the particular challenges faced by scholars who work in unstable (e.g., authoritarian, conflict, or post-conflict) settings, and the potential disincentives to undertake certain types of research in the first place. They have brought to our attention the difficulties that especially younger scholars – often under great pressure to publish – sometimes face in pushing back against pressures by editors and/or reviewers to share data in ways that stretch them beyond their own sense of ethical responsibility to their subjects. These are just a few of the concerns raised in the deliberations. We invite members to consult the QTD site for more of the rich debates that have unfolded there.
The leaders of the various QTD working groups have produced statements that summarize the results of their own deliberations, drawing on the wide group of scholars who participated in the deliberations. One can read in some of these statements an emerging consensus on some widely accepted norms, such as the ethical primacy of the principle of human subject protection, but also broadly agreed recommendations such as precise citation practices and the importance of being as explicit as possible about how evidence was gathered and how the researcher arrived at his or her conclusions. However, many of these working group reports individually, and the group of reports taken as a whole, speak strongly against the imposition of uniform standards across all different types of research and methods. They also suggest a lack of consensus on other issues, particularly on data access requirements. For those who hoped that the process would deliver a set of “rules” that could be handed to journal editors for implementation, this will be a disappointment.
We are not disappointed. At a procedural level, the QTD process was the most open and inclusive process we have ever seen take place under APSA auspices. This is a major accomplishment in itself. Beyond this, however, our view is that deliberation need not always result in compromise or agreement, let alone consensus. Instead, and as Lupia and Norton (2017: 75) emphasize, deliberation can clarify differences: “If differences arise, perhaps they should remain: open and acknowledged.” We think this is such a case.
It turns out, for example, that as a group, political scientists often hold quite different understandings of research transparency than the principles laid out in the JETS statement. A number of QTD working groups understand transparency above all to involve openness toward their subjects and openness about the researcher’s own positionality and subjectivity.
In light of this diversity in understandings of what transparency ought to mean in our discipline, our view is one endorsed in January 2016 by Bingham Powell and 19 other former APSA presidents in their letter to 27 journal editors: we think it is important that leading disciplinary journals “stress openness to a variety of approaches to achieve the broader goals” of the transparency movement “and avoid research chilling costs.” We value and endorse our longstanding disciplinary tradition of granting the editors of our journals wide discretion in editorial standards and procedures. But we believe that in making these judgments, journal editors should focus their energies above all on ensuring that they are selecting reviewers who are themselves members of the appropriate research community to evaluate the articles that come to them, rather than mechanically applying the JETS understanding of transparency in all cases. Particularly with respect to the questions of data access and the protection of human subjects, we strongly urge journal editors to defer to individual authors, who ultimately possess the knowledge and expertise to assess the relative costs, benefits, and potential unintended consequences of releasing their data – and who also bear the ultimate responsibility for whatever consequences might ensue when data are made public.
To reiterate Lupia and Norton’s point, the outcome of deliberation need not be to overcome differences. The point of legitimate deliberation cannot be to force consensus; indeed, where differences persist, “it might be better to enshrine them institutionally… or to develop a modus vivendi that preserves these differences” (2017:75).
This is our position as well. For those communities where a consensus on data sharing or transparency standards exists, journals can certainly accommodate that consensus (as they do, for example, for certain types of quantitative analysis). However, where such consensus does not yet exist, we believe that openness to a variety of approaches to transparency will prove most helpful in cultivating the rich diversity of rigorous, productive research traditions that together define our Association.
Kathleen Thelen
President, American Political Science Association
Rogers Smith
President-Elect, American Political Science Association