Reproducible Research Working Group


CTSA Coordinating Center Project Manager

Mission Statement

The CTSA BERD Reproducible Research Working Group's mission is to
  1. list and discuss all areas related to challenges and solutions to reproducibility of biomedical research and its relationship to questionable research practices and ethical conduct of research
  2. compile and create educational materials on reproducible research methodology
  3. Find and develop technical solutions for data manipulation, analysis, and reporting and disseminate these to BERD members and beyond
  4. Develop and present short courses are web-based materials on how to implement reproducible research practices
The work of the RR Working Group is a scientific collaboration that will not involve the use of any proprietary data.

Suggestions for Working Group Activities

Templates Used by Brian Nosek's Group

Click to see Replication Templates

Discussion with Frank Harrell and Brian Nosek

How do we encourage reproducible research and the building of and sticking to pre-specified analytic plans? How do we assist medical researchers in not switching dependent variables when they do not get the results they wanted? How do we produce quality data analysis?

One suggestion for our work would be to do post-mortem analysis of published studies. Frank suggested that we build an archive of projects or compedium for statistical work. We could analyze the quality, review the study design, analysis, and measurements. In the psychology papers, Brian is looking at pre-specified endpoints and how predictive they are of reproducible studies. These brings us to the question of comparisons that we would like to make in reproducible research. Are we talking about looking at methodological or statistical comparisons of reproducible research?

Brian invited us to join the Open Science Collaboration discussion of Reproducible Research on Google (!forum/openscienceframework). This is a collaboration with individuals from a variety of fields, including non-scientists and non-psychologists, social scientists, cognitive investigators, methodologists. Frank suggested that our work should be more targetted. Should we be looking at biomedical or psychology studies and addressing a particular question, such as confirmation bias? Brian thought another good focus for BERD Reproducible Research projects would be methodology quality and observer variability.

Bottom line - what qualifies as reproducible? We would like to downplay point estimates. Shouldn't we be looking at the smaller confidence intervals in two studies?

Brian spent the year at Stanford and will be returning to University of Virginia in July.


24 Nov 2010

Attending: Dunn, Ware, Kloke, Thisted, Lindsell, Harrell

12 Apr 2011

Attending: Lindsell, Szabo, Thisted, Schumm, Harrell

16 Nov 2011

16 Nov 2011 and Later

This topic: CTSpedia > WebHome > ReproducibleResearch > ReproducibleResearchGroup
Topic revision: 27 Jan 2022, WikiAdmin
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback