By Eduard Klapwijk (SURF) and Sven van der Burg (Netherlands eScience Center)
One of the most important reasons to share your analysis code is to enable others to reproduce your research results. But not every piece of code that works on your machine will routinely work on someone else’s laptop or computing environment. How to make sure that your code is indeed reusable for others? In this blog post, we will present some steps that you can take to increase the chances that your code – and consequently your research results – will reproduce beyond your specific working environment.
What is reproducibility ,and why is it important?
The definition of reproducible research that we use here is nicely explained in the Turing Way handbook as work that can be independently recreated from the same data and the same code that the original team used. As a consequence, the data, code, and methods need to be available in full detail to make recomputation of the results possible. Such a level of transparency in itself will make your research more reliable, because it allows the work to be examined by others. However, there are situations where you are not allowed to redistribute the data: in those cases, make sure to be as specific as possible in citing the exact datasets that need to be accessed (and possibly, how to do that) to run your code.
When others can reproduce the results, it does not guarantee the findings to be true, but it provides a minimum condition for the findings to be believable and informative. And vice versa, if someone else cannot recreate the same or at least similar results independently, we cannot trust the results in the first place.
Thus, the availability of reproducible analysis and software code is critical because it allows others to evaluate the correctness and trustworthiness of research results. Sharing code is also important because it is a first-class research object that increases transparency of the research process and can be reused by others.
Steps to increase code reproducibility
Make your code available
Obviously, your code should be publicly available somewhere to be re-run or to be simply examined by reading through the code. A good start is to publish your code on a public git platform like GitHub (this is the most popular platform; GitLab or Bitbucket are major alternatives).
There are other benefits to publishing your code there. Your project will be under version control. This means that you and others can, from now o,n track the exact history of changes. You will also have an external copy of your code on the online git platform. If, for some reason, you lose your local copy, you can always ‘clone’ the repository on GitHub or another platform back to your local system (to continue working there).
In addition, make sure you also publish a copy of the git repository on a research repository like Zenodo, where it will receive a persistent identifier. You can also find more tips on this in one of the previous ODISSEI FAIR blog posts. This will help to avoid link rot and make your project more FAIR. And if your code runs on CBS microdata or LISS data, do not forget to submit it to the ODISSEI Code Library.
Try to avoid dependency hell
Now that your code is accessible to others, there might be someone who downloads or copies your code and wants to run it on their own machine. Exciting! But there may be problems around the corner because our code often depends on other code that in turn depends on other code. If something is missing or has changed in that chain of dependencies, the code will break.
One of the things you can do to avoid this so-called dependency hell is to clearly record dependencies on other software (with version numbers). There are also many tools to record dependencies, which will help you to create a time capsule of dependencies that can be re-created. Tools like venv, conda or poetry for Python and renv for R will help you to manage virtual environments in which specific libraries or dependencies are isolated and installed for your specific project. Using these tools, you can easily delete and re-create your environment, which is much better than debugging. The easier someone can re-create your environment, the more reproducible it is.
Document your code
Barebone code without any explanation is hard to follow, even for an experienced programmer. Documentation of your code is, therefore, key to explaining what it is for and what it should be doing.
One general piece of documentation to accompany your code is a README file. It is usually the first thing a user or collaborator sees in your project. It is a text file that introduces the project, explains how to set up or install the code, and how others can reuse your materials or contribute if they want. You can head over to the website makeareadme.com for a short explanation and template.
Within the contents of your code, it is also very helpful to include comments as a form of in-code documentation. You can add short human-readable sentences to explain what certain chunks of code are doing. But keep in mind that in many cases, naming itself is documentation: Giving explicit, descriptive names to your code segments (functions, classes, variables) already provides very useful and important documentation.
Coding conventions
Readable code – for others and our future selves – should be descriptive, clean and consistently formatted, and use sensible, descriptive names for variables, functions and modules. In order to help you format your code, you can follow guidelines known as a style guide. A style guide is a set of conventions that we agree upon with our colleagues or community, to ensure that people produce code which looks similar in style. The most important thing about a style guide is that it provides consistency, making code easier to read and also easier to write – because you need to make fewer decisions. Examples of style guides that you can use are PEP 8 for Python code and the Tidyverse style guide for R.
Modular coding
Modularity refers to the practice of building software from smaller, self-contained, and independent elements. Each element is designed to handle a specific set of tasks, contributing to the overall functionality of the system (see these slides from the Netherlands eScience Center for more details about modular coding).
One of the best ways to improve your code and to make it more modular is to write functions. Functions allow you to automate common tasks in a more powerful and general way than copy-and-pasting. Writing a function has four big advantages over using copy-and-paste:
- You can give a function an evocative name that makes your code easier to understand.
- As requirements change, you only need to update code in one place, instead of many.
- You eliminate the chance of making incidental mistakes when you copy and paste (i.e. updating a variable name in one place, but not in another).
- It makes it easier to reuse work from project-to-project, increasing your productivity over time.
A good rule of thumb is to consider writing a function whenever you’ve copied and pasted a block of code more than twice (i.e. you now have three copies of the same code). In your own project, you can identify code that would fit better in a function by looking for pieces of code that you repeat throughout your project.
Further improvements to your project
By implementing the previous steps in your project, you already have a project that is much more reproducible and reusable than most research projects. A few more things that you can do are:
· Checking how FAIR your project is and what you need to do to improve it. For example, you can add the howfairis badge to your README file.
· Adding information about how to cite your project and using cff-initializer to create a CITATION.cff file for your project.
· Adding a license to your project. You can use https://choosealicense.com/ to find a suitable license for your project.
· Adding data to your project. Make sure you are allowed to publish the data (most importantly, it should be de-identified in the case of human participants). Publish the data in a data repository and include the link to your data set in your GitHub repository. Alternatively, you can include a data file in your GitHub repository. In case you are unable to share the data, include dummy data in the project.
Have your code checked
If you took a lot of steps to make your research reproducible, then definitely have it codechecked by someone else! This will boost the credibility of your work (you can receive a nice reproducibility certificate) and will also help to check errors at the right time, if you do it before publication of your results. A check by another pair of eyes is the best way to learn how reproducible and reusable your code is.
If you are allowed to share the data, you can consider submitting your data and code to Reprohack or CODECHECK (you can read more about the experience of getting codechecked in this blog post). Even if you don’t submit your project for a check, it would be helpful to take into account their guidelines: both initiatives emphasise that documentation of your code is key!
Acknowledgements
This blogpost is based, for a large part, on the Reproducible research through reusable code workshop by Sven van der Burg and Eduard Klapwijk. In turn, those workshop materials are based on the Code Refinery Reproducible Research lesson about dependencies, Modular Code Development from Good practices in research software development, Functions explained from R for Reproducible Scientific Analysis Software Carpentry lesson, Functions chapter from R for Data Science (2e).
Picture by Philip Oroni For Unsplash+