Document Your Experiment

Recipes for Repeatable Experiments

Venkatesh-Prasad Ranganath

Jan 6, 2018·2 min read

1. Document the experiment in a README.md file.
2. Use markdown to write README.md file. This will help publish the experiment via services such as Bitbucket, GitHub, and GitLab.
3. Describe the purpose of the experiment.
4. Add a Requirements section describing the tools required to run the experiment.
5. Add an Instructions section providing instructions to run the experiment. If the experiment involves building tools, then include instructions to build tools. If the experiment involves specific external tools, then include instructions and pointers to get these tools. Don’t forget to mention the versions of the tools to be obtained.
6. Add an Output section describing the output generated by the experiment. Mention the names of the generated files, their format, and the meaning/purpose of the data in them. If the description of the generated output fits well with the instructions to run the experiment, then intersperse the description with the instructions in the Instructions section.
7. Add an Attribution section mentioning copyright, licensing, and author/contribution information.

Note:

1. This is the ordering of information that has worked best in my efforts.
2. The section names are suggestive. Use ones that fit your purpose.
3. If information can be captured better by spreading it around, then do so. See DyCo4j below.
4. Independent of section names, ordering, and such nuances, make sure all of the above mentioned information is captured.

Here are few examples:

• DyCo4J: Tool info, Output format
• Ghera: Repository info, Build Instructions
• Why do Users Kill HPC Jobs
• Identifying Lingering Security Posts from StackOverflow Data Dumps