Experiment Methods Template

This template lists the minimum information that should appear in a well-described methods section. It can act as a checklist for authors before submission, and it can serve as a reference for reviewers to request a revision with more information.

> View the template <

Why a thorough writeup of methods is necessary

The goal of a methods section is to provide all the information needed to fully replicate an experiment without any help from the original author. Incomplete and ambiguous reporting of experiment methods can also make it challenging to understand analyses, interpret results, and critically evaluate conclusions. Furthermore, vaguely described procedures and parameters can be highly frustrating for readers, reviewers, and replicators. Thoroughly describing methods is a critical component of high quality science.

Questions

Who is this for? This guide is primarily for experiments with visual stimuli – common in fields such as vision science, cognitive science, human factors, human-computer interaction, and data visualization. However, it can likely be adapted for other types of experiments too.

Is this like Pre-Registration? A methods section describes what was done (past tense). A pre-registration describes what authors intend to do (future). Many factors can cause experiment plans to change, and aspects like the number of dropped subjects can’t be predicted. So while this report and a pre-registration have much overlap, they are not the same.

How should this information be presented? Although narrative prose can be a useful medium for presenting an argument, it can be inefficient and ambiguous for describing experiment methods. When possible, use bulleted lists in a manuscript to clearly convey sets of conditions and parameters, and use numbered lists for sequential procedures.

Are figures necessary? Example figures are critical for understanding visual stimuli, as text descriptions are often ambiguous or difficult to understand (e.g., “The objects in the outer ring of the target region to the left side of the space after the cue are a different color than the distractors presented in the lower right quadrant of the cue”).

  • An example image should be shown for every stimulus condition.
  • A diagram of the trial procedure can substantially improve clarity.

Is this everything needed in a methods section? Not necessarily. This template should be though of as a minimum set. It represents the smallest acceptable amount of information needed to understand an experiment. The goal is for a reader to be able to directly replicate an experiment as precisely as possible, so any other information needed for such a replication should be included.

Can all/some of this information be put in supplemental materials? No. All of this information is critical to understanding an experiment sufficiently to replicate it.

Terminology

Unusual terminology regarding the breakdown of trials within an experiment can cause a lot of confusion, especially in determining how many trials were presented. Stick to common standardized terminology when possible:

  • Experiment
  • Session - each time a subject returns for more data collection
  • Block - A group of similar trials
  • Run - An uninterrupted sequential set of trials
  • Trial

Thanks for helpful feedback: Ben Wolfe, Bernice Rogowitz, Caitlyn McColeman

Github repository for any issue or requests

.