I frequently have trouble getting my students to properly consider two alternative sources of error, systemic error and experimenter error. I would define experimenter error as mistakes in using measurement equipment and flaws in experimental design. This source of error usually gets plenty of representation in their reports. I consider systemic error to be fundamental aspects of the system or phenomenon being measured that introduces variance in the data beyond the control of the experimenter. Variation in development and behavior of various organisms in a study and sampling error are both examples of the second kind of error. I’m reading a terrific book that gave me an interesting idea of how to illustrate these two kinds of error to my students.
I highly recommend Daniel Kahneman’s Thinking, Fast and Slow. In it, he writes about a simple experiment. Take a blank piece of paper and use a pencil to draw a line up from the bottom center of the paper 5cm, without a ruler. Take a second piece of paper and draw a line down from the top until it is again 5cm from the bottom of the paper. Both lines should attempt to estimate the same location on the paper, but from different directions. Kahneman explains that the two lines will represent the upper and lower limits of your uncertainty for the location of 5cm from the bottom. Psychology of this result is interesting and is the result of anchoring in our estimates. The “take home” notion for us is that the two lines can be taken together as a measurement of your own error.
Students can then cut the smaller line out and glue/tape it to the page with the longer line. They should measure the space between the two lines. Ask students to then write about what the space between the two lines represents. From here, they can have many discussions:
- How do we make the gap between the lines smaller?
- This is a consideration of precision.
- How do we ensure that the actual 5cm point is within the gap more often?
- This is a consideration of accuracy.
- This is also a great discussion because the idea of “actual 5cm” is more concrete than the true population mean can usually be.
- What things push the location of the gap up and down?
- These things typically introduce experimental error. For example: poor timing techniques are rarely evenly distributed and will skew the results either up or down.
- What things make the gap wider or smaller?
- These things are typically magnifying or shrinking the impact of systemic error. Most of these issues can be reduced, but rarely eliminated.
- How can we communicate the properties of the gap to our readers?
- These things will address the appropriate content for an error analysis section in a report.
Give it a try and let us know how your students’ perceptions of error analysis were impacted!