This is a great talk on the use of statistics. Mona Chalabi is a data journalist, and here she outlines three ways of questioning statistics, based on her assessment that “the one way to make numbers more accurate is to have as many people as possible be able to question them.”
The three questions she provides were, I thought, generally quite obvious; as a teacher of information literacy they echo quite substantially the sorts of questions I encourage my students to ask. But there were two points that she made that I thought were really important and relevant to EDC, especially algorithmic cultures.
The first was about overstating certainty and how statistics can be used in a way that makes them describe situations as either black or white, with little middle ground. Sometimes this is a result of how they’re collected in the first place, and sometimes it’s how the statistics are communicated, and sometimes it’s in how they’re interpreted. I think this is one of the reasons that I’m hesitant about learning analytics; its innate tendency towards what can be quantified might lead to an overestimation of certainty, either in the way data about students is collected, communicated or interpreted. And, as we’ve seen, that data can become predictive, or a self-fulfilling prophecy.
The second point that I thought was really interesting was how Mona was responding to this situation of certainty. She takes real data sets, and turns them into hand-drawn visualisations so that the imprecision, the uncertainty, can be revealed. She says, “so that people can see that a human did this, a human found the data and visualised it”. A human did this, and so we anticipate uncertainty. Inherent here is a mistrust in the ability of technology to replicate nuance and complexity, which I think is misguided. But there’s also an underlying assumption about statistics – that a computer is able to hide the imprecision in a way that humans cannot. That computer data visualisations are sleek, while human data visualisations are shaky. This is a fascinating conceptualisation of the relationship between humans and technology, of the ways in which both humans and technology can be used instrumentally to make up for the weaknesses of the other.