Make meaning of the assessment. Engaging teaching staff with an analysis of their assessment results

Abstract

An appropriate analysis of student responses to multiple-choice type questions is a central part of reviewing whether there is an appropriate alignment between learning objectives and assessment tasks. There is a lack of familiarity amongst many academic staff about the process of analyzing these types of assessments, especially where a formal understanding of statistics is required. Many of the established literature methods are complex, and difficult for overworked academics to apply. We have analysed student responses to a number of multiple-choice assessments and examined different formats for presenting the resulting data to academic staff. The commercially available tool WinSteps was used to provide formal reports, and Microsoft Excel was used to present the data in alternative visual formats. The WinSteps reports provide a rich source of useful data about student responses, but this data can be difficult to interpret and is not particularly engaging for non-specialists. We have proposed a set of simple and effective formats that allow academic staff to visually identify individual questions that should be reviewed before being used in a subsequent assessment. Preliminary feedback from academic staff has indicated which formats were useful and where further development in the assessment reporting process would be of benefit.