Jose Mestre, Carolyn Anderson, Hua Hua Chang, Gary Gladding and Katherine Ryan are leading a multidisciplinary effort that combines psychometrics and assessment, methodology, and science education to help retention of students in science and engineering. Findings from previous research show that lower performing students do not accurately predict their scores on exams. Such students tend to overestimate their performance, whether they predict performance before or after the exam. This unjustified optimism seems to reflect a tendency to confuse familiarity with the material and competence with the material. This is a critical problem because poor performance in introductory physics leads to poor retention of students in the engineering and science fields.
Prof. Mestre and colleagues are tackling this problem from multiple perspectives. First, they are using advances in computer adaptive testing (CAT) to diagnose students’ problem solving and conceptual deficits prior to taking high-stakes course exams in an introductory physics course. The goal is to build a cognitively diagnostic computer adaptive testing (CD-CAT) tool that accurately predicts students’ future performance on course tests prior to their administration. This use of CAT for diagnostic purposes in a science field is novel and has not been tried; the CD-CAT approach will home in on students’ strengths and weaknesses prior to the real course exam, and provide detailed diagnostic information that students may use in targeting their studying to remedy weaknesses. In addition, the investigators are devising and evaluating interventions to address students’ weaknesses. Among the interventions are multi-media web-based presentations of solution strategies for an array of problem types in which students show weaknesses.
Prof. Mestre’s work is funded by the National Science Foundation.