Situational Judgement Test (SJT) pilots have taken place at a number of medical schools in order to test the viability and reliability of the SJT as a method of assessment. This was in addition to the Parallel Recruitment Exercise (PRE), which was undertaken in order test the logistics of SJT delivery ahead of live implementation for Foundation Programme 2013.
Micro pilots
The piloting stage initially started during March and April 2010 with some SJT micro pilots, involving a small number of medical students at four medical schools; Cambridge, Manchester, Sheffield and UEA. Students were presented with a number of sample SJT questions and asked to complete them under exam conditions. A focus group was then held where participants shared their views on the item content, format and structure. In general, the feedback in relation to SJTs was positive. This feedback was then analysed and shared with the Project Group to help inform the next stage in the piloting process.
Large scale pilots
Between October 2010 and April 2011, pilots took place at seventeen UK and two non-UK medical schools involving over 1,000 students. Final year medical students were asked to take a two hour invigilated SJT, answering 60 questions. Feedback on five of the items was provided following the SJT.
The pilot data has been analysed and the results show that the Situational Judgement Test is a reliable, valid and appropriate method for selection in this context. These findings support the considerable published research on SJTs.
The analysis and evaluation of the design, development and piloting work in 2010-2011 can be found on the report pages.
Parallel Recruitment Exercise (PRE)
A full-scale pilot, or PRE, took place during the 2011/2012 academic year for all applicants to the Foundation Programme 2013. More than 90% of applicants took part, which enabled us to ensure that the secure, reliable and robust processes are in place ahead of implementation for applications to FP 2013.
Information on what the PRE involved and the full analysis and evaluation can be found on the PRE pages and in the Final Report of the PRE.