Part 3 MRCOG May 2018 – examination analysis
31 May 2018
This statement is an update to that issued on 17 May 2018 (below) regarding the May 2018 Part 3 MRCOG examination.
Following the examination, an external, independent expert has conducted a psychometric analysis of the results, as per the RCOG's standard examination processes. This analysis provides us with important information about candidates’ performance across the examination and on each specific question, as well as examiner performance. The analysis included a comparison of candidates’ performance across the different days of the examination, and particularly days 1 and 2 as the tasks were repeated across these two days. The rationale for repeating tasks is set out in our previous statement below.
The analysis has demonstrated that there was no significant difference in the percentage of candidates who passed on any day of the examination. In particular, the pass rate between day one and day two of the examination was almost identical, with a difference of only 0.12%. In addition, the pass rates from the London and Singapore centres were almost identical. Importantly, each day’s examination scored well for reliability and internal consistency.
These results confirm the robustness and validity of the examination. We are currently in the process of generating results letters which will be available, as indicated previously, on 13 June 2018.
Part 3 MRCOG May 2018 – statement
17 May 2018
The quality assurance of the MRCOG examination is of fundamental importance to the Royal College of Obstetricians and Gynaecologists. This is a particular focus for the Examinations and Assessment Committee at this time.
After each Part 3 MRCOG Clinical Examination, the candidate, task and examiner marks are assessed using detailed psychometric analysis to ensure we have confidence in the examination and the examiners. As part of this quality assurance process the Examination and Assessment Committee agreed that on one occasion we repeat the examination stations to give additional data on examination and examiner quality. This data is important to assure the ongoing validity and reproducibility of the examination.
We agreed to this recommendation because there is a breadth of scientific and educational literature showing that prior knowledge of topics in an examination of this type has no impact on marks or pass rates. In addition, our detailed analysis of the previous Part 2 OSCE, where the topics examined on separate days were the same, but the cases were subtly different, showed no difference in candidate performance and pass rates between different sittings. In the absence of advantage we considered whether there would be a disadvantage to candidates if they were aware of the previous day's topics. In all previous Part 3 Clinical Examinations a topic would not be examined the following day and candidates' last-minute revision may have taken that into account. There is no evidence in the literature that changes to a last minute revision schedule have a material effect on results as the skills, attitudes and abilities assessed in this type of examination are not primarily knowledge-based. Rather, the previous OSCE and the current Part 3 examination test a candidate's skills and ability to perform as an O&G specialist providing care for patients.
All candidate performance, examinations, and examiner performance will continue to undergo detailed psychometric analysis to ensure that the quality and validity of the Part 3 MRCOG Clinical Examination remains at the highest level.