Information and guidance on externality (use of external expert assessors) in specialty training in O&G
Why externality is important
The use of external experts is recognised to be a beneficial method of providing objectivity in some of the processes undertaken by deaneries to manage the quality of education and training and is a formal requirement of the General Medical Council’s (GMC) Quality assurance framework for all specialties. Although the onus is on deaneries to engage active external scrutiny for these processes (assessing a trainee’s progress through the Annual Review of Competence Progression [ARCP], quality visits, School Board membership and recruitment), the GMC and the medical royal colleges collectively agree that there should be a clearer understanding of what that engagement means.
Specialty externality is particularly important in the processes used for managing the quality of postgraduate medical education, where it has three main functions:
- as an objective guarantor of the way the quality of education is managed by deaneries
- as a conduit for conveying patient and trainee safety concerns arising from educational quality assurance activity to other healthcare regulators, relevant service providers and medical royal colleges’ clinical standards departments
- as a collector and disseminator of good practice in education and training.
There are, of course, other dimensions to externality – lay and geographical – but this section of the site is concerned principally with the way in which the RCOG engages with the deanery quality management processes of assessing trainee progress (Process 1) and quality visits (Process 2), specifically through the contribution of SAs.
Policy statement on externality
The overarching principles for externality are derived from the GMC’s standards Promoting Excellence and the Gold Guide and are approved by the RCOG Education Board as part of the College’s developing Education Quality Strategy. The administrative support for externality is located in the Directorate of Education.
The RCOG will:
- appoint suitable individuals to act as SAs on its behalf
- by focusing on the experience and achievement of trainees and trainers, verify that appropriate standards are being attained by trainees and so help deaneries maintain the quality of O&G provision
- seek to confirm that assessment processes are sound and operated fairly
- collect, record and disseminate good practice identified, to promote comparability of the trainee and trainer experience between deaneries
- seek to confirm that the curriculum can be effectively delivered
- contribute to the deaneries’ effective management of educational quality, including the development of trainers
- provide standards for SAs so that they undertake their responsibilities in a consistent manner and in accordance with the RCOG’s education policy
- expect SAs to adhere to extremely high standards of probity, in particular taking care to declare any conflicts of interest, whether professional, personal or geographical
- engage with the RCOG Quality & Knowledge Directorate, GMC and other stakeholders on patient and trainee safety issues arising from the experience of RCOG SAs
- recognise the professional contribution of SAs by awarding 5 continuing professional development (CPD) points for each external activity (maximum 25 in a 5-year cycle)
Process 1: assessing trainee progress – ARCP and appeal panels
The Gold Guide lays out the requirements for ARCP and appeal panel membership. There are three purposes for engaging an external representative in assessing trainee progress:
- to provide an independent perspective on the conduct of the ARCP process
- to assess whether there are any issues with the delivery of the curriculum
- to determine whether an individual trainee’s progress is assessed consistently against the objectives described in the RCOG training matrix of educational progression and associated guidance for ARCP outcomes
The requirement for O&G is that 10% of ARCPs, constituting a representative sample from all years of training, are reviewed by an SA. The reviews will usually be conducted face-to-face/remote but the sampling of standard or ‘routine’ ARCPs, where no major issues are anticipated, may be conducted through a review of paperwork or the trainee’s ePortfolio. This would not be the case where there are known performance issues, in which case a face-to-face/remote ARCP is expected. The SA answers specialty-based queries, identifies issues that arise from the ARCP or training processes, provides advice and direction on concerns about individual experiences of training as they arise and provides advice on a decision, if invited to do so. The SA will also specifically review the ARCP process in relation to the RCOG standards, as set out in the training matrix and ARCP outcome guidelines.
The RCOG has developed a standardised ARCP Feedback Form. SAs are required to submit their completed form to Externality administrator and the deanery as soon as possible. Failure to submit a form will mean that the agreed RCOG CPD points may not be awarded.
In cases of appeal panels, the deaneries may request an SA to ensure assessment of the process and the training opportunities. As such, these situations are trainee-identifiable and, thus, there is no RCOG report template. SAs are only required to provide the deanery with whatever report the deanery requires. If, however, there are issues of concern relating to patient or trainee safety, SAs should report directly to the College by an individual confidential report or letter but only after consultation with the Postgraduate Dean.
Process 2: Deanery visits to local education providers (LEPs)
Deaneries undertake assessments of local education providers (LEPs) through a process of internal visiting that is specific to each deanery to comply with the GMC’s Quality assurance framework. Visits may be organised on a routine basis or as part of a targeted assessment. SAs represent the RCOG on the visit team and, as such, provide feedback to the RCOG, including the deanery’s own visit report. While it is anticipated that external advice will definitely be required for targeted visits, this is not necessarily the practice for routine monitoring visits. Deaneries vary in whether external advice is sought for such visits, although good practice would be to ensure that a sample of routine visits should involve external advisors.
Targeted visits will be led and undertaken by deaneries in conjunction with College involvement and may result from concerns from a variety of sources, such as:
- lack of compliance with GMC training standards
- outcomes of GMC trainee and trainer surveys
- deanery quality management systems information (e.g. annual reports)
- specific concerns raised by individuals
As deaneries have a variety of documentation and the RCOG wishes to minimise the additional burden on SAs, the deanery visit report is considered sufficient information for the RCOG to collate issues from a national perspective. It is essential that the deanery is made aware that the final visit report is a requirement of the RCOG in supplying an SA. The deanery’s visit report may include the direct input of the SA or the deanery may request a separate report. If the SA has to provide a separate report to the deanery, the SA should use the standardised deanery visit report form . Whatever is reported to the deanery should be submitted to the the College.. This should be done as soon as possible after the visit. No trainee or trainer identifiable data or information should be included. As for Process 1, CPD points may not be awarded if a report is not submitted.
Managing serious concerns
The College is required to have a process in place for identifying serious issues when they arise and mechanisms for reporting to the deanery and regulator. If an SA identifies a serious issue (for example, pertaining to patient or trainee safety), this should be referred directly to the Postgraduate Dean as soon as possible and the College should be informed. If the issue has been assessed inadequately at local level and cannot be taken up by the Postgraduate Dean for whatever reason, the issue may be reported by the College to the GMC via the Vice President (Education), or to the Care Quality Commission (CQC) via the College through the published response to concerns processes. The RCOG reserves the right to contact the Trust Chief Executive Officer jointly with the relevant Postgraduate Dean where service reviews and education quality visits have come to similar conclusions.
Internal process for appointing and allocating SAs
Individuals wishing to act as SAs should complete an application form linked below and send to the Coordinator for Quality. Applications will be reviewed by the RCOG Externality Lead, who will either confirm or otherwise and the individual will be notified accordingly.
The RCOG will maintain the register of approved SAs. Resources for SAs will be developed and maintained on the RCOG website.
Deaneries should contact the administrator at least 8 weeks in advance of when an SA is needed, and with a firm confirmed date for the relevant event. Deaneries will be informed that the RCOG expects to receive the deanery final visit report as a condition of supplying an SA.
The Administrator will then circulate the request with the details to the SAs and ask for expressions of interest to undertake the event, including a declaration of any possible conflicts of interest due to professional or personal relationships.
Following the event , the SA should send the coordinator a report depending on the assessment so that an acknowledgement letter can be issued. The SA is then eligible to claim the relevant CPD points. Most ARCP sessions are run remotely; however, if a face to face appointment is agreed, travel expenses should be refunded by the host Deanery. The host Deanery should ensure that SAs are provided with the relevant claim forms.
Improving the quality of training
The information that is acquired through these processes will be:
- Recorded and presented annually to SEAC meetings and the Education Board
- Shared with the Clinical Quality directorate if appropriate (where there are clear service and/or patient/trainee safety issues)
- Triangulated with other quality data to review the need for future developments.