Information and guidance on externality (use of external expert assessors) in specialty training in O&G:
Why externality is important
The use of external experts is recognized to be a beneficial method of providing objectivity in some of the processes undertaken by deaneries to manage the quality of education and training and is a formal requirement of the General Medical Council’s (GMC) Quality assurance framework for all specialties. Although the onus is on deaneries to engage active external scrutiny for these processes (assessing a trainee’s progress through the Annual Review of Competence Progression [ARCP], quality visits, School Board membership and recruitment), the GMC and the medical royal colleges collectively agree that there should be a clearer understanding of what that engagement means.
Specialty externality is particularly important in the processes used for managing the quality of postgraduate medical education, where it has three main functions:
- as an objective guarantor of the way the quality of education is managed by deaneries
- as a conduit for conveying patient and trainee safety concerns arising from educational quality assurance activity to other healthcare regulators, relevant service providers and medical royal colleges’ clinical standards departments
- as a collector and disseminator of good practice in education and training.
There are, of course, other dimensions to externality – lay and geographical – but this section of the site is concerned principally with the way in which the RCOG engages with the deanery quality management processes of assessing trainee progress (Process 1) and quality visits (Process 2), specifically through the contribution of SAs.
Policy statement on externality
The overarching principles for externality are derived from the GMC’s standards in The Trainee Doctor (Standards for Deaneries section, 4.1–4.7) and the Gold Guide (7th edition, page 55 from point 4.60) and are approved by the RCOG Education Board as part of the College’s developing Education Quality Strategy. The administrative support for externality is located in the Directorate of Education.
The RCOG will:
- appoint suitable individuals to act as SAs on its behalf
- by focusing on the experience and achievement of trainees and trainers, verify that appropriate standards are being attained by trainees and so help deaneries maintain the quality of O&G provision
- seek to confirm that assessment processes are sound and operated fairly
- collect, record and disseminate good practice identified, to promote comparability of the trainee and trainer experience between deaneries
- seek to confirm that the curriculum can be effectively delivered
- contribute to the deaneries’ effective management of educational quality, including the development of trainers
- provide standards for SAs so that they undertake their responsibilities in a consistent manner and in accordance with the RCOG’s education policy
- expect SAs to adhere to extremely high standards of probity, in particular taking care to declare any conflicts of interest, whether professional, personal or geographical
- engage with the RCOG Quality & Knowledge Directorate, GMC and other stakeholders on patient and trainee safety issues arising from the experience of RCOG SAs
- recognise the professional contribution of SAs by awarding 5 continuing professional development (CPD) points for each external activity (maximum 25 in a 5-year cycle)
Process 1: assessing trainee progress – ARCP and appeal panels
The Gold Guide (7th edition) lays out the requirements for ARCP and appeal panel membership. There are three purposes for engaging an external representative in assessing trainee progress:
- to provide an independent perspective on the conduct of the ARCP process
- to assess whether there are any issues with the delivery of the curriculum
- to determine whether an individual trainee’s progress is assessed consistently against the objectives described in the RCOG training matrix of educational progression and associated guidance for ARCP outcomes
The requirement for O&G is that 10% of ARCPs, constituting a representative sample from all years of training, are reviewed by an SA. The reviews will usually be conducted face-to-face but the sampling of standard or ‘routine’ ARCPs, where no major issues are anticipated, may be conducted through a review of paperwork or the trainee’s ePortfolio. This would not be the case where there are known performance issues, in which case a face-to-face ARCP is expected. The SA answers specialty-based queries, identifies issues that arise from the ARCP or training processes, provides advice and direction on concerns about individual experiences of training as they arise and provides advice on a decision, if invited to do so. The SA will also specifically review the ARCP process in relation to the RCOG standards, as set out in the training matrix and ARCP outcome guidelines.
The RCOG has developed a standardised ARCP Feedback report form (Word document, 19 kb). SAs are required to submit their completed form to Externality administrator and the LETB/deanery as soon as possible. Failure to submit a form will mean that the agreed RCOG CPD points will not be awarded.
In cases of appeal panels, the deaneries may request an SA to ensure assessment of the process and the training opportunities. As such, these situations are trainee-identifiable and, thus, there is no RCOG report template. SAs are only required to provide the deanery with whatever report the deanery requires. If, however, there are issues of concern relating to patient or trainee safety, SAs should report directly to EPQ by an individual confidential report or letter but only after consultation with the Postgraduate Dean. The RCOG requests that deaneries notify EPQ to confirm the SA’s attendance after the panel has met so that the requisite CPD points are awarded.
Process 2: LETB/deanery visits to local education providers (LEPs)
LETB/deaneries undertake assessments of local education providers (LEPs) through a process of internal visiting that is specific to each deanery to comply with the GMC’s Quality assurance framework. Visits may be organized on a routine basis or as part of a targeted assessment. SAs represent the RCOG on the visit team and, as such, provide feedback to the RCOG, including the deanery’s own visit report. While it is anticipated that external advice will definitely be required for targeted visits, this is not necessarily the practice for routine monitoring visits. LETBs/deaneries vary in whether external advice is sought for such visits, although good practice would be to ensure that a sample of routine visits should involve external advisors.
Targeted visits will be led and undertaken by LETBs/deaneries in conjunction with College involvement and may result from concerns from a variety of sources, such as:
- lack of compliance with GMC training standards
- outcomes of GMC trainee and trainer surveys
- deanery quality management systems information (e.g. annual reports)
- specific concerns raised by individuals
As LETBs/deaneries have a variety of documentation and the RCOG wishes to minimize the additional burden on SAs, the LETB/deanery visit report is considered sufficient information for the RCOG to collate issues from a national perspective. It is essential that the deanery is made aware that the final visit report is a requirement of the RCOG in supplying an SA. The LETB/deanery’s visit report may include the direct input of the SA or the LETB/deanery may request a separate report. If the SA has to provide a separate report to the LETB/deanery, the SA should use the standardised deanery visit report form (Word document, 19 kb). Whatever is reported to the LETB/deanery should be submitted to the Externality Administrator. This should be done as soon as possible after the visit. No trainee- or trainer-identifiable data or information should be included. As for Process 1, no CPD points will be awarded if a report is not submitted.
Managing serious concerns
Responding to Concerns is the fourth element of the GMC’s Quality assurance framework and supports the RCOG’s role in maintaining safe training environments.
The College is required to have a process in place for identifying serious issues when they arise and mechanisms for reporting to the deanery and regulator. If an SA identifies a serious issue (for example, pertaining to patient or trainee safety), this should be referred directly to the Postgraduate Dean as soon as possible and EPQ should be informed. If the issue has been assessed inadequately at local level and cannot be taken up by the Postgraduate Dean for whatever reason, the issue may be reported by the College to the GMC via the Vice President (Education), or to the Care Quality Commission (CQC) via the Vice President (Clinical Quality) through the published response to concerns processes. The RCOG reserves the right to contact the Trust Chief Executive Officer jointly with the relevant Postgraduate Dean where service reviews and education quality visits have come to similar conclusions.
Role description for SAs
- Professionally qualified in O&G in the UK (consultant in O&G)
- Extensive experience of managing training (e.g. as Head of School, Training Programme Director, Director of Medical Education or equivalent)
- Extensive awareness of the O&G curriculum, workplace-based assessment, Training ePortfolio, RCOG training matrix of educational progression and guidance for ARCP outcomes
- Trained and recognised as an Educational Supervisor and in procedures for dealing with doctors in difficulty
- Experience in educational governance (e.g. member of School Board, Chair of Specialty Training Committee)
- Experience in taking part in quality assurance (e.g. member of a deanery visit team, ARCP panel, analysis of quality data in other deanery quality management activities)
- Knowledge of the GMC’s Quality Improvement Framework and standards (The Trainee Doctor and standards for curricula and assessments) and evidence of engagement in improving the quality of training
- Detailed knowledge of requirements of the Gold Guide
- Represent the RCOG to ensure that the RCOG’s standards regarding delivery and assessment of the curriculum are being implemented
- Provide an independent view of quality management processes (ARCP and visiting)
- Observe the equity of the ARCP process and provide advice on RCOG tools and standards
- Fully participate in the deanery visit process by interviewing trainees and trainers, constructively advising on the training environment and contributing to relevant paperwork
- Prepare and supply appropriate feedback to the RCOG in the form of reports, including how various initiatives to improve training and promote standardisation are being implemented (these include the educational matrix, ARCP outcome guidance and ongoing work regarding assessment and subspecialty training)
- Prepare and supply confidential reports where appropriate and agreed with the Postgraduate Dean
- Complete the StratOG training package
- Be able and willing to complete at least one external assessment for RCOG per training year.
- SAs will be approved for a period of 3 years, renewable on application
- The RCOG will maintain a register of approved SAs via SEAC and review the contribution of SAs on an annual basis. If the SA fails to provide reports or attend visits, they will be removed from the RCOG register
- Each visit with completed report will result in 5 CPD credits (note that a maximum of 25 credits may be claimed in each 5-year cycle for this activity)
Internal process for appointing and allocating SAs
Individuals wishing to act as SAs should complete an application form and send to the SEAC Secretary, who will scrutinize it against the person specification requirements and make a recommendation to the Externality Lead. The Externality Lead will either confirm or otherwise and the individual will be notified accordingly.
The RCOG will maintain the register of approved SAs. Resources for SAs will be developed and maintained on the RCOG website.
LETBs/Deaneries should contact the administrator at least 8 weeks in advance of when an SA is needed, and with a firm confirmed date for the relevant event. Deaneries will be informed that the RCOG expects to receive the deanery final visit report as a condition of supplying an SA.
The Administrator will then circulate the request with the date to the SAs and ask for expressions of interest to undertake the event, including a declaration of any possible conflicts of interest due to professional or personal relationships.
Following the visit/ARCP appeal, the SA will send the Trainees’ Coordinator a report depending on the assessment.
Improving the quality of training
The information that is acquired through these processes will be:
- Recorded and presented annually to SEAC meetings and the Education Board
- Used as evidence for the Annual Specialty Report to the GMC
- Shared with the Clinical Quality directorate if appropriate (where there are clear service and/or patient/trainee safety issues)
- Triangulated with other quality data to review the need for future Global Education directorate developments