The purpose of examinations has changed over time. Yuval Noah Harrari writes in his book,hom*o Deus:
“Originally, schools were supposed to focus on enlightening and educating students, andmarks were merely a means of measuring success. But naturally enough schools soon beganfocusing on achieving high marks. As every child, teacher and inspector knows, the skillsrequired to get high marks in an exam are not the same as a true understanding of literature,biology or mathematics. Every child, teacher and inspector also knows that when forced tochoose between the two, most schools will go for the marks” [1].
For the RCoA exams, the balance between ‘enlightening andeducating’ and accumulation of marks seems to tip firmlytowards the latter. Although this is common for many schoolsand colleges, the RCoA relies particularly heavily on testingknowledge of minutiae in order to separate candidates.Obsolete respirometers, a variety of hygrometers, physicalprinciples and other basic science that permeate the exam maybe a useful test for assigning marks to differentiate between apass or fail. It is for the examinee, however, extremely tediousstudying material that is of limited or no relevance to practice. Furthermore, it does not differentiate between those who areclinically competent and safe and those who are not. Many verycompetent doctors have been lost to training, or the specialityentirely, because of the RCoA exam hurdle.
The GMC state that “The FRCA examinations are high-stakesummative assessments that have the potential to impact ontrainee careers and patient safety.” A life consuming effort isrequired by most to pass the exam. Moreover, it presents asignificant opportunity to develop and improve practice. It isthen, of course, incumbent on the college to write an exam thatpertains to modern anaesthetic practice.
Box 1, a question from the current RCoA Primary guide, is awell-written question requiring some knowledge of membranepotentials and deductive thinking [2]. However, prioritisingknowledge of membrane potentials in the context of the largeamount of clinically relevant knowledge required for safe andhigh-quality care is questionable. It is important to rememberthat this question displaces a question on, for example, difficultairway management, trauma and resuscitation, TIVA or regionalanaesthesia. The latter two, increasingly cornerstones of modernpractice, were entirely absent from both my Primary and Finalexams. Box 2 is a question from the current RCoA Final guide [3].
RA-UK’s guide to the regional anaesthesia part of the RCoAcurriculum contains a collection of papers and other resourceswith the ultimate goal of “providing safe and effective regionalanaesthesia”. This goal-orientated practical material is muchmore engaging for the trainee and more likely to be committedto memory than material from either of the above questions.
To the credit of the College, the exam enhances the perceptionof the specialty, forms an exacting test requiring perserveranceand commitment, and has certainly improved over timewith regard to its content. Furthermore, improvements haveundoubtedly been made regarding the quality of the exam,notably a gradual move away from a bank of low-quality MTFs.Nonetheless, last year my Final exam mark was based entirely onthe MCQ part of the exam in consequence of a significant errorin the provision of the CRQ exam.
Certainly, we can all take pride in passing an exam that isrelatively difficult. However, anaesthetists’ development seems tobe suffering at the expense of indulging the College with regardto exam content. The reality is that once the exam is passed, mostanaesthetists jettison much of this knowledge from memory inorder to make way for that which is useful for them and the careof their patients.
Ben McCartney
ST6 Anaesthetic trainee
Altnagelvin Area Hospital, Derry
References
- Harari YN. hom*o Deus, a brief history of tomorrow. New York: Vintage, 2017.
- Wilkinson M. Guide to the FRCA examination. The Primary, 4th edition. Oxford: Blackwell’s, 2013.
- Guide to the FRCA examination – The Final, 4th Edition. London: Royal College of Anaesthetists, 2020.
A response
Thank you for the opportunity to respond to this letter, whichraises some interesting points and challenges. The College fullyrecognises and acknowledges the personal impact and immensestress to anaesthetists in training, and others, when studying forthe FRCA examinations. This impact was echoed in the recentexternal review that we commissioned of the FRCA as well asour Faculty examinations (FFPMRCA and FFICM) [1]. In responsewe have published a programme for the development of theseexams, with an Examinations Development and Assurance Group(EDAG) being constituted to implement the necessary changesover the coming months and years. Significantly, this reviewincludes a consideration of the impact of the assessment burdenon anaesthetists in training. The College is also committed togiving anaesthetists in training a greater role in the developmentof the exams; for example, there will be trainee representation onthe EDAG for all the exams.
It is important to emphasise that the exam items and questionsare under constant review to ensure their validity and reliability.There is inevitably a degree of inertia in the system of questionwriting, reviewing, and editing as it takes a large amount oftime and work to develop new question items that are valid andreliable. Currently this burden largely falls on examiners, butwe are looking at how we can open this up to a larger group ofquestion writers. The Examinations Committee is considering how interested anaesthetists, including anaesthetists intraining, can be involved in the development and writingof exam materials, with the examiner body editing andrefining to ensure quality assurance and alignment with thecurriculum.
Many anaesthetists will be aware of doctors who arecompetent in the clinical environment but have struggled topass exams. This is an area requiring balance and judgment. Ifthe standard of the exam is too high, too few doctors will passthrough the system having an impact on both individualsand healthcare provision; but at the same time it is a matterof great importance that high standards are maintained.In particular, an understanding of the basic science thatunderpins clinical practice is a core element of anaestheticknowledge and should remain so. However, testing in allareas of the curriculum must be proportionate and balanced.There is limited evidence to compare postgraduate medicalexamination performance with clinical outcomes [2], and theexternal review has recommended that further research inthis area should be considered. To address this, the examsteam and senior examiners will be working with anaesthetistsin training, on secondment as part-time education fellows, toconduct research projects looking at exam validity and theattainment gap, as suggested in the review.
Finally, it is important to remember that examinations areonly one part of the College’s assessment strategy. High stakesexaminations are deemed by most professions to bea necessary part of such a strategy, but organisations suchas ours must remain open to feedback to drive changes. Theexam in its current format is fit for purpose and approved bythe GMC, but the recent exam reviews provide the Collegewith an opportunity to make some significant changes toalign our exams better with contemporary assessment theoryand practice, and make sure that the overall assessmentburden is proportionate.
Roger Sharpe
Chair, FRCA Examinations
Royal College of Anaesthetists
References
- Royal College of Anaesthetists. Independent review of the assessment processes of the Royal College of Anaesthetists, 2023. https://rcoa.ac.uk/sites/default/files/documents/2023-02/Independent-review-assessment-processes-RCoA-FINAL.pdf (accessed 17/5/2023).
- Wakeford R, Ludka K, Woolf K, McManus IC. Fitness to practise sanctions in UK doctors are predicted by poor performance at MRCGP and MRCP(UK) assessments: data linkage study. BMC Medicine 2018; 16: 1-16.
