Exclusive: Leaked GMC report rejects GP exam bias claims

A leaked draft of a highly anticipated report on low MRCGP exam pass rates for international medical graduates has ruled out bias in the test, but called for more diverse examiners and more support for overseas GP trainees.

RCGP: report commissioned by GMC rules out bias in MRCGP exams
RCGP: report commissioned by GMC rules out bias in MRCGP exams

The findings are a boost for the RCGP ahead of a judicial review into the fairness of the clinical skills assessment (CSA) element of the MRCGP exam next month.

The delayed Independent Review of the Membership of the RCGP (MRCGP) was commissioned by the GMC and led by Professor Aneez Esmail - a University of Manchester professor of general practice and expert on racism in the medical profession.

The version obtained by GP is a draft circulated among key people for comment ahead of final publication, expected later this year.

The GMC commissioned the review in March to investigate results of applied knowledge test (AKT) and clinical skills assessment (CSA) elements of MRCGP exams from October 2010 to December 2012.

On the difference in pass rates, it said ‘it is difficult to attribute this to bias because of the nature of the test and the reasons for the differential pass rates’.

It said that the vast majority of international graduates come from the ‘Indian subcontinent and from other countries where the discipline of general practice is poorly developed’.

‘IMG candidates will therefore have much less direct experience of general practice than their UK counterparts,' the report says.

‘In our view, this must disadvantage this group in subtle ways and explain the much larger differences in outcomes between UK and non-UK graduates. This will also be one of the reasons that there are significant differences in outcome in the CSA examination.'

The report found that the ‘method of assessment’ is not the reason for differential outcomes between UK and non-UK graduates, but said that it was ‘worth pointing out’ that the pool of CSA examiners ‘was not representative of the ethnic background of GPs more generally’.

‘It is important that the candidates are assessed in a way that reflects the diversity that they see,' it added.

Although the report added that 'the nature of the examination is such that it is open to subjective bias', adding that 'we cannot ascertain if the standardised patients (played by actors) behave differently in front of candidates from non-white ethnic groups...nor can [we] confidently exclude bias from the examiners, it says: 'It is clear to us that the RCGP is aware of these potential biases and takes steps to mitigate them.'

'There is mandatory training of RCGP examiners in equality and diversity issues, there is training and monitoring of the actors to ensure consistency in the presentation of the cases, and there is a well developed programme of continuing training and feedback to examiners of their performance,' the report said.

However, the report said it was 'unacceptable' that the feedback given to candidates was limited.

It reads: ‘We were told that the number of candidates precluded individualised feedback. We feel that this is not acceptable especially in an examination that charges candidates £1,694 and which is a high-stakes exam such as the MRCGP. If candidates fail the exam they need to know why and both the candidate and their trainee in their workplace need to be given formative feedback so that the candidate knows where they are weak and identify areas for improvement.’

It called for better support for international medical graduates, who are only given ‘generic advice on some of the characteristics of candidates who fail the exam and how to improve their chances’.

It reads: ‘So long as this country depends on recruiting large numbers of international medical graduates, then we need to acknowledge that most of these international medical graduates come into medicine from a different starting point. Many will require much more training and support before they can be considered equal to their British colleagues and perhaps the differential outcome in the CSA examination is a reflection of this. Yet they are treated exactly the same as British graduates whose undergraduate and postgraduate training once qualified is so very different.’

BMA GP trainee subcommittee chairman Dr Krishna Kasaraneni said the draft report's findings echoed BMA calls for more diverse cases in CSA tests and better feedback for trainees.

He was unwilling to comment further on the fairness of the test until the final review was published, but said: 'There are wider questions about the selection of GP trainees and how those who require additional support or training are identified earlier.

'We would welcome a further study demonstrating what happens to trainees who are forced to withdraw from training, why they are asked to withdraw, and whether [workplace assessment] processes are as fair as we would hope they are.

'We also are concerned about significant variations in CSA pass rates by deanery, and this clearly needs more research to learn why there are these differences and what can be learnt from better-performing deaneries and [local education and training boards].'

An RCGP spokesman said: 'The RCGP takes equality and diversity issues very seriously and we have always strongly refuted any allegations that the MRCGP exam is discriminatory. The early draft report of the independent review appears to support this.

'Our assessment procedures are designed to ensure safe patient practice and we are always looking to enhance and improve our processes. We recognise many of the issues raised in this draft and have been working to address these for some time. Once we receive the final report, we will ensure that if any new issues emerge, we develop a response to these as well.'

Have you registered with us yet?

Register now to enjoy more articles and free email bulletins

Register

Already registered?

Sign in

Before commenting please read our rules for commenting on articles.

If you see a comment you find offensive, you can flag it as inappropriate. In the top right-hand corner of an individual comment, you will see 'flag as inappropriate'. Clicking this prompts us to review the comment. For further information see our rules for commenting on articles.

comments powered by Disqus