The British Association of Physicians of Indian Origin (BAPIO) decided this month not to appeal against a judicial review that found the CSA was not racially discriminatory.
But BAPIO leaders have warned that unless changes to the MRCGP exam materialise within the next year, it could revive the challenge.
The group has agreed to work with the RCGP to look for ways to improve the fairness of the CSA, which it remains convinced is unfair.
BAPIO took legal action against the RCGP and the GMC after data showed UK graduates from black and minority ethnic backgrounds were four times more likely to fail than their white counterparts.
BAPIO president Dr Ramesh Mehta has said CSA tests must be made 'culturally neutral' to avoid competent doctors failing purely because of language and cultural barriers.
However, the demand is at odds with the recommendations of a major GMC-commissioned review into the fairness of CSA tests, published last year.
Dr Mehta said BAPIO believed cultural and language differences were independent of a candidate's ability to work and succeed as a GP.
'The question is of their language skills, communication skills and the cultural differences. We think that's the problem, we don't believe there is any problem regarding their competency to practise as GPs,' he said.
But the GMC-commissioned report, by Professor Aneez Esmail from the University of Manchester, concluded the CSA had to go beyond testing clinical knowledge, to ensure doctors were 'safe to practise in the UK'.
At the time, the GMC highlighted this as an 'important observation'.
CSA 'not just a clinical exam'
The report said: 'The CSA is not a culturally neutral examination and nor is it intended to be. It is not and nor should it be just a clinical exam, testing clinical knowledge in a very narrow sense. It is designed to ensure doctors are safe to practise.'
Dr Mehta told GP he continues to believe the CSA unfairly discriminates against international and ethnic minority candidates on the grounds of cultural differences.
He warned that it was not in the UK's best interests to fail so many trainees, given the cost of training them, and the GP workforce crisis.
'We want the best doctors, but at the same time we want a fair system,' he said.