The college says that 2,137 GPs across the UK are from the EU, and is calling for the next government unilaterally to guarantee their right to remain in the country post-Brexit.
Polling commissioned by the college from YouGov shows that three in five people believe EU healthcare staff should have their status guaranteed 'from the outset' in Brexit talks.
College leaders estimate that if these doctors are forced out of the UK, 3,456,481 patients stand to lose their family doctor.
The Conservative party said last week it stood by its pledge to train and retain an extra 5,000 GPs by 2020, despite the commitment not featuring in its manifesto.
But the college warns that all parties must guarantee the future of EU health professionals working in the UK - warning that with general practice already in crisis it can ill afford to lose around 5% of its total workforce.
Official data published earlier this year showed that the full-time equivalent GP workforce fell 445 in the three months to December 2016 - on top of a 100 FTE GP drop over the previous year.
The loss of EU doctors would hit some parts of the UK particularly hard - in Northern Ireland, EU nationals represent 11% of the GP workforce, while in Scotland and Wales the figure is 4%.
The college has called for general practice to be added to the list of 'shortage occupations', to make it easier for overseas doctors to live and work in the UK. GP leaders at last week's LMC conference 2017 in Edinburgh also backed this stance.
RCGP chair Professor Helen Stokes-Lampard said: 'We desperately need thousands more GPs right across the UK, and in England we have been promised 5,000 more by 2020 – but these figures indicate that we risk losing well over 2,000 family doctors from the NHS if their position is not secured as part of Brexit negotiations, and that is just not safe or acceptable.
EU workers in general practice – and the NHS as a whole - play a vital role in ensuring that care is delivered free at the point of need for anyone who needs it. Losing this skill and experience would be disastrous for the sustainability of our health service, and our ability to deliver the care our patients need.'