Soaring pressure on general practice - with a growing workforce crisis, rising workload and a decline in funding over the past seven years - is undermining practices' ability to offer timely access to appointments, the college warned.
The college said the crisis facing general practice was a 'case of national concern' that would increasingly put patients at risk.
In 2016/17 there were 80m cases when patients had to wait more than a week to see a GP or a nurse at their practice, but by 2021/22 this figure will rise to 102m if current trends are allowed to continue, the college warned.
Analysis of NHS patient survey data by the college also revealed significant variation between CCG areas in access to general practice.
More than a third of patients wait a week or more on average for an appointment in some CCG areas, the college warned - including Corby and Fareham and Gosport. In around one in 10 CCG areas across England, a quarter of patients wait more than a week to see a GP or a nurse.
In Bradford, where GP access figures are the best, one in 10 patients are still waiting more than a week for an appointment, the college warned.
The college called on the government to accelerate investment promised in the GP Forward View, warning that the profession was struggling to cope with a 16% rise in workload and a fall in funding since 2010.
RCGP chair Professor Helen Stokes-Lampard said: 'Our patients should be able to see a GP when they need to, so we’re highly concerned that patients are finding it so difficult to make an appointment, and that in so many cases they have had to wait more than a week to see a GP. This is a clear risk to patient safety – and if nothing is done soon, it is clear that this is set to get worse.
'If these patients can’t secure an appointment with their GP when they need one, it’s probable that they will return at some point to another area of the NHS, when their condition may have worsened, and where their care will cost the health service significantly more – something which could’ve been avoided if they’d been able their GP in the first instance.
'GPs and our teams are now making more patient consultations than ever before - over 370m each year - and with workload continuing to escalate, and with continuing resource and workforce pressures, the worrying outcome is that we will be unable to see all our patients who need to be seen.'