A UK study of over 13,000 cancer patients found 58% were referred by their GP after one consultation and 82% after two visits.
Experts said it showed GPs had improved the detection of cancer symptoms, but warned more can be done to help GPs spot symptoms earlier.
They also suggested that GPs should be allowed to refer suspected cases more easily to speed up detection rates.
An RCGP audit in November 2011 found that GPs referred three quarters of suspected cancers within two visits.
In the latest study, researchers analysed data from the RCGP's National Audit of Cancer Diagnosis in Primary Care 2009-10 involving 13,035 patients and 18 different cancers.
As expected, more visits to a GP before referral resulted in a greater delay before diagnosis.
GPs referred cases of suspected breast cancer fastest, with just 2.9% of patients visiting three times or more before referral. Multiple myeloma took the most visits before referral, with 46% of patients visiting three times or more.
Authors said efforts to improve the speed of cancer diagnosis further should focus on research and support to help GPs to spot cancer symptoms in fewer visits.
They added: 'A more liberal policy for referral and investigation of patients with non-specific symptoms may increase the number of cancer patients diagnosed after one or two consultations, but at the expense of additional patient anxiety and healthcare utilisation costs for patients who will be investigated but found not to have cancer.'
They recommended further research to investigate whether greater use of diagnostic tests in primary care could reduce the number of consultations required before referral.
Co-author Professor Greg Rubin, the RCGP and Cancer Research UK clinical lead for cancer, said: 'We’ve found that most patients who go to their GP with cancer symptoms are being promptly referred to a specialist. NICE referral guidelines have helped people with classic symptoms to be seen more quickly but, for patients with less typical symptoms, the decision to refer isn’t always as simple.
'Reducing the number of pre-referral consultations can result in a more timely diagnosis of cancer. We need to consider ways of making the process of primary care assessment even smarter, for instance by wider use of clinical decision support tools or more efficient investigation pathways.'
Study author Dr Georgios Lyratzopoulos, research fellow at Cambridge University, said: 'These results show the progress we’re making in spotting cancer at the earliest opportunity. We now understand the typical symptoms of some cancers, like breast and melanoma, very well and that helps doctors to spot them quickly.'
Sara Hiom, early diagnosis director at Cancer Research UK, said: 'These findings are encouraging but there is still room for improvement. Progress is clearly being made but one in five people have to make more than two visits to their GP, although it’s not surprising that this is usually for those cancers that are harder to spot. And we know for some people, difficulty making an appointment can be a barrier to going to the GP in the first place.'
The study was published in the British Journal of Cancer.