A BMA Northern Ireland survey found 56 per cent of primary care doctors said violence and abuse from patients was a problem, compared with 53 per cent in secondary care.
It also found 54 per cent of respondents across both sectors thought violence from patients was a problem - an increase of 9 per cent since 2006.
BMA Northern Ireland is calling for patients and their relatives to stop abusing doctors trying to provide treatment.
Patients with a history of violence should have a warning marker on their health record to be shared between all healthcare organisations, it adds.
Dr Paul Darragh, chairman of the BMA council in Northern Ireland, said the findings were 'incredibly worrying'.
'The abuse is often random, with no particular motivation behind the physical violence,' he said. 'The effect of threats, abuse and assaults impact not only on doctors on the receiving end, but also the wider healthcare team and other patients.'