Researchers found that smoking can age the brain by up to 10 years. But they did not find a similar relationship among women.
Smoking has been linked to the risk of developing dementia before but the extent of the risk was unclear.
The study of 7,236 civil servants of both sexes was led by Séverine Sabia at University College London.
Her team examined the results of three separate cognitive assessments of these participants over the course of a decade, starting in 1997-99.
These assessments included tests for memory, vocabulary and reasoning. The average age was 56 years.
The study team then compared these scores against six assessments of each volunteer’s smoking history over 25 years, including the study period.
Although mental decline occurred in never smokers, it was faster in those who were smoking at the start of the study. Men who continued to smoke during follow-up had even greater decline across all cognitive tests.
Researchers said: 'We observe that the effect size associated with smoking is similar to that associated with 10 years of age.'
Those who quit smoking in the 10 years before the study began were still at risk of greater mental decline than never smokers.
Yet, long-term ex-smokers did not suffer this effect.
Researchers concluded that the link between smoking and cognitive decline is likely to be underestimated, especially in older people, due to the higher risk of death dropout among smokers.
But they said it was unclear why no connection was seen in women.
They concluded: 'Our study illustrates the importance of examining risk factors for cognitive decline much earlier in the life course.'