Guidance on the website from accredited sources carries a quality kitemark, but this will not appear next to BTS advice.
The society did not provide sufficient evidence for how it formulates guidance, according to NHS Evidence.
It also said the manner in which the society gathers information for guidance 'needs to be more clearly documented'.
BTS chief executive Sheila Edwards was 'extremely disappointed' and said the group felt 'a bit like guinea pigs' after the decision.
But the society will not appeal the decision and will focus on reapplying in January 2011.
Ms Edwards said: 'We are continuously improving our processes and will take on board what they said.'
The society is the first organisation seeking NHS Evidence accreditation not to be accepted. A first draft of the society's application was turned down in November 2009.
The BTS produces the Scottish Intercollegiate Guidelines Network (SIGN) guidance, and guidance on conditions such as asthma, smoking and TB.
Professor David Haslam, NHS Evidence advisory committee chair and former RCGP president, said it was right for NHS Evidence to challenge organisations on how guidance is developed.
While he understood the BTS may be disappointed, he paid tribute to its hard work and commitment during the 'tough' accreditation process.