Author:

Anna Wagstaff

Two British medical epidemiologists, Richard Doll and Austin Bradford Hill, conducted the epidemiological work that proved beyond doubt that tobacco use causes the great majority of lung cancers. Their work was a crucial contribution to informing global policies aimed at reducing tobacco use, which have saved millions of lives. The size and methodological rigour of their study on Smoking and Carcinoma of the Lung, published in British Medical Journal in 1950 provided a level of certainty about the causal link that had been missing from smaller methodologically flawed studies published in the 1930s and ’40s. Bradford Hill would go on to codify how to determine causative links between the environment and disease in a 1965 paper setting out nine ‘guidelines’ that remain a standard reference point in courts of law. But it was the novel use of a prospective design for the study of Mortality of Doctors in Relation to their Smoking Habits – ‘the British Doctors’ Study’ – that finally convinced the medical world, and therefore policy makers, that cigarettes were the major culprit behind rising rates of lung cancer, and prompted them to act. The study, which asked almost 60,000 UK doctors about their smoking habits, and prospectively correlated the results with deaths from lung cancer, produced such startling results that the preliminary data were published halfway through its intended five-year duration. Although there were only 36 lung cancer deaths in the first 29 months of the study, after five years there were 70, almost all of them in heavy smokers. The incidence in non-smokers was negligible.

Historical context

Lung cancer still accounts for by far the greatest number of cancer deaths worldwide, and tobacco use accounts for around eight in ten of all lung cancer cases. Yet when public health officials across Europe and the US began to raise concerns about a rapid rise in lung cancer in the early 20th century, pinpointing the cause behind this rise proved surprisingly difficult.

Medical science lacked the analytical tools for the job. The previous decades had seen rapid advances in identifying the pathogens behind infectious diseases, guided by Koch’s ‘four postulates’ for establishing a causal relationship between a causative microbe and a disease. But no such analytical tools had been developed to explore causality in cases of non-infectious diseases.

Suggestions that smoking could be responsible for the rise in lung cancer were therefore widely met with scepticism in the medical world, on the grounds that the association did not meet all of Koch’s criteria for causality: it was not the case that everyone who smoked got lung cancer, whereas it was the case that some people with lung cancer had never smoked.

While epidemiology was a well-established branch of medicine, its use had been confined largely to looking at differences between entire populations – rainfall, temperature, workplace exposure to potential toxins, water sources and so on. Methodologies for investigating individual ‘lifestyle’ differences in ways that could deliver robust conclusions, had not been developed.

A contributory factor clouding the picture at that time came from laboratory science. The idea that environmental exposure to toxins could cause cancer was well accepted, with multiple examples including testicular cancer in men who had worked as chimney sweeps in childhood, or cancers in people working with various chemicals for instance in making dyes. By the beginning of the 1920s scientists were already testing materials for cancer causing properties by smearing them on mice to see if a skin cancer developed. However, when this approach was tried with tar from cigarette smoke, the negative results strengthened scepticism over the link.

Meanwhile, although the time-lagged correlation between the increase in cigarette sales and the rise in lung cancer in the first decades of the 20th century had not gone unnoticed, other correlations were seen as more likely causes – not least the rapid rise of car use and more particularly the massive road-building programme that exposed people to fumes from hot tar. Many in the medical community also questioned whether the rise in lung cancer was real, or simply the result of recent improvements in diagnostic reporting.

In hindsight, the complete absence of any suspicion regarding the link between smoking and lung cancer as late as the 1950s can be hard to believe. That this was so, however, was repeatedly stressed by Richard Doll in his many interviews. He claimed that he himself did not expect to find smoking was a major problem, “I would have bet on roads and motorcars”. The lack of concern about smoking among the medical community at that time can be gauged from the astonishing statistics of doctors’ own smoking habits recorded in the British Doctors’ Study, which showed smoking rates, by age group, ranged from 83% to 91%. Undoubtedly it was precisely that extremely high smoking prevalence, which reflected the wider male smoking habits at the time, that kept smoking off the list of probable causes of lung cancer, which while markedly rising in incidence still affected only a small minority of smokers.

The studies

When the marked and persisting rise in deaths from lung cancer came to the notice of UK government statisticians, they turned to the Medical Research Council (MRC) for advice. Austin Bradford Hill, a Professor of Medical Statistics at the London School of Hygiene and Tropical Medicine, was Honorary Director of the MRC Statistical Research Unit at the time, and the task of confirming whether or not the increase was real, and if so, what was the cause, fell to him.

In 1947 Bradford Hill set to work planning a study “on a sufficiently large scale to determine whether patients with carcinoma of the lung differed materially from other persons in respect of their smoking habits, or in some other way which might be related to the atmospheric pollution theory”. In 1948 Richard Doll joined the MRC and started working with Bradford Hill on the research project.

Though smoking was not on the radar of any public health or medical organisation at the time, the possibility had been raised in the literature as far back as 1912, and a few small-scale and poorly controlled studies into a possible link between smoking and lung cancer had been already been done in Germany and the US, with an unpublished study in the UK, all of which pointed to a possible connection. Though none of these studies provided robust evidence, their findings did all point to a possible causal association.

Then in May 1950, a few months before the publication of the Doll and Bradford Hill, two US epidemiologists – Ernest Wynder and Everts Graham – published the findings of the largest and most rigorous study to that date, under the title “Tobacco smoking as a possible etiologic factor in bronchiogenic carcinoma”. That study, which compared smoking habits of almost 700 patients with lung cancer against patients hospitalised with other conditions, as well as patients hospitalised for non-cancer lung conditions, found that only 1.3% of lung cancer patients were non-smokers compared with 14.6% in the general hospital population, while 20.3% of lung cancer patients were ‘chain smokers’ compared with only 7.6% in the general hospital population. The authors concluded that “excessive and prolonged use of tobacco, especially cigarets, seems to be an important factor in the induction of bronchiogenic carcinoma.”

The Doll and Bradford Hill study was published in the British Medical Journal in September 1950. While its design resembled the Wynder and Graham study in many ways, it represented one of the largest questionnaire-based epidemiological studies, using some of the most advanced techniques for controlling for bias, of any statistical study of its day. The findings allowed the authors to conclude with a high degree of certainty that, “there is a real association between carcinoma of the lung and smoking.”

As Doll would later recall, a belief in their methodology gave them trust in the reliability of their findings, “… we were the first to have sufficient confidence in our findings to state that: ‘We conclude that smoking is a cause and an important cause of the disease…’ We were confident of our data because we had taken steps to ensure that our results were robust.”

The methodology

The study was originally intended to cover every patient admitted to 20 London hospitals with carcinoma of the lung, stomach, colon or rectum. One purpose of including patients with other cancers was to avoid bias from the interviewer, who might ask, or interpret responses, differently if they knew the patient had lung cancer. In practice, this attempt at blinding turned out to be impractical, but the authors found alternative ways to control for this possible bias. The problem of patients’ responses being biased as a result of knowing about their diagnosis seems not to have arisen, as in those days patients were rarely told of their cancer diagnoses. Attempts were made, however, to control for recall bias, by returning to a selection of patients six months after the initial interview, to ask again about their smoking history, to see whether the two sets of responses were consistent.

Patients would be interviewed by one of a group of only four social workers, whose only job was to conduct this research. Responses from patients diagnosed with cancers other than lung cancer were used as controls, but the questionnaire was also designed to elicit information that could throw light on aetiological factors relevant to those cancers. In addition, for every cancer patient who was interviewed, the social worker who did the interview was required to use the same questionnaire with a patient of the same age group and gender, admitted to the same hospital, but for a non-cancer diagnosis.

Between April 1948 and October 1949 those 20 London hospitals notified the study of 2,370 patients admitted with lung, stomach, colon and rectal cancer. Given the level of uncertainty surrounding a diagnosis made early in a patient’s journey, checks were made at a later point to correct for cases where the initial diagnosis was materially changed.

Where patients notified to the study could not be interviewed, reasons for the failure to interview were provided, and an analysis made to exclude the possibility that this could bias the results in a certain direction.

Differences between lung cancer patients and controls regarding place of residence and social class were analysed, which showed a small but not significant difference in social class, but a more marked difference in place of residence, with more lung cancer patients coming from out of London. An analysis revealed that the difference was a feature of patients within specialist cancer hospitals, which received cancer patients from around the country; the data from general hospitals in London revealed no higher proportion of out of London patients among the lung cancer patients compared with non-lung cancer patients. (On reading the unexpectedly compelling results from the study, the director of the MRC, Harold Himsworth, nonetheless requested the study be repeated in other cities to double check the findings before it was published) .

The study report included a detailed section on “interpretation of results”, where the authors go through the potential sources of bias and how they had accounted for them. They also spell out that association does not equal causation, and argue their reasoning for why alternative explanations – that lung cancer causes people to smoke or a third factor links smoking with the development of lung cancer many years later – do not seem plausible.

The headline data from the 1950 Doll and Bradford Hill report is that 4.2% of patients without a lung cancer diagnosis reported never smoking, compared with only 0.3% among those with a lung cancer diagnosis – a 14-fold difference. The authors also give significant weight to the finding that the likelihood of lung cancer increased according to the amount of tobacco consumed – “in lung-carcinoma patients, there is … a significantly higher proportion of heavier smokers and a correspondingly lower proportion of lighter smokers than in the comparative group of other patients” – and that this finding was consistent across three alternative ways they had used to measure tobacco consumption over time. The point was reinforced with a population level graph showing the UK data on deaths per million from lung cancer against per capita consumption of tobacco and cigarettes between 1900 and 1950.

Bradford Hill had been asked to find out whether the reported rise in UK lung cancer deaths was real or not, and if it was, what was the main cause? With the 1950 study, he and Doll answered the question. The results were surprising and conclusive. But to their dismay, it seemed that neither the medical community nor policy makers were interested.

The British Doctors’ Study

Doll himself gave up smoking as a result of that study. Both he and Bradford Hill understood the importance of getting that message across to the public, to allow them to take informed decisions about their own health. Both, however, felt that their standing as scientists dealing objectively with statistical information would be compromised if they were to get involved in making recommendations or advocacy.

That is how they hit on the idea of doing a second study using a completely different – prospective – design. Commenting on their reasoning behind the second study, Doll and Bradford Hill said, “Some have considered that the only reasonable explanation [for the findings of the 1950 study and others showing similar results] is that smoking is a factor in the production of the disease; others have not been prepared to deduce causation and have left the association unexplained.”

“Further retrospective studies of that same kind,” they argued, “would seem to us unlikely to advance our knowledge materially or to throw any new light upon the nature of the association. If, too, there were any undetected flaw in the evidence that such studies have produced it would be exposed only by some entirely new approach. That approach we considered should be ‘prospective’.”

The novelty of this approach, at least in an epidemiological setting, can be judged by the fact that they attached a footnote to the word ‘prospective’, which gave the formal definition of the term according to the Oxford English Dictionary – “Characterized by looking forward into the future,” together with a quote from the poet Leigh Hunt, to illustrate how the word is used in practice!

The two researchers chose as the study population members of the British medical profession, as listed in the British Medical Register, numbering almost 60,000 people in all. The questionnaire, mailed out in 1951, enquired about their smoking habits, keeping the questions as simple as possible to maximise responses. Respondents were asked to classify themselves as either current smoker, former smoker, or never smoker, with further questions posed to those in the first two categories to enable them to be further classified into three levels of tobacco consumption. They chose prospectively to exclude responses from women doctors and doctors under the age of 35, because of the very low number of expected lung cancer cases. The analysis was made on the basis of almost 25,000 survey responses.

They then enlisted the collaboration of the Registrars General in the UK, responsible for registering births, marriages and deaths, to provide them with “the particulars of the cause of death... For every death of a doctor registered since the questionary was sent out.” They expected it would take at least five years to get a meaningful signal. As it turned out, they got that signal in less than half that time. Their first results, published in 1954, were based on notifications of 789 deaths, of which 36 were due to lung cancer. None of the doctors who died from lung cancer were never smokers, while the rate of lung cancer deaths among smokers rose steadily according to how much they smoked, following a very similar pattern to the findings of their previous retrospective study.

The difference between the number of observed deaths from lung cancer and the number that would have been expected if smoking were unrelated to the disease was calculated at that early point to be significant, at P=0.04. As the authors point out, that figure was almost certainly an underestimate, because doctors ill and dying from lung cancer (or indeed other diseases) were likely to be over-represented among the group that did not respond to the survey.

Five years into the study, the number of lung cancer deaths had reached 70, adding further certainty to the preliminary findings. In 1957 the Government ask the MRC for a formal opinion on whether smoking was the cause of the rapid increase in lung cancer cases, and the MRC formally considered all the studies and answered that it was. The Minister of Health called a press conference to announce the opinion of the MRC. The impact of that announcement will have been somewhat undermined by the fact that as he made the announcement, he was himself smoking a cigarette.

However, as Doll would later comment, choosing to do the study among doctors, which was done “principally because they were easy to follow up,” made a big difference in getting their results taken seriously. “It turned out to have been very fortunate to have chosen doctors, from a number of points of view. One was that the medical profession in this country became convinced of the findings quicker than anywhere else. They said, ‘Goodness! Smoking kills doctors, it must be very serious,’ and of course a very high proportion gave up.”

Though the study was intended only to run for five years, the value of the results it generated continued to be followed up for 50 years, making a major contribution to shaping national and global policies as well as personal decisions about quitting.

By 1956 on deaths from coronary heart disease was strong enough to confirm smoking as an important risk factor. Further questionnaires about changes in smoking habits were sent in 1957, 1966, 1971, 1978, 1991, 1998 and 2001. The 1978 questionnaire also sought information from all male doctors born in the 20th century about a wider range of characteristics (including alcohol consumption and self-reported body mass index) and invited them to participate a randomised trial of prophylactic daily aspirin to prevent death from stroke, myocardial infarction, or other vascular conditions. The results from the trials informed the design of larger trials to learn more about these issues.

Some of the most impactful evidence developed as part of these studies relates to research showing how risk of lung cancer reduces over the years after giving up smoking. Much of that evidence came from a 1990 study, led by ‘Smoking, smoking cessation, and lung cancer in the UK since 1950’, which was modelled closely on the 1950 retrospective study of hospital patients, with the results of both studies being included in the analysis. Data on the benefits of quitting was also derived from the successive Doctors’ Study questionnaires.

The findings showed that the cumulative risks of life-long smoking by 75 years of age are 15.9% or men who continue to smoke cigarettes and 9.9%, 6.0%, 3.0%, and 1.7% for those who stopped around 60, 50, 40, and 30 years of age.

That research continues to have a massive impact at a global level in saving lives and protecting health, by providing those most at risk – current smokers – with reliable data specific to their own situations, to help inform their decisions about quitting, and motivate them to stay the course.