One of the things I was most annoyed by while revising my book on health inequality was the discontinuity in some of the sources of data. Perhaps the most important of these is the interruption to the series on class differences in mortality that goes back to 1931 in England and Wales. I had thought it would be an easy matter. In the 1st edition of the book there is a table that shows how social class differences in mortality during working age steadily increased up to 1991. In 1931, mortality in working age men in the most advantaged social class made up of professionals and managers was around 10% less than the average for all working age men and mortality in the least advantaged class made up of nonskilled manual jobs was about 11% higher than average. By 1991, the equivalent figures were 34% lower for the most advantaged class and 89% (yes, that is not a typo) higher for the least advantaged. All of these figures are adjusted to take account of the fact that the different social classes may be made up of people with different ages. For example, men may be older by the time they get into management (although if this influenced the result it would in fact be the other way around as older people have higher mortality).
So I thought, no problem, lets look up what happened to these figures in 2001 and maybe 2011 as well. However, there had been 2 large changes to the way the figures are calculated.
The first and least problematic of these is that the way in which it is decided which occupations go into which social classes has been clarified and put on a much more scientific basis. If you want to look more closely into this the web site http://tinyurl.com/h3yxh34 might help. Although this "class schema" , called the NS-SEC, is not the same as the "Registrar General's Social Classes" (RGSC) that were used betwee 1931 and 1991, we know from lots of work that different health measures do vary by NS-SEC in a very similar manner to RGSC. And because there is a clear logic to why occupations are put in the different SECs, it means that the study of health inequality using this measure should get a lot more scientific than it was before.
What is more infuriating is where the numbers needed for the calculation of class differences in mortality now come from.
Between 1931 and 1991, the numbers came from 2 sources. To get a rate of mortality you need a numerator (the number of deaths in a social class) and a denominator (the numbers of people in that social class). Up to 1991, the numerator came from death certificates, because people's death certificate includes what their job is. And the denominator came from the Census, because at the Census you know how many people in each occupation there are in the country, add up the appropriate occupations into the social classes, and Bob's your uncle. The limitation here is that social class differences in mortality can only be calculated every 10 years when there is a Census. This way to calculate health inequality is called the "Unlinked method".
However, in the 1980s there was a bit of a panic about the way in which class differences in mortality had been rising to much. Some people guessed that the unlinked method might give a biased impression. What if people's relatives told the Registrar of Deaths a higher status job than the one they really had before they died? In fact, if this had been happening (think about it) it would have reduced health inequality, not increased it. But never mind, the outcome was the most wonderful data set. The ONS Longitudinal Study linked 1% of the population at each Census of England and Wales to future events, such as mortality. So instead of the numerator and denominator being take from different data sources, we could now calculate class specific rates of mortality from data from the same people: they gave their occupation to the Census and this could be linked to the information on their death certificate. But this actually led to some pretty big problems of its own.
I am just beginning to realize what a complicated topic this is! It stands as an example of the fact that "Big Data" may entail a lot more thought than some people seem to realize.
To cut a long story short, eventually the estimate of the size of class differences in mortality came to be taken from yet another set of numbers. This time the numerator was taken from death certificates again (with the occupation of the deceased person taken from the certificate) and the denominator was taken from something called the Labor Force Survey (LFS). There are some advantages to this. The LFS is done every year (unlike the Census). Although it does not count everyone in England and Wales, it is a large survey and the numbers can be multiplied ("grossed up") to give an idea of how many people are in each occupation in each year. BUT the LFS will have non-response. Unlike the Census, it is no obligatory for a person to take part in the LFS. So we have now moved from a denominator taken from an obligatory census of everyone in England and Wales, to one taken from an annual survey of (I think) around 40,000 people, some of whom may refuse to take part.
On the one had, this should not necessarily lead to an under-estimate of social class differences in mortality, because non-responders to surveys tend to be people living in more adverse conditions. Since everyone gets a death certificate, it is most likely that more disadvantaged people are more likely to have their death recorded than their occupation. Which would tend to give higher death rates in more disadvantaged groups. In addition, the way the measure is calculated has changed. This is probably a sensible change,
However this may be, the National Statistics office for England and Wales (Scotland and N Irelnd have thier own organisations) have given us as near as possible an estimation of what has happened to social class differences in mortality since 1991. Instead of the Standardised Mortality ratio used between 1931 and 1991, which compares the mortality rate in all social classes to a notional "average", we now have a measure that just gives the mortality rate per 100,000 adjusted to take account of age. ONS usefully goes back to 1971 and calculates the mortality rates in this way, then presenting a ratio of the rates in the most versus the least advantaged social classes. Having done this, what we see is that in 1971 this ratio was 1.8 (working age men in the least advantaged class were 80% more likely to die than those in the most advantaged), while in 2001 and 2010 the ratio was 28, (men in the least advantaged class were 180% more likely to die). Bear in mind that this is a comparison between the very best and the very worst employment conditions, not between the best and the average or the worst and the average. So it is bound to look rather a large difference. Which it is.
What would the comparison look like if we could go back to 1931 or even 1961? There is just no way of knowing.