Tuesday 8 April 2014

Nature, the journal, and the Nature of Samples

 

As you will know by now, I lead a quiet life, avoiding trouble wherever possible. The natural calm of my afternoon was shattered by a Nature News piece with the startling and quite specific headline: “Stress alters children's genomes”. As if the drama of the finding was not enough, Jyoti Madhusoodanan’s article added further details: “Poverty and unstable family environments shorten chromosome-protecting telomeres in nine-year-olds”.

http://www.nature.com/news/stress-alters-children-s-genomes-1.14997

They shorten the genome. Not, they are associated with shorter genomes. They take a healthy long genome and shorten it. Imagine what the shock of the headline must have done to my genome. On your behalf, I read on, undaunted. Nature had posted up a nice picture of telomeres, which is usually enough to win over the unconvinced. Ever the sceptic, I read on, and came across these prize paragraphs:

When researchers examined the DNA of 40 boys from major US cities at age 9, they found that the telomeres of children from harsh home environments were 19% shorter than those of children from advantaged backgrounds. The length of telomeres is often considered to be a biomarker of chronic stress.

The telomeres of boys whose mothers had a high-school diploma were 32% longer compared with those of boys whose mothers had not finished high school. Children who came from stable families had telomeres that were 40% longer than those of children who had experienced many changes in family structure, such as a parent with multiple partners.

At this point I wondered who would be silly enough to imagine that n=40 was a suitable basis for concluding anything, and whether anyone would be even more silly to imagine that if there were observed differences between bright and dull mothers that it followed that those differences were caused by independently existing stressful environments, rather than being due to prior differences in the genetics and the behaviour of brighter mothers.

To spell it out: dull mothers are at risk of all sorts of things, from their genes upwards and outwards; brighter mothers might be spared those risks for reasons which range from their genes upwards and outwards.

However, I knew that I must have got it wrong, because such foolish errors would never be tolerated by Nature. So I had a look at the paper.

http://www.pnas.org/content/early/2014/04/02/1404293111.full.pdf+html

Here is the method statement:

Initially, we identified 40 families based on a three-step process. In the first step, the sample was constrained to boys meeting the following conditions: (i) boys provided saliva at age 9 (wave 5) in-home interview, (ii) whose mother self-identified her race as black or African American, (iii) for whom no data were missing on the criterion variables described below, and (iv) who were male.

Next, we arrayed the subsample on an index of advantage–disadvantage from birth to age 9 based on an equally weighted combination of (i) family economic conditions, (ii) parenting practices, and (iii) family structure/stability.

Finally, we took children with the 20 highest scores in the disadvantaged index whose mothers had experienced at least one depressive episode and the children with the 20 lowest scores on the index whose mothers had never experienced depression. Thus, boys who scored highest on this index (n = 20) lived in homes with high levels of poverty, high levels of family instability, harsh parenting, and maternal depression. Boys who scored lowest (n = 20) lived in affluent, stable families and were not exposed to either harsh parenting or maternal depression. We then assayed the children for Telomere Length.

So, from one perspective we can say they chose a bad genetic group and compared it with a good genetic group. From another perspective we can say they chose a bad luck group and compared it with a good luck group.

What steps did the authors take to compare these two perspectives? What steps did they take to distinguish, for example, between between bad luck on the one hand and bad decisions on the other; between the slings and arrows of blind fate on the one hand and the natural consequences of damn fool decisions on the other?

As far as I can tell, none. They assume that all this bad stuff rains down on one group and that the other group is spared, but that in genetic terms both groups are identical, or close enough to warrant a comparison of the effects of these extraneous life events.

I looked up the telomere lengths and did a t test. For the harsh environment 9.6 (4.2) and for the nurturing environment 10.3 (2.5). 20 boys in each sample. Non-significant. Nowhere close. So, I presume it is only significant if you put together a model of variables to be controlled for, but otherwise not.

To my primitive eyes the sample size seems far too small to conclude anything much, and far too small for comparisons of individual gene effects. In addition, there is a highly questionable assumption that all that these boys inherited from their parents was an “environment”. Incidentally, so many fathers were untraceable in the “unlucky” group that fathers had to be left out of the whole study. A look at the genetics of the mothers would be a start. Looking at the telomere length of the mother’s DNA might also be worth a look.

Have I got it entirely wrong, and is there some innocent explanation I have missed?

I would like some assistance from anyone who can help me understand why this paper, which I think flawed, is considered suitable for publication by the editors of Nature.

11 comments:

  1. I should add, the paper was commended in Nature, not actually published there.

    ReplyDelete
  2. you nailed it with: "they chose a bad genetic group and compared it with a good genetic group."

    Ironic that "Nature" prefers it not be due to nature (!)
    ...presumably to assuage the populace.

    ReplyDelete
  3. It's science Jim, but not as we know it!

    Gordo

    ReplyDelete
  4. Andrew Sabisky9 April 2014 at 00:16

    this is BS. Data like this is at the very most some kind of sketchy basis for a prior, not anything to form a coherent conclusion from. Even some decent minority of the general public without much scientific training knows this. What a joke.

    ReplyDelete
  5. Too busy moving to write much, but two seconds on pubmed led me to "Sizing the ends: Normal length of human telomeres". To quote:

    "3.3. Inheritance of telomere length
    The variability of telomere lengths between individuals has led to the emergence of the telomere inheritance theory. An early study tested this hypothesis by studying average telomere lengths in monozygotic and dizygotic twins and found that telomere length differences among individuals seem to be largely genetically determined (Slagboom et al., 1994). Another group confirmed this inheritance by studying the relative telomere lengths on homologous chromosomes of monozygotic and dizygotic twins. In this study, the authors found that the relative length of an individual telomere having the same genetic origin from monozygotic twins is identical."

    ...

    "A subsequent study from the same group showed that relative lengths of some specific telomeres of identical genetic origin are similar in parents and offspring (Graakjaer et al., 2006).

    ...

    "Besides a genetic component, several environmental factors, such as obesity, smoking, and psychological stress, have been shown to be negatively correlated with telomere lengths (Collins et al., 2003, Epel et al., 2004, Epel, 2009, Simon et al., 2006 and Valdes et al., 2005)."

    Note that these last ones are merely correlational.

    ReplyDelete
    Replies
    1. Of course the twin studies used weren't done in different environments, so the potential effects of very bad environments weren't measured.

      Delete
  6. Thanks. As I suspected, but your instant 2 second research is most welcome.

    ReplyDelete
    Replies
    1. The original article was published in PNAS: http://www.pnas.org/content/early/2014/04/02/1404293111.full.pdf+html

      I see a "Gene–Environment Interactions with Telomere Length" and a "Social Predictors of Telomere Length" section, but none for "Genetic predictors of telomere lengths". They do not even consider the possibility that the "advantaged" (quite possibly higher genetic quality) groups had longer telomeres encoded in their DNA, even though there are great individual differences in TM-lengths that are to a very large degree genetically determined.

      This study seems embarrassingly bad.

      Delete
  7. If you look at the variance for the TL length it's enormous, also the actual measurement of individual TL length has a error of +/- 10% which is itself larger than the difference between the groups. Even leaving aside genetic confounds, the problematic selection process and the ridiculously small sample in the regression only 1% of the variance is explained by environmental difference. Also the article doesn't give us any idea of what the normal mean and variance is for the TL measure.
    Like you I am not sure how they massaged the .19 difference out of the figures since the differences in length and log length aren't significant. How do you get these papers published ?

    ReplyDelete
  8. PS - you're in good company: in Svante Paabo's new book "Neanderthal Man," he repeatedly derides the journals Nature & Science.

    He shows a penchant for accessible & meticulous documentation of complex procedures - & bonus: a pretty good sense of humor.

    ReplyDelete