How to tell if that new research study your friend posted on Facebook should be shared…or deleted.

Last week CNN posted an article with the title “Drinking non-cow milk linked to shorter kids, study suggests“.

Let me start by saying, that this statement is in fact correct…but not for all intents and purposes.  What does that mean?  Let’s start at the beginning.

In the media model, article views equals the ability to sell more advertising space which equals revenue for shareholders.  In the science model, replication of a well designed study by other well designed studies and producing the same result (a process that can take years, sometimes decades!) equals an answer that may then be worthy of writing an article about. These organisational models are living in parallel universes where time between them runs at different speeds.  And this is a shame for the consumer.

Imagine a world where as a consumer, you had access to an instant meter of how valid the results of any research study was according to some universally accepted scoring criteria so you weren’t at risk of consuming erm…trash.

Before you read another click-baiting, crowd-pleasing, over-shared, under-researched article, I’m going to jot down a few things for you to take note of, or to take a few extra minutes to research after you read any article reporting on a new scientific finding.  I challenge you NOT to either share the research nor commit the findings to memory until you’ve availed yourself of the facts surrounding the research design.

The most important question to ask is: Did the research study control for confounding variables?


A confounding variable happens when a researcher can’t tell the difference between the effects of different factors on a variable.  There are so many different things that can have an impact on the results of the study and so understanding what “data noise” to remove is critical to making sure that pattern in the data that a researcher might see is unlikely to be due to chance alone.

When reviewing the validity of research results, these are some of the red flags when it comes to research design:

    • Self reported behaviour surveys
      Humans can barely remember what they did on the weekend let alone on a daily or weekly basis five years ago!  That’s not to say that these studies aren’t relevant, simply that the evidence from them would not be considered as strong as say a study where the experimental design had people follow a pattern of behaviour (for the control and placebo groups) across a specific period of time and followed up with them regularly for self reporting across that time period.
    • The lack of a control group, test group (and in some cases) a placebo group
      A control group is a group of participants to whom the treatment isn’t applied, in the test group the variable that the researcher wants to test is introduced and in the placebo group, the participants think they are part of the test group, but they are receiving some sort of alternative to the treatment that will not yield the expected result.  The human brain and body are pretty powerful…when we think we’re getting we can actually experience improvements that don’t really exist!

      However, a placebo group is not always feasible depending on what is being tested so a bit of common sense needs to be applied here. For instance, if you were trying to test some sort of effect related to drinking water, you could have a control group who didn’t drink water, but given everyone knows what water tastes like, attempting to create a “water placebo” would be pretty tough. But a control and test group should be the bare minimum!  And in cases of medication where a placebo can be easily applied, there should ALWAYS be a placebo group.

    • Non randomised
      If the research is experimental in nature (and not survey based), and the report doesn’t say it’s randomised, then it probably isn’t.  A randomised experiment means that participants in the experiment (those put into either control, test or placebo groups) were randomly assigned assigned to them.  i.e. that the researchers weren’t in control of choosing who would be assigned to which group.  If they are, they can introduce all sorts of unconscious bias that could affect the results.
    • Non Blinded or Non Double Blinded
      A blinded study is where the participant in the research is unaware which group (i.e. test, control or placebo) they have been assigned to.  A double blind study is where neither the participant in the research, nor the researcher themselves, knows which group the participants have been assigned to.  That means the researcher might only see a number in place of an individual’s name and details when seeing the results. And the experiment may be designed in such a way that those responsible for data collection, and perhaps physically collecting data from the participants, do not communicate with the researcher (or may not even be known to them).
    • Small sample sizes
      A “sample” is basically a little portion of the broader “population”.  A population in research doesn’t have to mean the population of a country, it may just be the population within a particular category pertinent to the research.  For example “people with Diabetes”, or “people who have been treated for depression”, or “women who have given birth to at least one child”.  The sample size has, because there are random effects that can occur in small samples that even themselves out when you test the same thing on a larger sample size.

      Most good research might start with a smaller sample size to test an initial hypothesis (theory).  They’ll release initial results but caution that due to small sample sizes, more research should be done to see if the results can be replicated on a broader scale so that it can be .  If this is the first research in a particular area to come out and it’s got a small sample size you MUST treat it with caution. It means that it is essentially “baby research” it’s not fully formed yet nor capable of making truly informed conclusions about its own existence!

 

  • Non peer reviewed – i.e published in a dodgy journal 
    Yep, not all journals are created equal.  A good piece of research should be published by a journal that has a process whereby other scientific peers review the research methodology before accepting it for publishing.  Sometimes good journals will create a single-blind process for doing this – meaning that those reviewing the research don’t know who the author is.  That’s important – because humans have an innate bias to trust people who are perceived to be more credible, despite there potentially being a lack of evidence to support that trust.

    If it has been archived or cited here: https://www.ncbi.nlm.nih.gov that’s a good start.  Apparently this tool https://harzing.com/resources/publish-or-perish helps you figure out how many times the article has been cited in journals (although I’ve not used it before) and this one helps you figure out the ranking of the journal: http://www.scimagojr.com/journalrank.php?area=2700&type=j.

    Monash Uni have a bunch of good links and info about assessing journal quality here including:

  • Statistical significance without IMPORTANCE
    Statistical significance is generally agreed that there is either a 5% or lower (sometimes 1% for more rigorous research) probability that the results obtained were due to chance versus the variable being tested.

    That’s a great first step for sure, but significance does not mean importance. Once the study has met the above criteria, ask yourself one, final and very important question “Is this question the right question to be asking, and is the assumption that underpins the question being asked a correct assumption?”

 

In the next article I’ll use this cnn article to test drive some of the knowledge above.

The Politics of Hormones

For the last two days every human being and animal I’ve crossed paths with has pissed me off.  And I can’t control it.  It was only when my partner gingerly pointed out one day that I seem to want to kill myself (or everyone else) on the same 3 days every month, that I realised that it was overwhelmingly brought on by my monthly hormone cycle.

Double-flowered carnationsThis morning I attempted to make myself feel better by having a healthy breakfast, buying some carnations and painting each fingernail a different colour of the rainbow.  It turns out chucking a hissy fit and have a rage-fueled cleaning spree did the trick instead.  As I was attempting to stuff these beautiful carnations into a ridiculously narrow vase, one of the flower heads popped off and I quickly realised this vase just wasn’t going to work and tension began to build.  I pulled the stems out and briefly glanced around the room only to be struck by the vision capabilities of a vampire, zeroing in on thousands of grotty specs of dust.  The kitchen was a pigsty.  The tension burst.  I screamed and threw the flowers stems across the room like a two year old.  Thankfully no one else was there to witness it.

And there you have it – human hormones in action.  Have too little of one and you become depressed, too much of another and you’re angry.  These regulatory substances that our own bodies produce, sometimes in cycles, sometimes in response to medications or environment, are created and then transported by our blood to stimulate specific cells or tissues into action.  Basically, they’re little chemical messengers that help to keep everything in balance.  Testosterone, Estrogen, Insulin, Oxytocin and even Dopamine as a peripheral hormone – over 50 of them in total.

But as anyone who has attempted to live a balanced life knows…balance is not exactly simple, it’s a constant juggling act.  So here’s my poor female  body, prepping itself to make babies every month while the rest of me has to catch up and balance all its internal chemicals just right to keep me sane.  I’m lucky it can at least do that.

Many of us would like to think we have absolute final control over our actions and disposition, but our predisposition toward certain behaviours or traits are governed by these chemical compounds which are originally a product of genetics or epigenetics, but then subject to change based on our own epigenome and our environment.  For instance, what we choose to eat or not to eat (given foods are also chemicals), will also effect the chemical composition of our guts and our bodies.  Here’s some examples of the power of hormones:

  • Pre-op Female Transexuals take more Estrogen to stimulate breast formation, heightening of voice and change of fat distribution across the body although this cannot change the effects of the androgens (testosterone and others) on the shape of the skeleton. Visa versa for Pre-op Male Transexuals.
  • Men who take more anabolic steroids like testosterone to make bones stronger and muscles bigger can have the side effect of high blood pressure, sharp aggressive moods, increase in body heat and reduced sperm count.Chemical structure of oxytocin.
  • Oxytocin has different effects in men verses women.  In men it improves the ability to identify competitive relationships whereas in women it facilitates the ability to identify kinship.  It is released by the body in high doses during pregnancy and around childbirth in order to ensure proper boding between the mother and the child and it is also released during sex, which is why women tend to interpret sex as having more meaning than males do, because while it can also be released in men – the presence of testosterone interferes with its release.
Cover of "Emotional Awareness: Overcoming...

Cover via Amazon

I have been reading Emotional Awareness which is a book that covers a 3 week conversation between the Dalai Lama and scientist Paul Eckman (who is behind vast tomes of research on human emotion and, in particular, how it is expressed in minute facial expressions.  His work formed the basis of the TV show “Lie to Me”). In the book they talk about the difference between moods and emotions – where a mood is generally not tied to a specific event or circumstance but has a continuous emotional effect, whereas an emotional episode is much more able to be linked to a trigger event.  In some ways, moods trigger emotional episodes that reflect the mood itself, while emotional episodes are often in response to some sort of external trigger.

When I look at this in the context of hormones, I can see how hormones would often be the precipitants of moods, which would then trigger a variety of emotional responses.  The stronger the hormone and stronger the mood, the stronger the emotional response, and of course like all learned human behaviour, these patterns would continue to strengthen over time if performed again and again and can be how people slip into ongoing depression or aggression.

All this also got me thinking about how religion deals with hormones and their effect on humans.  Two particular instances I can think of relate specifically to women:

  1. In Sharia (Islamic) law, two women must bear witness to a crime in order to be heard in court.  This is stated to be due to the emotional nature of women, although on further research I found the story was related directly to Mohammad wanting to save one of his wives from being stoned to death by being caught all one with another man by a woman.  It seemed he often made up rules to suit his worldly purposes.
  2. In Jewish law, a man cannot touch a woman who is in the bleeding section of her monthly cycle.

Interestingly, there seems to be no similar law governing the display of aggression in men which is similarly a hormone fueled disposition.  Perhaps from an evolutionary perspective this is because the male aggression hormone was highly useful during times of empire expansion and the need to protect land, while the female hormal response in its monthly cycle had no broader benefit (from the patriarchs perspective).

Because for the majority of history, both East and West have lived in patriarchal societies, the best way to look at hormones being a determinant of military and political outcomes is through a male sex hormone like Testosterone.  As an example, research in 1992 showed there was no difference in Testosterone levels between black and white boys during adolescence, but in adulthood, black male testosterone levels were much higher which then directly links to other statistics we see on the number of incarcerations of black verses white males.  Of course there are so many other factors involved including race related discrimination and levels of education…etc but hormones remain a factor.  For instance in contrast, there was significant racism in Western countries toward those of Chinese descent and yet their incarceration rate remains lower.  So if we widen out that thinking from groups to countries, we see the possibilities for testosterone levels in any particular age generation to change and potentially fuel the ability for one clan (or country) to change military outcomes, or to resort to military resolutions over negotiation.  Environmental factors also come into play with research showing in adult male rats that experience short term starvation can significantly reduce testosterone levels.

So here we have a mix of chemicals in our bodies which have the potential to guide us to achieve great good or great destruction and yet we have only scratched the surface in controlling their effects through various hormone replacement therapies that often have many unwanted side-effects of their own.

Fitbit: Wireless Personal Trainer

Fitbit: Wireless Personal Trainer (Photo credit: mrcd@sbcglobal.net)

Interestingly I recently did an interview with Jonathan Teo as part of a Fireside chat run by our meetup group Lean Startup Melbourne.  He led initial investor rounds into Twitter and Instagram and more recently Snapchat and is one of the key people in the tech industry who has been able to predict new technology trends.  One of the key trends being talked about in the tech industry at the moment is wearable tech (being able to monitor body functions insitu) – which is in part being quickly moved forward and expanded by breakthroughs in nanotechnology and size/power of computer chips.

At our last meetup, one of the speakers was from Lab on a Chip in Melbourne who talked about the real possibility in the near future of capabilities being produced that would enable the immediate mapping of genomes and epigenetic markers and then the smart releasing of medications or hormones through a patch (just like a smoking patch…but way smarter!).
Here’s hoping there’s a nanotech-hormone-patch for PMS before I smash a plate….or five.