Sleep Deprivation Effects | Part 3

This part of Matthew Walker’s book on sleep is perhaps the most enthralling and simultaneously terrifying.  Sleep deprivation, more than anything else has been linked to heart disease, diabetes, cancer, gene transcription error and more.

My sister doesn’t live on normal-people time, both with night shift work once or twice a month, as well as very irregular sleeping hours.  And this line from the book in particular had me really hoping she would read this book and consider her sleep habits:

“The scientific evidence linking sleep disruption and cancer is now so damning that the World Health Organization has officially classified nighttime shift work as a “probable carcinogen”. (p166)”

Sleep Deprivation And The Brain

  • Research by David Dinges (University of Pennsylvania found in research that those who obtained six hours of sleep a night for ten days became as impaired in performance as going without sleep for twenty-four hours straight (p136)
  • With chronic sleep restriction over months or years, an individual will actually acclimate to their impaired performance, lower alertness and reduced energy levels and won’t be able to recognise their sub optimal existence. (p137)
  • Researchers in Australia found that after being awake for 19 hours, people who were sleep deprived wee as cognitively impaired as those who were legally drunk. (p138)
  • Infact each hour of sleep loss increases the likelihood of a crash.  E.g. At 6-7 hours you’re 1.3 times more likely to have a crash, and at less than 4 hours, you’re 11.5 times more likely to have a crash. After around 16 hours of being awake, the human brain begins to fail. (p139)
  • Sleep loss PLUS alcohol is not additive, it is multiplicable.
  • Vehicle accidents caused by drowsy driving exceeds those caused by alcohol and drugs combined.  Drowsy driving alone is worse than driving drunk.  When you’re drunk you will be LATE in reacting.  When you’re asleep, you stop reacting altogether. (p140)
  • Truck drivers are 200 to 500 percent more likely to be involved in a traffic accident.  And when a truck driver loses his or her life in a drowsy-driving crash, they will, on average, take 4.5 other lives with them. (p141)
  • The most dangerous time of flight in long haul travel is landing, which arrives at the end of a journey, when the greatest amount of sleep deprivation has often accrued. (p143)
  • A rare collection of individuals are able to survive on si hours of sleep and show minimal impairment.  The explanation appears to lie in the sub variant of a gene called BHLHE41 (also known as DEC2).  Expressed as a percentage of the population, the number of people who are like this are zero!  So it is very rare indeed. (p145)
  • Analysis of brain scans revealed the largest effects Matthew Walker has measured in his research to date – on the amygdala – which showed a 60% amplification in emotional reactivity in participants who were sleep-deprived. (p146)
  • Insufficient sleep doesn’t push the brain into a negative mood state and hold it there, instead it swings excessively to both positive and negative extremes. (p148)
  • Studies of adolescents have identified a link between sleep disruption and suicidal thoughts, suicide attempts and suicide completions in the days after. (p148)
  • Insufficient sleep also determines relapse rates in numerous addition disorders, associated with psychoactive substances. (p149)
  • Dr Allison Harvey from the University of California, Berkeley has found that should you improve sleep quality in patients suffering from several psychiatric conditions using cognitive behavioural therapy for insomnia (CBT-I), you can improve symptom severity and remission rates. (p151)
  • In one of Matthew Walker’s own experiments to understand the impact of students pulling “all nighters” – when comparing the effectiveness of learning between the two groups, there was a 40 percent deficit in the ability of the sleep deprived group to cram new facts into the brain relative to the group that obtained a full night of sleep.  That is the difference between acing an exam and failing it miserably. (p154)
  • In another test on 133 undergrads to learn a visual memory task, it was found that a night of sleep strengthened the newly learned memories, boosting their retention.  Additionally, the more nights of sleep participants had before they were ested, the better their memory was. Those who didn’t sleep the first night after learning, had no memory consolidation – i.e. if you don’t sleep the night that you learn, you lose the memories. (p156)
  • Dr Maiken Nedergaard at the University of Rochester found that a kind of sewage network called the glymphatic system exists within the brain.  This system collects and removes contaminants that are generated by the hard work performed by neurons in your brain.  It is at night, during deep NREM sleep that there is a 10-20 fold increase in the power cleansing going on in your brain.  The REASON the cleaning is so effective during this time is that the glial cells shrink in NREM sleep which allows he cerebrospinal fluid to clean out the gunk from that day’s neural activity. (p160)
  • Should you experimentally prevent a mouse from getting NREM sleep, there is an immediate increase in amyloid deposits (associated with Alzheimers) in the brain.  Put another way, wakefulness is low-level brain damage, while sleep is neurological sanitation. (p161)

 

Sleep Deprivation And The Body

  • A 2011 study tracked more than held a million men and women of varied ages, races, and ethnicities across with different countries.  Progressively shorter sleep was associated with a 45 percent increased risk of developing and/or dying from coronary heart disease within seven to twenty-five years. (p165)
  • A Japanese study of over 4,000 male workers over a 14 year period found that those sleeping 6 hours or less were 400 to 500 percent more likely to suffer one or more cardiac arrests than those sleeping more than six hours. (p165)
  • Part of the reason the heard suffers so dramatically under the weight of sleep deprivation is blood pressure.  Lack of sleep can pump up the pressure in the veins of your entire body. (p165)
  • Daylight savings is a “global experiment” in which 1.5 billion people are forced to reduce their sleep by one hour or less for a single night each year.  In the Northern Hemisphere, the switch to daylight savings time in March results in most people losing an hour.  In tabulation millions of hospital records, we find a frightening spike in heart attacks the following day.  The opposite happens when people gain an hour. (p169)
  • Does diabetes impair your sleep, or does short sleep impair your body’s ability to regulate blood sugar, thereby causing diabetes? In this experiment it was found that formerly healthy participants were 40 percent less effective at absorbing a standard dose of glucose, compared to when they were fully rested. (p171)
  • Do we eat more when sleeping less? In this experiment, the same individuals ate 300 calories more each day (1k calories per week) vs when they were getting a full night’s sleep. (p173).  Note on p175 – we don’t eat more when we are sleep deprived because we burn extra calories to stay awake.
  • A recent discovery has been made that sleep loss increases levels of circulating endocannabinoids, which are chemicals produced by the body that are very similar to the drug cannabis.  Like marijuana use, these chemicals stimulate appetite and increase your desire to snack. (p174)
  • In an experiment comparing patterns of brain activity when participants are shown “good” food vs “bad” food, found tat supervisor regions of the prefrontal cortex required for thoughtful judgements and controlled decisions are silenced in their activity by lack of sleep. The more prial deep-brain structures that drive motivations and desire were instead amplified in response to the food images – so high calorie foods became significantly more desirable to the sleep deprived, by 600 extra calories on average. (p176)
  • Evidence for the effect of sleep loss on obesity has been gathered over the past 30 years, back in 1940 when humans had close to 9 hours sleep a night, obesity was less than 5%, and continues its reverse trajectory through time with now the average sleep being 6.5 hrs and the obesity rate at 35%. (p177)
  • When losing weight, the amount of sleep you get affects the type of weight you lose.  If 6 hours or less, 70% of the weight lost is muscle, when sleeping correctly, 50% of the weight lost is fat, whilst preserving muscle. (p178)
  • Take a group of lean, health young males in their mid-twenties and limit their to five hours sleep for one week.  The hormone blending effect ages the man by 15 years in terms of testosterone virility. (p179)
  • Routinely sleeping less than six hours a night results in a 20 percent drop of follicular-releasing hormone in women. (p179)
  • Women who work erratic hours were 80% more likely to suffer from issues of sub fertility. 33% higher rate of abnormal menstrual cycles too. (p180)
  • Women who do become pregnant and routinely sleep less than eight hours a night are significantly more likely to suffer a miscarriage in their first trimester, relative to those consistently sleeping eight hours or more a night. (p180)
  • The less an individual  sleeps in the week before facing the active common cold viru, the more likely it was they would be infected.  In those sleeping five hours on average, the infection rate was almost 50%, in those sleeping seven or more hours a night in the week prior, the infection rate was just 18%. (p182)
  • A study in 2002 showed that sleep profoundly impacts responses to a standard flue vaccine.  Those participants who obtained seven to nine hours sleep in the week before getting the flu shot generated a powerful antibody reaction.  Those who were sleep restricted produced less than 50 percent of the immune reaction to their well slept counterparts.  Similar results have been reported for hep A and hep B vaccines too. (p183)
  • A brief dose of short sleep can affect your cancer fighting immune cells.  One night of 4 hours of sleep can sweep away 70% of your natural killer cells vs a full eight hour night of sleep. (p184.)
  • Lack of sleep also significantly affects cancer cell progression once taken hold.  Experiments from the University of Chicago (results found when mice were injected with malignant cells and tumor progression tracked over 4 weeks).  Sleep deprived mice suffered a 200 percent increase in the speed and size of cancer growth relative to the well-rested group. (p185)
  • Not getting enough sleep when fighting a battle against cancer is like pouring gasoline or an already aggressive fire. The scientific evidence linking sleep disruption and cancer is now so damning that the World Health Organization has officially classified nighttime shift work as a “probable carcinogen”. (p166)
  • Thousands of genes in the brain depend on sleep for stable regulation.  Deprive a mouse of sleep for a day and the activity of these genes will drop by over 200%.  Like a stubborn file that refuses to be transcribed by a printer, when you don’t lavish these DNA segments with enough sleep, they will not translate their instrutinal code into printed action and give the brain and body what they need. And the effect on humans is as pronounced as it is in mice. (p187)

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Advertisements

Sleep and Memory | Part 2

The second half of Why We Sleep starts like this “Amazing breakthrough!  Scientists have discovered a revolutionary new treatment that makes you live longer.  It enhances your memory and makes you more creative.  It makes you look more attractive.  It keeps you slim and lowers food cravings. It protects you from cancer and dementia.  It wards off colds and the flu.  It lowers your risk of heart attacks and stroke, not to mention diabetes.  You’ll even feel happier, less depressed and less anxious.  Are you interested?”

Hyperbolic as this may seem, nothing about this fictitious advertisement is inaccurate when it comes to sleep.

All the notes I highlight on sleep below are related to scientific studies performed by Matthew Walker and his team, or others in the scientific community.  For ease of writing and reading flow, I won’t note the sources here but I’ll give you the page in the book which elaborates in each case and the book index contains all of the sources.

 

SLEEP AND LEARNING

  • Which sleep confers the greater memory savings benefit? (deep NREM or REM)?  An early night, right in deep NREM. (p113)
  • Experimental results of Jenkin and Dallenbach have now been replicated time and time again with a memory retention benefit of between 20 and 40 percent being offered by sleep, compared to the same amount of time awake. (p113)
  • The more sleep spindles an individual obtains during a nap, the greater the restoration of their learning when they wake up (p110)
  • In 2006 a team in Germany ran a study to insert small amounts of electrical voltage during deep sleep.  When done pulsing in rhythmic time with the brain’s own waves, both the size of the brain waves and the number of sleep spindles were increased by the stimulation and provided a memory enhancement of 40% over the control group. (don’t try this at home!) (p117)
  • A swiss team suspended a bed frame on ropes and rocked the bed from side to side at controlled speeds.  Slow rocking increased the depth of deep sleep and boosted the quality of slow brain waves and more than doubled the number of sleep spindles.  (p118)
  • Using MRI scans, scientists have since looked deep into the brains of participants to see where those memories are being retrieved from before sleep relative to after sleep.  Information packets are recalled from completely different locations. Before sleep, participants fetch newly learned information from the hippocampus.  After sleep, from the neocortex. (p114)
  • Sleep stimulation efforts to date are indiscriminate – that is, you can’t really choose what to remember or what to forget. Science now has a new method called “targeted memory re-activation”.  Before going to sleep in this test, participants were shown not just images at different locations and this would be accompanied with a sound (e.g. a meow for an image of a cat).  When asleep, participants were played back the sounds on low volume with speakers at either side of their bed.  The memories recalled the next day were biased significantly toward those reactivated during sleep using the sound cues. This is the type of thing you could use for some SERIOUS brainwashing. (p119).
  • In another experiment, when participants were given words to remember as well as an indicator as to whether to remember or forget the word, after sleep (vs the non sleep group), memory was selectively boosted for those that had been tagged to be remembered.
  • Which stage of sleep determines what memories should be priorities vs removed? NREM sleep – and the very quickest of sleep spindles.  Eternal Sunshine Of the Spotless Mind here we come! (p122)

 

SLEEP IMPACT ON OTHER TYPES OF MEMORY

  • The term “muscle memory” is a misnomer.  Muscles themselves have no suc memory: a muscle that is not connected to a brain cannot perform any skilled actions, now does a muscle store skilled routines.  Muscle memory is, in fact, brain memory. Training and strengthening muscle can help you better execute a skilled memory routine, but the memory routine, the memory program – resides exclusively in the brain. (p123)
  • In an experiment teaching right handed people to type a number sequence with their left hand, those who slept showed a 20% jump in performance speed  and a 35% improvement in accuracy vs those who were tested before bed.  Those then tested before bed were re-tested in the morning and got the same bump in performance.  So the brain keeps improving skill memories without practice. (p125)
  • When the brain transfers skill memories, its not look fact / info based memory, the motor memories are shifted to brain circuits that operate below the level of consciousness.  (p127)
  • The type of sleep responsible for overnight motor skills enhancement is directly related to the amount of stage 2 NREM, especially in the last two hours of an eight hour night of sleep. (p127)

SLEEP IMPACT ON SPORTS

  • The less sleep an athlete has, the higher the the likelihood of injury.  At 6 hours average, the change of injury is 72%. What!!  Compared to just 18% at 9 hours average sleep or 35% at 8 hours average sleep. (p129)
  • One example of this is the difference in an NBA player’s stats when measuring performance on games played on nights after 8 hours or more vs those with less than 8 hours (p130):
    • 8 hours or more:
      • +12% increase in minutes played
      • +29% increase in points/minute
      • +2% increase in three-point percentages
      • +9% increase i free-throw percentage
    • Less than 8 hours:
      • +37% increase in turnovrs
      • +45% increase in fouls committed
  • At the most dramatic time of motor learning in any human’s life (when we learn to stand and walk as an infant), there is a consistent spike in stage 2 NREM sleep, incring sleep spindles, right around the time of transition from crawling to walking. (p131)

 

I’m splitting these posts into more parts so you can read it sooner – as at my current rate it might take me a couple of weeks to get through all the notes.  The next post will be on the impacts of sleep deprivation and health outcomes and then we’ll move on to the awesome world of dreams!

To drink cows milk or not to drink cows milk – that is the question

 

Righty-ho so let’s break down this CNN article by Wayne Drash which claims “Drinking non-cow milk linked to shorter kids, study suggests”.  The plan here is to assess some research about a very specific effect of cows milk on the height of young children so I’m not going to go into a bunch of other stuff around cows milk – just addressing the points of this research.

Remember from this article I wrote on “How to tell if that new research study your friend posted on Facebook should be shared or deleted” these were the red flags we needed to look out for:

  1. A self reported behaviour survey – especially one where the participant is being asked to recall something they did a long time ago
  2. The lack of a control group, test/treatment group (and in some cases) a placebo group
  3. Non randomised
  4. Non Blinded or Non Double Blinded
  5. Small sample sizes
  6. Non peer reviewed – i.e published in a dodgy journal
  7. Statistical significance without IMPORTANCE

So let’s review.  Firstly, we have to look BEYOND the article to the research itself.  The only link the article had was to the American Journal of Nutrition – there was no original source link but I found the original research paper by searching “Jonathon Maguire non-cow milk” (the name of the researcher and a keyword on the research topic) on trusty ol’ Google.

This is the link to the abstract for the research in the American Journal of Clinical Nutrition: http://ajcn.nutrition.org/content/early/2017/06/07/ajcn.117.156877.abstract

Sometimes you can take the full title of the abstract and do another Google search and find the full version posted somewhere (e.g. Association between noncow milk beverage consumption and childhood).  Very often I can find it on Researchgate which I have access to. Unfortunately I tried that in this case and it hasn’t been around long enough for any other sources to be hosting the full version.  If you’re a uni student, your uni email should give you access to a whole host of credible journals and so you’ll generally be able to access the full version.

Ok so for now we’ll have to work with what we’ve got – the research abstract.

  • Self reported behaviour survey: YES
    So we need to approach with caution.  The abstract says its a cross sectional study of kids enrolled in an existing research program called the “Applied Research Group For Kids”.  I looked that up and found this: https://www.ncbi.nlm.nih.gov/pubmed/24982016.  Looks pretty solid and as a longitudinal study, they’re basically asking a bunch of questions on a regular basis so participants aren’t having to remember what they did ages ago.  The program is also listed on the US National Library of MedicineNational Institutes of Health so I feel pretty safe about this.
  • Lack of control, test and placebo group: N/A
    Not really relevant here as its a longitudinal study which is just collecting a bunch of different data over time.
  • Non randomised: N/A
    As above, not relevant per above
  • Non Blinded or Non Double Blinded: N/A
    Not relevant as participants were not assigned to control, treatment or placebo groups because they weren’t testing any particular treatment
  • Small Sample Sizes: NO
    The sample size is 5,034.  This sample size is ok but I would use caution in applying these results more broadly because these are 24-72 month old Canadian kids, the majority of them are of caucasian background. Now given that 75 percent of African Americans and American Indians and 90 percent of Asian Americans are lactose intolerant – lactose intolerance develops in Asian and African genetic heritage in much higher rates than caucasian – this study may not be applicable in those cases. As a bit of background to lactose tolerance/intolerance, basically when you’re a baby you have a bunch of enzymes called “Lactase” which essentially helps your body to break down the sugar molecules in milk called “Lactose” and some of us don’t keep producing those enzymes once we’re done with breastfeeding! (some more info on this here).
  • Non Peer Reviewed: NO
    Nope this was published in a good journal.  Here are the results for the journal’s credibility.
  • Statistical Significance Without Importance: YES
    This is really the biggest problem with this whole study.  This is the assumption that the researchers have made “Cow milk consumption in childhood has been associated with increased height, which is an important measure of children’s growth and development”.

    That is true, but it’s also a misleading statement because they are not defining how much height is good.  Saying that “height is an important measure of child development ipso facto a taller child is a healthier child” is a fallacy.  This is like saying “Vitamin A is good for me so more is better”.  Well, that’s not true.  Too much Vitamin A can cause dizziness, nausea, headaches, coma and oh yes…death.Now I’m sure these researchers have good intentions, but they seem to have ignored previous research as a baseline or benchmark for their assumptions.

    Sure, western society holds up males in particular as being “better” for being taller.  But does that mean they’re healthier? Nope. All evidence points to shorter being healthier. This is info from the main US govt health site on this topic https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1071721/.  This overview of research to date on the topic is well worth the 15 minute read if you’ve got the time. Seriously, don’t take my word for it.

    Furthermore height is the main mediator of higher risk of hip fractures later in life for men – and is determined by their earlier intake of milk.  Here is the original study abstract for the above statement here: https://www.ncbi.nlm.nih.gov/m/pubmed/24247817/.

    The concern I have is how the researchers of the kids and nonmilk study choose to describe the background to their research:
    “Many parents are choosing noncow milk beverages such as soy and almond milk because of perceived health benefits. However, noncow milk contains less protein and fat than cow milk and may not have the same effect on height.”

    Is it true that soy and almond milk have perceived health benefits (that may, or may not be true depending on what research has shown vs what is widely believed). Yes that’s pretty true.

    And is it true that soy milk contains less protein than cows milk?  Let’s take a look…

  • 1 cup of Soy milk protein: 8 grams
  • 1 cup of Cow milk protein: 8 grams
  • 1 cup of Almond milk protein: 1 gram

    Gee…that doesn’t seem right…have they averaged out protein across Soy and Almond Milk perhaps to give “nonmilk” drinks an overall lower protein number? Because by my lay-person eye it looks to me as if they have the same amount of protein.  Of course milk may have other nutrient soy does not have and I’m not disputing that but why say something when it’s not true across the board?
    Surely you’d make sure you were looking specifically at who drank almond milk vs soy milk as a non milk alternative and understand the difference between both.
    Let’s ask a more important question.  Is it fat OR protein that contribute to height?
    Erm, neither exactly. Protein is part of it, but it’s actually calcium as well as the Insulin-like Human Growth Factor hormone that is present in cows milk at a much higher dose than that of human breast milk which is thought to contribute to height gain.

    And lets as a final and important question.  If western societies continue to promote health through consuming animal products through infancy through adolecence when more than than half of the world’s population is intolerant, what are we saying?  That only those who are lactose tolerant can be truly healthy? I’d love to see more studies that divide people into three groups:

  • Group 1: Conventional treatment (e.g. milk)
  • Group 2: Plant based supplement that meets all the same / similar nutritional properties that are thought to affect the variable (e.g. perhaps this is soy + other factors not present in soy as fortification)
  • Group 3: None or “on the market” options

Why are we only doing studies that suit a caucasian genetic makeup?

So what does all the above mean for this study ?
Well, the findings of association itself are not wrong.  But the assumption that is being created in the way the introduction is phrased are disturbingly misleading.  What should happen next?

As the abstract itself states at the end “Future research is needed to understand the causal relations between noncow milk consumption and height”.

This is absolutely true because milk consumption (or the specific properties within it) may not be the cause of different height growth.  Epigentic forces could be contributing to the outcome here.  i.e. mothers and fathers who drink noncow milk and plan to give noncow milk to their children could have a variety of other different diet habits that have a different impact at the point of conception and methylation.

Recent epigenetic research shows for instance that mothers who take less than around 300 grams of protein per day during the early stages of pregnancy alter the DNA methylation status at the site of the Vitamin E receptor gene and this contributes to the child’s percentage fat mass later in life.  I wonder, does it impact their height too?  Because if it does, this could be a critical factor in helping to explain this Canadian research as having an epigenetic cause.  It is possible that women who choose to drink noncow milk, may also have a lower protein intake in general which in turn contributes to these epigenetic changes upon conception!  PS that is just a completely untested hypothesis of mine…but it’s not entirely crazy I don’t think..

But furthermore, future studies should first seek to establish what is considered to be a bottom level AND top level healthy height in childhood through to adulthood.  And ensure that studies around the benefits for shorter adult height are fully considered in this assessment.

Only then is it really possible to start drawing insight around what types of foods/drinks to give children and give them the best chance of health throughout their lives.

—-

Lastly, while trying to find images for this post I came across the cutest awkward cow toy I have ever seen. 😀  I think I’m going to buy one!

How to tell if that new research study your friend posted on Facebook should be shared…or deleted.

Last week CNN posted an article with the title “Drinking non-cow milk linked to shorter kids, study suggests“.

Let me start by saying, that this statement is in fact correct…but not for all intents and purposes.  What does that mean?  Let’s start at the beginning.

In the media model, article views equals the ability to sell more advertising space which equals revenue for shareholders.  In the science model, replication of a well designed study by other well designed studies and producing the same result (a process that can take years, sometimes decades!) equals an answer that may then be worthy of writing an article about. These organisational models are living in parallel universes where time between them runs at different speeds.  And this is a shame for the consumer.

Imagine a world where as a consumer, you had access to an instant meter of how valid the results of any research study was according to some universally accepted scoring criteria so you weren’t at risk of consuming erm…trash.

Before you read another click-baiting, crowd-pleasing, over-shared, under-researched article, I’m going to jot down a few things for you to take note of, or to take a few extra minutes to research after you read any article reporting on a new scientific finding.  I challenge you NOT to either share the research nor commit the findings to memory until you’ve availed yourself of the facts surrounding the research design.

The most important question to ask is: Did the research study control for confounding variables?


A confounding variable happens when a researcher can’t tell the difference between the effects of different factors on a variable.  There are so many different things that can have an impact on the results of the study and so understanding what “data noise” to remove is critical to making sure that pattern in the data that a researcher might see is unlikely to be due to chance alone.

When reviewing the validity of research results, these are some of the red flags when it comes to research design:

    • Self reported behaviour surveys
      Humans can barely remember what they did on the weekend let alone on a daily or weekly basis five years ago!  That’s not to say that these studies aren’t relevant, simply that the evidence from them would not be considered as strong as say a study where the experimental design had people follow a pattern of behaviour (for the control and placebo groups) across a specific period of time and followed up with them regularly for self reporting across that time period.
    • The lack of a control group, test group (and in some cases) a placebo group
      A control group is a group of participants to whom the treatment isn’t applied, in the test group the variable that the researcher wants to test is introduced and in the placebo group, the participants think they are part of the test group, but they are receiving some sort of alternative to the treatment that will not yield the expected result.  The human brain and body are pretty powerful…when we think we’re getting we can actually experience improvements that don’t really exist!

      However, a placebo group is not always feasible depending on what is being tested so a bit of common sense needs to be applied here. For instance, if you were trying to test some sort of effect related to drinking water, you could have a control group who didn’t drink water, but given everyone knows what water tastes like, attempting to create a “water placebo” would be pretty tough. But a control and test group should be the bare minimum!  And in cases of medication where a placebo can be easily applied, there should ALWAYS be a placebo group.

    • Non randomised
      If the research is experimental in nature (and not survey based), and the report doesn’t say it’s randomised, then it probably isn’t.  A randomised experiment means that participants in the experiment (those put into either control, test or placebo groups) were randomly assigned assigned to them.  i.e. that the researchers weren’t in control of choosing who would be assigned to which group.  If they are, they can introduce all sorts of unconscious bias that could affect the results.
    • Non Blinded or Non Double Blinded
      A blinded study is where the participant in the research is unaware which group (i.e. test, control or placebo) they have been assigned to.  A double blind study is where neither the participant in the research, nor the researcher themselves, knows which group the participants have been assigned to.  That means the researcher might only see a number in place of an individual’s name and details when seeing the results. And the experiment may be designed in such a way that those responsible for data collection, and perhaps physically collecting data from the participants, do not communicate with the researcher (or may not even be known to them).
    • Small sample sizes
      A “sample” is basically a little portion of the broader “population”.  A population in research doesn’t have to mean the population of a country, it may just be the population within a particular category pertinent to the research.  For example “people with Diabetes”, or “people who have been treated for depression”, or “women who have given birth to at least one child”.  The sample size has, because there are random effects that can occur in small samples that even themselves out when you test the same thing on a larger sample size.

      Most good research might start with a smaller sample size to test an initial hypothesis (theory).  They’ll release initial results but caution that due to small sample sizes, more research should be done to see if the results can be replicated on a broader scale so that it can be .  If this is the first research in a particular area to come out and it’s got a small sample size you MUST treat it with caution. It means that it is essentially “baby research” it’s not fully formed yet nor capable of making truly informed conclusions about its own existence!

 

  • Non peer reviewed – i.e published in a dodgy journal 
    Yep, not all journals are created equal.  A good piece of research should be published by a journal that has a process whereby other scientific peers review the research methodology before accepting it for publishing.  Sometimes good journals will create a single-blind process for doing this – meaning that those reviewing the research don’t know who the author is.  That’s important – because humans have an innate bias to trust people who are perceived to be more credible, despite there potentially being a lack of evidence to support that trust.

    If it has been archived or cited here: https://www.ncbi.nlm.nih.gov that’s a good start.  Apparently this tool https://harzing.com/resources/publish-or-perish helps you figure out how many times the article has been cited in journals (although I’ve not used it before) and this one helps you figure out the ranking of the journal: http://www.scimagojr.com/journalrank.php?area=2700&type=j.

    Monash Uni have a bunch of good links and info about assessing journal quality here including:

  • Statistical significance without IMPORTANCE
    Statistical significance is generally agreed that there is either a 5% or lower (sometimes 1% for more rigorous research) probability that the results obtained were due to chance versus the variable being tested.

    That’s a great first step for sure, but significance does not mean importance. Once the study has met the above criteria, ask yourself one, final and very important question “Is this question the right question to be asking, and is the assumption that underpins the question being asked a correct assumption?”

 

In the next article I’ll use this cnn article to test drive some of the knowledge above.