Author: Brian Kurilla

Facebook’s mood-manipulation experiment shows evidence of a small effect with potentially large consequences

If you pay attention to news about science or social media, then chances are you’ve already heard about how back in 2012 Facebook carried out a giant psychological experiment on hundreds of thousands of its users without informing them.

The goal of the experiment was to learn more about how a person’s mood is affected by the types of posts they see in their News Feed. And so for one week, Facebook subtly manipulated what appeared in users’ News Feeds and then analyzed the emotional tone of those users’ subsequent status updates.

Needless to say, public furor ensued.

Since the findings of the study were published in the June 2014 issue of Proceedings of the National Academy of Sciences (PNAS), reactions to the now famous “Facebook mood-manipulation experiment” have ranged from suspicion about Facebook’s true motives for conducting the research to anger over whether the experiment violated ethical guidelines for conducting research on human participants.

Given the attention this story has received in the news and on social media, I decided to take a close look at the actual PNAS paper.

So in this post, I’ll describe the methods and findings of the Facebook experiment and offer a critical assessment of what I see as being the major implications of this study.

Let’s begin with a brief rundown on what actually happened back in 2012.

What actually happened in Facebook’s “Mood-Manipulation Experiment?”

From January 11-18, 2012, members of Facebook’s Core Data Team collaborated with researchers from the University of California, San Francisco and Cornell University to conduct an experiment on a psychological phenomenon called emotional contagion.

Emotional contagion is the idea that some emotions – such as happiness, sadness, and anger – behave like infection diseases, spreading from person to person as though they are actually contagious.

What the researchers involved in the Facebook experiment wanted to know was whether emotions can spread from person to person even in the absence of direct human contact. If so, they reasoned that seeing positive emotional posts in your Facebook News Feed (e.g., seeing a friend post about being excited to start a new job) should lead you to experience a more positive mood, whereas seeing negative emotional posts in your News Feed (e.g., seeing a friend post about being annoyed that they have to work on Saturday) should lead you to experience a more negative mood.

So the researchers randomly selected 689,0003 Facebook users based on their user ID and for one week covertly manipulated the number of positive and negative posts that appeared in their News Feeds.

Positive posts were defined as those that contained at least one positive emotion word (e.g., love, happy, etc.), and negative posts were defined as those that contained at least one negative emotion word (e.g., hate, angry, etc.).

During the experiment, whenever users logged on to Facebook they were randomly assigned to either an “experimental condition” or a “control condition.” During the experimental condition, either positive or negative posts were omitted from a user’s News Feed. In the control condition, a similar percentage of posts were omitted from a user’s News Feed entirely at random (i.e., not based on emotional content).

How did they assess people’s mood?

The researchers assessed the mood of each user in the study by looking at the number of positive and negative words contained in their status updates following the mood manipulation procedure.

Status updates containing a greater number of positive emotion words were assumed to reflect a positive mood, whereas status updates containing a greater number of negative emotion words were assumed to reflect a negative mood.

To maintain compliance with Facebook’s Data Use Policy, status updates were collected and coded via an automated data collection and filtering system so that no text was actually ever seen by any of the researchers.

What did the experiment find?

The researchers found that when positive posts were omitted from News Feeds, Facebook users went on to post status updates that contained fewer positive words and more negative words, suggesting the experience of a more negative mood. The opposite occurred when negative posts were omitted from News Feeds – Facebook users went on to post status updates that contained fewer negative words and more positive words, suggesting the experience of a more positive mood.

Figure 1 from the original paper is presented below, showing the main results of the study.

Facebook exp fig 1

Figure 1 from Kramer, Guillory, & Hancock (2014).

So the findings of this experiment suggest that we unwittingly adopt the same emotional state we see others convey online.

When we see friends share posts expressing positive emotion on Facebook, we tend to follow along and share positive status updates of our own. Unfortunately, the reverse is also true. Seeing friends share posts expressing negative emotion on Facebook leads us to join in and share status updates conveying negative emotion as well.

Now that I’ve described the methods and major findings of Facebook’s mood-manipulation experiment, let’s move on to a critical assessment of what these findings actually mean and what their implications might be.

Are the findings of the Facebook experiment important, and should we worry about Facebook controlling our moods?

As I see it, there are two ways of looking at the results of the Facebook study to gauge how important and relevant they are to the real-world.

The first is to consider the size of the effect of Facebook’s News Feed manipulation on people’s moods, and the second is to think about the number of people whose behavior and mood might be affected by seeing fewer positive posts (or fewer negative posts) in their News Feed.

The latter approach is especially important considering the enormous size of Facebook, currently estimated at 1.28 billion users worldwide.

1. How Large is the Effect of Facebook’s Newsfeed Manipulation on People’s Moods?

As it turns out, even though reducing the number of positive and negative posts in users’ News Feeds did indeed have a statistically significant effect on mood (again, as measured by the number of positive and negative words people used in subsequent status updates of their own), the size of the effect was extremely small.

How small exactly?

I’ll demonstrate in two different ways:

A) First, consider that psychological scientists generally classify effect sizes in their research (symbolized here by the letter d) as follows:*

Small Effects: d = 0.2 or lower

Medium Effects: d = around 0.5

Large Effects: d = 0.8 or higher

Effect sizes reported in the Facebook study meanwhile ranged from a puny 0.001 to a slightly less puny 0.02! In fact when positive posts were omitted from News Feeds, the number of positive status updates from users decreased by less than half of 1% (0.10% specifically).

So removing positive and negative posts from a person’s News Feed doesn’t just have a small effect on mood. It has a miniscule effect.

*This is the conventional classification of a specific measure of effect size called Cohen’s d, which is the measure of effect size that was used in the Facebook study. Note that an effect size of d = 0 would mean an experimental manipulation had absolutely no effect on behavior.

Need more convincing? Consider the following hypothetical scenario:

B) Assuming your News Feeds was manipulated so as to omit positive posts, what’s the probability that you would show signs of a worse mood than someone whose News Feed had not been manipulated?

Given the largest effect size reported in the paper (d = 0.02), the answer is 50.56% – essentially a coin toss! *

*Known as the common language effect size (CL), this estimate was derived from Cohen’s d (the measure of effect size used in the Facebook study) using the following formula: CL = Φ (δ/√2), where Φ is the cumulative distribution function of the standard normal distribution, and δ is the value for Cohen's d.

So although manipulating the number of positive and negative posts that appear in a person’s News Feed certainly does affect mood (or at least how a person chooses to express their mood online), the size of the effect is tiny, and probably too small to be of much concern to a single individual.

So there’s no reason to lose sleep over the fear that Facebook might be controlling your mood by manipulating the contents of your News Feed!

In fact, if you were one of the 689,003 people in the Facebook experiment, there would only have been a 0.8% chance that you would have shown signs of a negative mood after having positive posts omitted from your News Feed.*

*This number was calculated as follows: [Estimated number of people affected by the News Feed manipulation] / 689,003. See below for details on calculating Estimated number of people affected by the News Feed manipulation.

Importantly, the authors of the Facebook study fully acknowledge that omitting positive and negative posts from users’ News Feeds has only a tiny effect on mood.

Nonetheless, they conclude their paper by saying the following:

“Although these data provide, to our knowledge, some of the first experimental evidence to support the controversial claims that emotions can spread throughout a network, the effect sizes from the manipulations are small (as small as d = 0.001). These effects nonetheless matter given that the manipulation of the independent variable (presence of emotion in the News Feed) was minimal whereas the dependent variable (people’s emotional expressions) is difficult to influence given the range of daily experiences that influence mood (10). More importantly, given the massive scale of social networks such as Facebook, even small effects can have large aggregated consequences (14, 15): For example, the well-documented connection between emotions and physical well-being suggests the importance of these findings for public health. Online messages influence our experience of emotions, which may affect a variety of offline behaviors. And after all, an effect size of d = 0.001 at Facebook’s scale is not negligible: In early 2013, this would have corresponded to hundreds of thousands of emotion expressions in status updates per day.”

 *bolded text added by me

The portions of this text that I’ve highlighted raise a very interesting question, and point to the need to evaluate the importance of this research a second way, besides simply considering effect size as we’ve already done.

We need to think about the number of people whose behavior and mood would be affected by seeing fewer positive and negative posts in their News Feed, something that is especially important given the enormous size of Facebook.

2. Given the present size and scale of Facebook, how many people might be affected by seeing fewer positive posts (or fewer negative posts) in their News Feed?

Even though the contents of your News Feed has just a tiny effect on how you choose to express your mood online (as already discussed above), decreasing the number of positive and negative posts that show up in the News Feeds of a large enough group of users could affect the behavior of a tremendous number of people, given the enormous size of Facebook.

But how many people exactly?

Assuming the number of positive and negative posts in a person’s News Feed has just a small effect on how a person chooses to express their mood online (Cohen’s d = 0.02, as reported in the study) and that the baseline probability of posting a positive status update on Facebook is 46.8% (based on the finding reported in the Facebook study that 46.8% of posts contained positive words before the start of the experiment), I estimate that if positive posts were omitted from the News Feeds of a quarter of all Facebook users (25% of 1.28 Billion = 320,000,000), this would lead to a subsequent decrease in the number of positive status updates from 2,546,887 people worldwide.*

*This number was calculated as [Number of FB users with positive posts reduced in their News Feeds (320,000,000)] / NNT. NNT refers to the number of patients one would need to treat in an experiment to produce one more favorable outcome in the treatment group compared to the control group. It was calculated using the following formula devised by Furukawa and Leucht (2011)NNT = 1 / [Φ (δ – Ψ (CER)) – CER]; where Φ is the cumulative distribution function of the standard normal distribution and Ψ is its inverse, CER is the control group's event rate (in this case, the baseline probability of posting a positive status update to Facebook or 0.468) and δ is Cohen's d (in this case, 0.02). NNT was determined to be 125.64.

And even though that’s only a tiny fraction of the group of people whose News Feeds would have been manipulated (0.8% of the original group of 320 million), let’s make sure we keep this number in perspective.

We’re talking about 3.29x the population of Charlotte, NC and 26% of the entire population of North Carolina.

That’s a lot of people posting fewer positive status updates on Facebook.

And although it might not spell disaster or the end of the world, one has to wonder about the real-world, offline consequences if 2.5 million people suddenly started sharing with friends and family on Facebook a slightly more jaded and pessimistic outlook on life.

Conclusion

So are the findings from the Facebook experiment important and relevant to the real-world?

I would argue the answer is yes.

People are more closely connected now than ever before, thanks to the internet, social media, and Facebook in particular. As such, the average individual’s sphere of influence is almost certainly larger today than at any other time in human history. With email, Twitter, and Facebook, we now have the capability to influence the opinions, feelings, and behaviors of people even thousands of miles away from us, and to do so almost instantly in real time.

So even though reducing the number of positive and negative posts in a person’s News Feed has only a tiny effect on an individual’s mood, tiny effects on the behavior of individuals can potentially add up to effects of great consequence when implemented in a large and highly integrated social network, especially one that comprises close to 14% of the world’s population.

———————————————————————————————————————————-

References:

Furukawa, T. A., & Leucht, S. (2011). How to obtain NNT from Cohen’s d: comparison of two methods. PloS one, 6(4).

Kramer, A.D., Guillory, J.E., & Hancock, J.T. (2014). Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences, 111 (24), 8788-8790.

For a guide to different measures of effect size related to Cohen’s d, see this excellent interactive visualization by Kristoffer Magnusson.

Just a few days of meditation might be enough to boost attention and memory

Do you practice meditation to relax, alleviate depression, or restore feelings of inner peace? If so, you’ll be happy to hear about research out of UNC, Charlotte suggesting that brief meditative practice not only improves mood and alleviates anxiety, but also boosts certain aspects of cognition, such as attention and memory.

This finding comes from a study conducted in 2010 by researchers Fadel Zeidan of the Department of Neurobiology and Anatomy at Wake Forest University School of Medicine and Susan K. Johnson of the Department of Psychology at the University of North Carolina, Charlotte. Their paper is published in the journal Consciousness and Cognition.

To investigate how meditation affects cognition, the team of researchers recruited 49 students from UNC, Charlotte, none of which had any prior experience meditating, and divided them into two groups – an experimental meditation group and a no meditation control group.

Students in the experimental meditation group received meditation training based on basic Shamatha skills for 20 minutes a day for a total of 4 days. Meanwhile, students in the no meditation control group listened to an audio recording of J.R.R. Tolkien’s “The Hobbit” over the same period of time (20 minutes a day for 4 days) and did not engage in any meditation.

In addition to administering several questionnaires to assess the participants’ mood, anxiety, and feelings of mindfulness, the researchers administered a series of cognitive tests designed to measure attention and memory (see below for details). These cognitive tests were administered at two different times during the study – before training began on Day 1 and after training concluded on Day 4.

The question was simple. Would a mere 4 days of meditation training be enough to produce measurable improvements in attention and memory? Their findings suggest the answer is yes. (more…)

Tar Trek: A new series of posts about science in the “Tar Heel” State

Introducing Tar Trek, a new series of regular posts here on Charlottology dedicated to showcasing and promoting discussion of the latest scientific research going on across the “Tar Heel” State.

Below is a quick rundown on what you can expect to find in this new series…

tar trek logo_2

In each Tar Trek post, I’ll provide a brief overview of the methods and major findings of a scientific study that was carried out at a college or university in North Carolina. I’ll also offer a critique of each study, based on what I see as being the major strengths and weaknesses of the research and based on my assessment of how I think the research could be of practical, real-world importance.

I’ll kick things off shortly in the next few days with a review of a paper published in the field of Cognitive Psychology, given that this is my field of expertise. However, in future posts I hope to delve into other fields of science as my time and understanding of the material permits.

In full disclosure, I should make clear that until now I’ve rarely taken the time to read many scholarly articles that weren’t at least tangentially related to my field. As such, my summaries and critiques of studies from branches of science very far outside my field (e.g., Quantum Physics, Molecular Biology, Chemistry, etc.) might at times be somewhat superficial, at least as I’m starting out here.

Nonetheless, I’ll do my best to communicate all scientific findings as accurately as possible without making too many bone-headed mistakes. That said, I’m bound to get some things wrong when writing about areas of research with which I’m unfamiliar, so feel free to chime in with corrections if you know more about a topic than I do. After all, sharing and discussing the exciting research we have going on in our state is the ultimate goal here.

Stay tuned for more…

B.K.

Finding the Best Job in Charlotte

Now that college graduation season is behind us, it’s time for new graduates to head out into the workforce and start looking for jobs.

And to provide some help to those looking to stay in Charlotte or relocate to the area, I’ve compiled some data on the labor market in the general region comprising Charlotte, NC, Gastonia, NC, and Rock Hill, SC.

Specifically, clicking on the image below will bring up three interactive graphics about the labor market in the Charlotte region:

Graphic A) A list of the top 40 employers in Charlotte, rank-ordered according to the number of people employed at each company.1*

Graphic B) Average annual salaries for specific occupations and major employment sectors in the Charlotte-Gastonia-Rock Hill region.2

Graphic C) A scatter plot comparing median annual salaries for various occupations in Charlotte to opportunities for employment, as indicated by number of people currently employed in each occupation.2

*Side note: Remember that Huffington Post article from back in March 2014 about how Charlotte is one of the weirdest cities in the country? Well, contrary to the author’s claims, the top 3 employers in the area are NOT Bank of America, Wells Fargo, and Red Ventures (though RV continues to grow). On the contrary, the three largest employers in Charlotte are Carolinas Healthcare System (32,500 employees), Wells Fargo (20,600 employees), and the Charlotte-Mecklenburg School System (18,143 employees). Click here, here, and here to read our other posts dedicated to fact-checking this Huffington Post article.

Click on the image below to view interactive graphics of the labor market in the Charlotte-Gastonia-Rock Hill region:

CLT_Labor_Stats

Instructions on How to Use the Interactive Graphics:

Note that Graphics A, B, and C can be sorted and filtered in a number of different ways to provide different perspectives on the labor market in the Charlotte area. Below are some instructions to help you get the most out of what each graphic has to offer.

1. Graphic A can be sorted alphabetically by company name or by number of employees (highest to lowest by default). You can re-sort the data by hovering your cursor over either column header (i.e., “Company” or “# Employees”) and clicking on the icon that appears.

Similarly, Graphic B can be sorted alphabetically by “Title” or by numeric value for each column of data, such as Median Salary, Mean Salary, Total Employed, 10th Percentile Wage, 25th Percentile Wage, 75th Percentile Wage, 90th Percentile Wage, or Location Quotient.

A note about this last column of data – sorting by Labor Quotient can be helpful for identifying a type of work that is relatively unique to the Charlotte area. As defined by the Bureau of Labor Statistics:

“The location quotient represents the ratio of an occupation’s share of employment in a given area to that occupation’s share of employment in the U.S. as a whole. For example, an occupation that makes up 10 percent of employment in a specific metropolitan area compared with 2 percent of U.S. employment would have a location quotient of 5 for the area in question.”

2. Graphics B and C can be filtered to display data for either specific occupations (e.g., Family and General Practitioners, Physics Teachers, Postsecondary, Retail Salespersons, etc.) or major employment sectors (e.g., Healthcare Practitioners and Technical Occupations, Education, Training, and Library Occupations, Sales and Related Occupations, etc.). Sorting in this way can be accomplished using the filters labeled B.1 and C.1 for graphics B and C, respectively.

3. Graphics B and C also support keyword searches. For example, if you wanted to see a list of all occupations containing the word “teacher” rank-ordered by median salary, you would simply set the filter in B.1 to “detailed” (for specific occupations) and then type “teacher” into the filter labeled B.2.

4. When viewing detailed occupations in Graphic C (i.e., when filter C.1 is set to “detailed”), the data can be filtered according to major employment sector by using the filter labeled C.3. This permits you to view the relationship between median salaries and opportunities for employment for only those occupations that fall under a chosen employment sector (e.g., Management Occupations).

5. Finally, hovering your cursor over a data point in Graphic C will bring up information related to that specific data point (occupation title, median salary, mean salary, total employed, etc.).

All occupation and sector titles displayed in Graphics B and C are the official designations used by the Bureau of Labor Statistics.

——————————————————————————————————————————

Sources for Data:

1 Charlotte Chamber of Commerce (data recent as of 2012)

2 Bureau of Labor Statistics, U.S. Department of Labor. May 2013 Metropolitan and Nonmetropolitan Area Occupational Employment and Wage Estimates.

Average Salaries for 681 positions at Charlotte-Mecklenburg Schools

Earlier this month the Charlotte Observer published their online database of salaries for employees of Charlotte-Mecklenburg Schools (CMS).

This searchable database lists the total compensation for 18,515 CMS employees between April 1, 2013 and March 31, 2014, with total compensation defined as “base salary, bonuses, state longevity pay, stipends for extra work and any other source of reportable income earned during that year.”

Although the Observer’s database is helpful for learning what a specific CMS employee earns (a.k.a., snooping) , it’s not quite as helpful for figuring out the average income across all employees with the same job title (e.g., middle school principal, 6th grade teacher, etc.), something that would probably be useful to anyone considering applying for a job at CMS or going into the field of education more generally. And let’s face it, anyone who’s planning on being a teacher in North Carolina these days would do well to be as informed as possible about the current state of affairs in our state before making any serious commitments. Indeed, compared to the rest of the country North Carolina ranks near the bottom in terms of both average as well as starting teacher salary.

So, I compiled the data from the Observer’s CMS salary database and calculated the median income for each position. Distributions are also shown for positions with more than one income value. The distributions reflect what is called the interquartile range – the distance between the 3rd quartile (income value at the 75th percentile) and the first quartile (income value at the 25th percentile).

For points of reference, I’ve also included the median income for men in Charlotte ($38,767, as denoted by the dashed blue line) and the median income for women in Charlotte ($29,218, as denoted by the dashed pink line).

You can view the median incomes and distributions for each position at CMS by clicking on the image below:

cms salaries

Note that the data can be sorted by income (highest to lowest or vice versa) or alphabetically by position. Also, hovering your cursor over a data point will bring up the following information about a position:

  • Position name
  • Count of number of cases (number of CMS employees in this position)
  • Median income
  • 25th percentile
  • 75th percentile

 

Set in Charlotte: A breakdown of all the movies, TV series, and other productions filmed in the Queen City

Summer is almost here and for many that means it’s time to head to the movies, either to catch the latest spectacular blockbuster or just to cool off in an air conditioned auditorium with a bunch of strangers.

And given that North Carolina has become an increasingly popular filming location over the last several years, there’s a pretty good chance that some of the movies you’ll enjoy this summer were at least partly filmed in the Tar Heel State.

According to the internet movie database (IMDB), there have been 2,251 productions filmed in North Carolina since the early 1960’s, with most (93%) occurring since 1990. Among these productions, approximately 673 have been feature films. Some of the most notable include The Last of the Mohicans (1992; Asheville, NC, Blue Ridge Mountains, NC, & Chimney Rock, NC), Forrest Gump (1994; Asheville, NC & Linville, NC), The Green Mile (1999; Blue Ridge Mountains, NC & Blowing Rock, NC), and more recently The Hunger Games (2012; Asheville, NC, Concord, NC, & Charlotte, NC, among others) and Iron Man 3 (Cary, NC & Wilmington, NC, among others).

But how many of the productions filmed in North Carolina were specifically filmed right here in Charlotte, and how has the entertainment production scene in our city changed in recent decades? (more…)

The myth that you can ever “achieve” happiness

Here’s a simple question for you – what leads to happiness and overall satisfaction with life?

Although what constitutes happiness is obviously highly subjective, I suspect many people adopt some version of the idea that happiness is achieved when a person accomplishes all or most of the major goals they’ve set for themselves over the course of their lifetime. Indeed, don’t we all want to be the person who someday when they’re older looks back on life with contentment and satisfaction for the things they’ve accomplished?

(more…)

When is the best time to buy tickets for flights leaving Charlotte?

SONY DSC

US Airway’s Boeing 737 taxiing to 18C for Takeoff with Charlotte’s skyline in the background. (Photo by Kyle Dunst)

Summer is fast approaching, which means that many in Charlotte are probably starting to plan vacations to distant states or countries, looking for a much needed break from work, school and/or the sometimes oppressive heat and humidity of the Southeast.

No doubt, Charlotte-Douglas International Airport will be a busy place throughout the coming months. And after reading a recent article on FiveThirtyEight.com about the potential benefits of using websites such as Kayak.com to predict whether airfare will increase or decrease in the near future, we got to wondering about the ideal time to purchase plane tickets in advance of a flight departing from Charlotte-Douglas. (more…)

Birthdays, Earthday, and Probability

I turned 32 a few days ago on April 22 (Earth Day), and like many people I sometimes wonder about who shares a birthday with me and what are the chances of meeting someone born on the same day as me.

There are several ways to answer this question, and I’ll consider two. The first is simple but involves making an estimate based on assumptions and no real data. (more…)