multiple question Three multiple choice questions An Ideological Asymmetry in the Diffusion of Moralized Content on Social Media Among Political Leaders W

Three multiple choice questions

An Ideological Asymmetry in the Diffusion of Moralized Content on
Social Media Among Political Leaders

William J. Brady and Julian A. Wills
New York University

Dominic Burkart
Princeton University

John T. Jost and Jay J. Van Bavel
New York University

Online social networks constitute a major platform for the exchange of moral and political ideas, and
political elites increasingly rely on social media platforms to communicate directly with the public.
However, little is known about the processes that render some political elites more influential than others
when it comes to online communication. Here, we gauge influence of political elites on social media by
examining how message factors (characteristics of the communication) interact with source factors
(characteristics of elites) to impact the diffusion of elites’ messages through Twitter. We analyzed
messages (N � 286,255) sent from federal politicians (presidential candidates, members of the Senate
and House of Representatives) in the year leading up to the 2016 U.S. presidential election—a period in
which Democrats and Republicans sought to maximize their influence over potential voters. Across all
types of elites, we found a “moral contagion” effect: elites’ use of moral-emotional language was robustly
associated with increases in message diffusion. We also discovered an ideological asymmetry: conser-
vative elites gained greater diffusion when using moral-emotional language compared to liberal elites,
even when accounting for extremity of ideology and other source cues. Specific moral emotion
expressions related to moral outrage—namely, moral anger and disgust—were impactful for elites across
the political spectrum, whereas moral emotion expression related to religion and patriotism were more
impactful for conservative elites. These findings help inform the scientific understanding of political
propaganda in the digital age, and the antecedents of political polarization in American politics.

Keywords: morality, emotion, politics, social networks, social media

Supplemental materials: http://dx.doi.org/10.1037/xge0000532.supp

For over 2 billion users of Twitter and Facebook, online social
networks constitute a major platform for the exchange of moral
and political ideas. Twitter now plays a major role in a wide range
of political events, from elections to revolutions, and this influence
appears to be growing. Political elites, such as President Donald
Trump, increasingly rely upon social media platforms to commu-
nicate directly with the public. Although Hillary Clinton’s cam-
paign spent $500 million more than Trump’s campaign on adver-

tising during the 2016 race (Allison, Rojanasakul, Harris, & Sam,
2016), Trump credited social media with allowing him to over-
come this disparity (Stahl, 2016). Unfortunately, social media also
provides certain political actors with the capacities to engage in
“cyberwarfare” and to “sow conflict and discontent” in society
(Timberg, Shaban, & Dwoskin, 2017). This paper examines the
role of moral-emotional expression and political ideology in the
communications of political elites on social media.

This article was published Online First December 27, 2018.
William J. Brady and Julian A. Wills, Department of Psychology, New

York University; Dominic Burkart, Department of Psychology, Princeton
University; John T. Jost, Department of Psychology, Department of Poli-
tics, and Center for Data Science, New York University; Jay J. Van Bavel,
Department of Psychology and Center for Neural Science, New York
University.

William J. Brady is now at the Department of Psychology, Yale University.
This research was presented by William J. Brady at the Princeton Neuro-

science and Social Decision Making meeting, the Boston Area Moral Cogni-
tion Group (BAM) meeting, and the 2018 Society for Personality and Social
Psychology (SPSP) Annual Convention. William J. Brady, Julian A. Wills,
John T. Jost and Jay J. Van Bavel designed research; William J. Brady, Julian
A. Wills, and Dominic Burkart performed research; William J. Brady, Julian

A. Wills, John T. Jost, and Jay J. Van Bavel planned analyses; William J.
Brady analyzed data; William J. Brady wrote the paper and all authors
contributed to revisions. This research was supported by the National Science
Foundation (Awards SES-1349089, SES-1248077, and SES-1248077-001) as
well as the Global Institute for Advanced Study (GIAS) and Research Invest-
ment Fund (RIF) at New York University. We thank Jino Kwon, Miaohan
Wang, and Stephanie Leung for assistance with variable coding. We are also
grateful to members of the NYU Social Perception and Evaluation Lab
(@vanbavellab), the Brown Social and Affective Neuroscience Lab, and the
Yale Crockett Lab for their comments and suggestions.

Correspondence concerning this article should be addressed to Jay J.
Van Bavel, Department of Psychology, New York University, 6 Washing-
ton Place, Room 455, New York, NY 10003. E-mail: jay.vanbavel@
nyu.edu

T
hi

s
do

cu
m

en
t

is
co

py
ri

gh
te

d
by

th
e

A
m

er
ic

an
P

sy
ch

ol
og

ic
al

A
ss

oc
ia

ti
on

or
on

e
of

it
s

al
li

ed
pu

bl
is

he
rs

.
T

hi
s

ar
ti

cl
e

is
in

te
nd

ed
so

le
ly

fo
r

th
e

pe
rs

on
al

us
e

of
th

e
in

di
vi

du
al

us
er

an
d

is
no

t
to

be
di

ss
em

in
at

ed
br

oa
dl

y.

Journal of Experimental Psychology: General
© 2018 American Psychological Association 2019, Vol. 148, No. 10, 1802–1813
0096-3445/19/$12.00 http://dx.doi.org/10.1037/xge0000532

1802

There is reason to believe that elected and nonelected political
elites can use social media to shape the moral and political atti-
tudes of the public. Many political elites cultivate very large
numbers of followers on social media, are especially influential in
their groups (Hogg, 2010), have a broad to spread ideas (Rogers,
2010), and are relatively extreme in terms of ideology and partisan
identification (McCarty, Poole, & Rosenthal, 2016). Thus, behav-
ioral research is needed to illuminate the processes that render
some political elites more influential than others when it comes to
online communication. In fact, studying these individuals may
provide the most powerful insights about why political information
spreads online and the consequences it might have on political
behavior. Yet, little is known about the efficacy of various types of
appeals to massive audiences on social media (see Jost, Barberá et
al., 2018).

Here, we gauge the influence of political elites in online social
networks by examining how source factors (characteristics of
elites) interact with message factors (characteristics of the com-
munication) to impact the diffusion of elites’ messages though
online social networks (see McGuire, 1985; Petty & Cacioppo,
1986). Information diffusion, which refers to the spread of infor-
mation through direct and indirect ties that occurs through social
sharing, is a major indicator of online social influence (e.g.,
Bakshy, Rosenn, Marlow, & Adamic, 2012; Barberá, Jost, Nagler,
Tucker, & Bonneau, 2015). Compared to traditional advertising
strategies, social media networks provide cost-effective means of
reaching large numbers of people. Users often share messages on
social media that represent beliefs, opinions, and values they
endorse as well as authors they trust (Metaxas et al., 2015). Thus,
the frequency with which a political candidate or party is men-
tioned on Twitter is correlated with offline election outcomes
(O’Connor, Balasubramanyan, Routledge, & Smith, 2010; Tumas-
jan, Sprenger, Sandner, & Welpe, 2011), and discussions of polit-
ical protests predict subsequent offline behavior (Mooijman,
Hoover, Lin, Ji, & Dehghani, 2018). For all of these reasons,
information diffusion through social sharing reflects the potential
for political power: that is, the extent to which elite opinions are
actually reaching large audiences and broad constituencies.

A number of message factors contribute to the diffusion of
moral and political messages in online social networks. When it
comes to news articles, emotional content (Berger & Milkman,
2012; Stieglitz & Dang-Xuan, 2012) and moralistic language (Va-
lenzuela, Piña, & Ramírez, 2017) both predict increased rates of
information diffusion. Political messages that contain both moral
and emotional content are especially contagious—an effect we
have termed moral contagion (Brady, Wills, Jost, Tucker, & Van
Bavel, 2017). In the context of online social networks, moral
contagion refers specifically to the diffusion of moralized content
resulting from a process whereby moral and emotional expressions
serve as information that influences people’s evaluations and can
shape their behavior (e.g., decisions to share content). For instance,
social movements that are promoted in terms of moral and emo-
tional content are more likely to be shared virally, presumably
because this type of promotion makes people more likely to treat
support for the movement as a moral imperative (Van Der Linden,
2017). Although these findings suggest that including moral and
emotional expression in communications may help political elites
to reach very large audiences, this idea has yet to be tested in a
sample of political elites using social media platforms.

An important theoretical assumption of social psychology is that
characteristics of the communication source (i.e., who is sending
the message) often interact with the framing or content of the
message (Chaiken, 1980; McGarty, Haslam, Hutchinson, &
Turner, 1994; McGuire, 1985; Petty & Cacioppo, 1986). One
source cue that is expected to interact with message content to
affect diffusion is political ideology (Jost, van der Linden, Pan-
agopoulos, & Hardin, 2018). For instance, we found that the
diffusion of moral-emotional language was greater within politi-
cally conservative (vs. liberal) online networks for the contentious
political topic of climate change (Brady et al., 2017). This is
consistent with research finding that conservatives are more sen-
sitive than liberals to high-arousal emotions such as anger, con-
tempt, anxiety, and threat, and more moralistic when it comes to
social issues (Hibbing, Smith, & Alford, 2014; Jost, 2017; Jost,
Glaser, Kruglanski, & Sulloway, 2003; Tomkins, 1995). In addi-
tion, there may be gender asymmetries in the effectiveness of
certain types of message content. It is possible that citizens are
more influenced by moral-emotional language when it is wielded
by male rather than female politicians insofar as females are often
evaluated negatively when they express high-arousal emotions
(Brescoll & Uhlmann, 2008; Lewis, 2000)— even when the emo-
tions are gender-normative (Hutson-Comeaux & Kelly, 2002;
Thomas, 2016).

To explore potential interactions between source and message
factors, we considered the roles of political ideology and gender,
as well as the specific contents in social media messages sent by
elected officials. Specifically, we analyzed a large sample of
Twitter messages (Total N � 286,255) sent by U.S. politicians in
the year leading up to the 2016 presidential election—a period in
which Democrats and Republicans sought to maximize their in-
fluence over potential voters.

Our sample of elite social media users included the two major
presidential candidates as well as every member of the U.S. Senate
and House of Representatives with a Twitter account during this
period. This sample enabled us to investigate the extent to which
processes of moral contagion were moderated by message source.
In other words, we asked whether some politicians—such as
conservatives or male politicians— benefit more than others (lib-
erals and female politicians) when it comes to the use of moral-
emotional language in their online communications? These are
important questions of a theoretical and practical nature about the
elite usage of social media in democratic society. Not only would
the answers to these questions illuminate the phenomenon of
propaganda in the digital age, but it might also help to better
understand some of the elements of affective polarization in Amer-
ican politics (McCarty et al., 2016).

Current Research

We describe the results of three large-scale studies of elite
communication that directly investigated these questions about
moral contagion among political elites and potential moderators of
the effect, which has thus far been explored only in the context of
social media usage by ordinary citizens (Brady et al., 2017). We
analyzed Twitter messages sent by U.S. politicians in the year
leading up to the 2016 presidential election (see Method). Specif-
ically, we collected tweets sent from the official accounts of
Donald Trump and Hillary Clinton (Study 1) as well as all mem-

T
hi

s
do

cu
m

en
t

is
co

py
ri

gh
te

d
by

th
e

A
m

er
ic

an
P

sy
ch

ol
og

ic
al

A
ss

oc
ia

ti
on

or
on

e
of

it
s

al
li

ed
pu

bl
is

he
rs

.
T

hi
s

ar
ti

cl
e

is
in

te
nd

ed
so

le
ly

fo
r

th
e

pe
rs

on
al

us
e

of
th

e
in

di
vi

du
al

us
er

an
d

is
no

t
to

be
di

ss
em

in
at

ed
br

oa
dl

y.

1803IDEOLOGY ASYMMETRY IN MORAL CONTAGION

bers of the Senate (Study 2) and all of U.S. Congress, including the
Senate and House of Representatives (Study 3). To gauge the
extent to which social media messages contained moral and emo-
tional language, we analyzed textual variation in the contents of
messages. We used a text-mining technique that searched for
keywords in messages based on previously validated dictionaries
for measuring morality, emotion, and moral emotion (Brady et al.,
2017; see Method below). Diffusion was indexed as the number of
retweets each message received, because retweet counts provide a
high-quality measure of information diffusion on Twitter (Barberá
et al., 2015; Stieglitz & Dang-Xuan, 2012).

Study 1

In Study 1 (N � 9,505) we investigated the use of moral and
emotional language by Donald Trump and Hillary Clinton and its
dissemination to determine whether (a) presidential candidates
exhibit a moral contagion effect, and (b) variation in source cues
such as ideology and gender moderate the effect. We also explored
whether positive versus negative moral emotion (valence analysis),
or specific expressions of moral emotions (e.g., moral anger vs.
moral disgust) had differential effects on diffusion for Trump and
Clinton.

Method

Data collection. All research was conducted in accordance
with the New York University (NYU) University Committee on
Activities Involve Human Subjects (IRB no. 12–9058). Data col-
lection was ruled “exempt” due to our use of public tweets only. A
public tweet is a message that the user consents to be publicly
available rather than only to a collection of approved followers. All
data were collected by connecting to Twitter’s API with the
userTimeline function via Python’s TweePy package. Data were
collected in October, 2016 with the goal of collecting every tweet
from Clinton and Trump dating back to November 8th, 2015 to
represent 12 months leading up to the 2016 U.S. presidential
election. Due to limitations from the Twitter API, which allows
collection of �3,000 of a user’s most recent messages, our target
time range of 12 months prior to the 2016 election was only met
by using the API in combination with web page-scraping since
Trump and Clinton tweeted more 3,000 times in one year. All
metadata for each tweet including retweet count, presence of
media or URL, and follower number for each account were pulled
directly from the API at the time of data collection. Thus, the
retweet counts and follower number attached to each elite account
were as of October 2016. We removed messages that elites had
retweeted, and thus our final data sets consisted only of original
messages composed by the elites’ accounts. All data and analysis
scripts are available at https://osf.io/reqx9/.

Measuring moral and emotional language. To measure mo-
rality and emotion, we searched tweets for the presence of key-
words based on previously validated dictionaries for measuring
morality and emotion, and formed three categories of distinctly
moral, distinctly emotional, and moral-emotional words. Moral
words were defined as those that appeared only in the moral
dictionary (e.g., justice, holy, pure); emotional words were those
that appeared only in the emotion dictionary (e.g., sad, enjoy,
annoyed); and moral-emotional words were those that appeared in

both (e.g., hate, murder, shame). These theoretically derived cat-
egories were based on previously validated dictionaries shown to
have high discriminant validity in multiple pilot studies (for in-
stance, participants rated moral-emotional words as more moral
than distinctly emotional words and more emotional than distinctly
moral words with a mean effect size of d � 2.23; see Brady et al.,
2017). Using this method, each tweet was assigned a count repre-
senting how many times each category of words appeared in the
tweet.

Measuring positive versus negative moral emotion. In or-
der to measure positive and negative valence, we assessed each
distinctly emotional and moral-emotional word’s valence assign-
ment based on the previously validated LIWC dictionary (Tausc-
zik & Pennebaker, 2010).

Measuring specific expressions of moral emotion. In order
to measure specific emotion expressions associated with moral-
emotional words, we used the R tidytext package (Silge & Rob-
inson, 2016), which allowed us to tokenize each tweet and then
classify each token as representing one or many discrete emotions
(overlap in classification was allowed; e.g., abused is labeled as
both disgust-related and anger-related) based on the NRC senti-
ment analysis lexicon (Mohammad, Kiritchenko, & Zhu, 2013).
Words that were classified by our dictionary method as moral-
emotional and also as a related to a specific emotion were counted
as an instance of a discrete moral emotion.

Measuring diffusion. Diffusion was defined as retweet count
per tweet, and retweet count was taken from meta data available
from the Twitter API.

Results

We regressed the retweet count for each message on the count
of distinctly moral, distinctly emotional, and moral-emotional
words present in each message. For all studies, we log-transformed
retweet count to form a normal distribution appropriate to perform
OLS regression. We choose this method rather than using tradi-
tional count models (e.g., negative binomial) because political
elites have an abnormally high number of retweets per tweet. Thus,
the number of 0 occurrences is extremely rare (for instance, the
minimum retweet count in President Trump’s sample was 370, and
Clinton’s was 42), and model testing revealed that log-transformed
OLS regression achieved better model fit than negative binomial
regression models. The 10 moral and moral-emotional words that
were associated with the two candidates’ most viral tweets are
shown in Figure 1.

For Trump, we observed that messages using moral language
were associated with a significant increase in retweet count,
exp(b) � 1.12, p � .001, 95% CI [1.08, 1.17]. This effect trans-
lates to a predicted 12% increase in retweet count for each dis-
tinctly moral word included in a message. Although the use of
distinctly emotional language actually associated with a slight
reduction in retweet count, exp(b) � 0.97, p � .028, 95% CI [0.95,
1.00], the use of moral-emotional language was associated with the
largest significant increase, exp(b) � 1.25, p � .001, 95% CI
[1.18, 1.32]. This effect translates to a 25% increase in retweet
count for each moral-emotional word added to the tweet. Thus, we
see that Trump clearly benefitted from the moral contagion effect
(see Tables S1–S2 in the online supplemental materials for full

T
hi

s
do

cu
m

en
t

is
co

py
ri

gh
te

d
by

th
e

A
m

er
ic

an
P

sy
ch

ol
og

ic
al

A
ss

oc
ia

ti
on

or
on

e
of

it
s

al
li

ed
pu

bl
is

he
rs

.
T

hi
s

ar
ti

cl
e

is
in

te
nd

ed
so

le
ly

fo
r

th
e

pe
rs

on
al

us
e

of
th

e
in

di
vi

du
al

us
er

an
d

is
no

t
to

be
di

ss
em

in
at

ed
br

oa
dl

y.

1804 BRADY, WILLS, BURKART, JOST, AND VAN BAVEL

Figure 1. List of moral and moral-emotional words included in Donald Trump’s and Hillary Clinton’s most
viral tweets leading up to the 2016 U.S. presidential election, in order of mean retweet count. Words that were
not used at least 10 times by the candidate are omitted. See the online article for the color version of this figure.

T
hi

s
do

cu
m

en
t

is
co

py
ri

gh
te

d
by

th
e

A
m

er
ic

an
P

sy
ch

ol
og

ic
al

A
ss

oc
ia

ti
on

or
on

e
of

it
s

al
li

ed
pu

bl
is

he
rs

.
T

hi
s

ar
ti

cl
e

is
in

te
nd

ed
so

le
ly

fo
r

th
e

pe
rs

on
al

us
e

of
th

e
in

di
vi

du
al

us
er

an
d

is
no

t
to

be
di

ss
em

in
at

ed
br

oa
dl

y.

1805IDEOLOGY ASYMMETRY IN MORAL CONTAGION

model details; all data and analysis scripts are available at https://
osf.io/reqx9/).

We also examined Hillary Clinton’s messages during the same
period and observed that her use of distinctly emotional language
was significantly associated with increased diffusion, exp(b) �
1.05, p � .001, 95% CI [1.02, 1.08]. However, neither her use of
distinctly moral language, exp(b) � 1.01, p � .625, 95% CI [0.98,
1.04], nor moral-emotional language, exp(b) � 1.02, p � .490,
95% CI [0.97, 1.07]) predicted diffusion. Unlike Trump, Clinton
failed to benefit from moral contagion (see Tables S1–S2 in the
online supplemental materials for full model details).

When we combined the Trump and Clinton corpuses and used
effects coding to signify the source of the message (using Clinton
as the reference category), we observed significant interactions
between the effect of moral language and source, exp(b) � 1.11,
p � .001, 95% CI [1.06, 1.17] and the effect of moral-emotional
language and source, exp(b) � 1.23, p � .001, 95% CI [1.14,
1.32], see Figure 2. Trump’s use of moral and moral-emotional
language resulted in significantly more diffusion in comparison
with Clinton’s, despite the fact that Clinton used more moral
language and moral-emotional language than Trump on average
(see Table 1).

To further explore the asymmetry between Trump and Clinton,
we examined the effects of word valence and discrete moral
emotions (see Method). Trump’s use of positive moral-emotional
language, exp(b) � 1.16, p � .001, 95% CI [1.03, 1.31], and
negative moral-emotional language, exp(b) � 1.24, p � .001, 95%
CI [1.08, 1.43], resulted in significantly more diffusion, in com-
parison with Clinton. Therefore, the asymmetry was not attribut-
able to a difference in terms of the valence of moral emotions
expressed (see SI Table S3 in the online supplemental materials for
model details). We did observe, however, that for Trump the moral
contagion effect was driven largely by the expression of moral

anger, whereas for Clinton discrete moral emotions were not
associated with increased retweet counts (see the online supple-
mental material, Section 1).

Study 2

Study 1 provided evidence that the source of the message affects
the dissemination of moral-emotional language used by presiden-
tial candidates. However, because Trump and Clinton differ on
countless dimensions, it is impossible to determine whether the
asymmetry was due to differences in political ideology, gender, or
some other variable. Although Trump and Clinton carry a great
deal of potential for social media influence, further evidence is
required to determine how generalizable potential source effects are to
other political elites. Study 2 was designed to investigate with more
precision the extent to which the specific source cues of political
ideology and gender contribute independently to the moral contagion
effect, and to include a much larger sample of political elites.

Table 1
Descriptive Statistics for Moral, Emotional, and Moral-
Emotional Language Used by Political Candidates and
Members of Congress

Liberal Conservative

Language
categories Clinton Congress Trump Congress

Moral .54 (.81) .50 (.76) .34 (.62) .38 (.65)
Emotional .75 (.87) .90 (.96) 1.20 (1.12) .84 (.93)
Moral-emotional .24 (.53) .26 (.54) .19 (.46) .25 (.53)

Note. Means represent the number of words from each language category
used on average, per message.

Figure 2. Donald Trump’s use of moral and moral-emotional language was significantly associated with
increased retweet rates, but this was not the case for Hillary Clinton. The graphic depicts the association between
distinctly emotional (blue [black]), distinctly moral (orange [dark gray]) and moral-emotional (green [light gray])
language with retweet counts. Error bands represent 95% confidence intervals. See the online article for the color
version of this figure.

T
hi

s
do

cu
m

en
t

is
co

py
ri

gh
te

d
by

th
e

A
m

er
ic

an
P

sy
ch

ol
og

ic
al

A
ss

oc
ia

ti
on

or
on

e
of

it
s

al
li

ed
pu

bl
is

he
rs

.
T

hi
s

ar
ti

cl
e

is
in

te
nd

ed
so

le
ly

fo
r

th
e

pe
rs

on
al

us
e

of
th

e
in

di
vi

du
al

us
er

an
d

is
no

t
to

be
di

ss
em

in
at

ed
br

oa
dl

y.

1806 BRADY, WILLS, BURKART, JOST, AND VAN BAVEL

Method

Data collection. Using the same methods as Study 1, we
collected Twitter messages sent by all 100 U.S. Senators in the
year leading up to the 2016 U.S. presidential election (N �
99,750). To statistically adjust for confounding variables that
can affect message diffusion, we pulled metadata for Senators
and their messages including follower number and the presence
of URL and/or media in each message. Metadata were pulled
directly from the API at the time of data collection. Thus, the
retweet counts and follower number attached to each elite
account were as of October, 2016. We removed messages that
elites had retweeted, and thus our final data sets consisted only
of original messages composed by the elites’ accounts.

Measuring morality and emotion, valence, and specific emo-
tion expressions. We used the same text-mining techniques as in
Study 1.

Measuring political ideology. To measure the ideological
orientation of each elite, we pulled an estimate of their ideology
based on voting recordings using the freely available DW-
NOMINATE database (Poole & Rosenthal, 1985). This database
assigns Congress members a continuous ideology value based on
their voting records where negative values indicate a liberal-
leaning voting pattern and positive values indicate a conservative-
leaning voting pattern. We also formed a measure of extremity of
ideology and it was defined as the scaled absolute value of the
DW-NOMINATE score (see Results below). We scaled the ex-
tremity variable so that the range of ideology was the same for
both conservatives and liberals (see online supplemental material,
Section 2).

Measuring gender. Gender was measured as an effects-coded
binary (male/female) variable based on the sex of the senator,
where males were defined as the reference group.

Results

We analyzed messages nested within elites in a multilevel
model and regressed retweet count on the use of distinctly moral,
distinctly emotional, and moral-emotional language, as well as
covariates known to affect retweet rate (URL, media, follower
number). Our multilevel model accounted for nonindependence of
data using Generalized Estimating Equations (GEE) with robust
standard error estimation (Hardin, 2005), and an exchangeable
correlation structure.

The analysis revealed that distinctly moral language, exp(b) �
1.13, p � .001, 95% CI [1.09, 1.18] and moral-emotional lan-
guage, exp(b) � 1.13, p � .001,

Looking for this or a Similar Assignment? Click below to Place your Order