I'd like to introduce Chris Sumner
(Applause.) (Whistles
CHRIS SUMNER: All right.
Either way, very much appreciated.
As the gentleman there said,
a small charitable organization called
or privacy in the U.S., and part
at behavioral residue, that's the sort
in an online context to see
without actually knowing that you're
So in this experiment, we're looking
with a social bot on Twitter,
So before I begin, I was also
for DEF CON, so I know there are
and there's
to this one.
So what I wanted
of this talk, which some
So if you are familiar
Tim Hwang, familiar with astro turfing,
a gentleman called Yaz,
of the presentation may be how can
But if these terms are fairly new,
will be pretty interesting.
And the other thing is that we oh yeah,
That's what I added later.
It also contains about five minutes
and specifically machine learning,
So just so you are aware, there
Okay.
So moving on, because we're
is all about trying to win money, or
and win money, or just hack things,
to laser, you know,
who is susceptible to a social bot
It was more to improve the odds,
at least improve the odds
So that's what we were really aiming
So if you're expecting laser precision
in an organization,
But if you want to move
a bit more of an advantage,
Okay, so with that out of the way,
Then I pretty much, I like the dude,
that looked pretty good.
So moving on, starting
this gentleman is called Tim Hwang,
a competition called
And essentially how that works
with social bots competing
a small amount
props essentially.
So they were given a target audience
to go and apply their bots to, and
all of these Twitter users had
they had tweeted or had some interest
Okay, so that's a lot of Twitter users.
But the social bot
they got one for a follow back,
for social response, and
points if they were killed by Twitter, so,
So that was the scoring mechanism
It was described as Blood Spot
and Network Analysis Nerds.
So I'm not going to go into this
because there's
about five minutes,
and it's really pretty good and
So all of the references and what we're
is in the speaker notes,
about writing notes.
One team won, obviously.
And it was a team led
on Twitter and his bot got 198
is pretty good, almost 40%.
So we're going to come back
in a short while.
He was in the audience when we talked
If you're in the audience today,
and we'll get you on stage
So most research in this area
at identifying social bots or bots
or other social networks,
in looking
with who responds to social bots
with potential strangers online.
So as the goon mentioned
last year we were looking
We weren't trying
We were looking at psychopathy
to improve the odds in your favor
whether they had certain traits
whether you could reduce
and that was kind
Not saying that app's what it should be
but that's what we were looking at.
As part of that research,
to have a look at social bots as well.
That was
of the social bot competition,
We asked a subset
in this experiment
in an experiment where they'd receive
in the future, and you might think, okay,
to begin with,
all the time.
So we felt that that was quite
of roughly 700 people take the bots
out of 3,000.
So we had two main questions.
The first was,
predisposed to interacting
And the other was, is it possible
of getting a response
for any social bot, so that
in context can avoid Twitter jail,
So a good question
Now, for the younger people
you maybe not remember "Different
to find
because it was
from my childhood..
So one group that are really interested
in this is marketing and sales types.
So the reason they're interested
all of their work was based on likes
for example, or follow account,
to engagement
for marketing services have got wise
to generate fake likes and friend counts
but it's less hard or it's less easy
So it behooves them to actually try
The next group that are interested are
about that in some more detail, trying
a message or their message
Another group, and this uses
It's a fantastic tool.
You may be interested in taking
from a particular corporation
of those groups are more likely
request or your social bot request.
So, again, picking
to focusing
to respond.
Then the other group that are interested
of research are privacy researchers,
is called Erhardt Graeff at M.I.T.
and he's got
Should Do Before Social Bots Take
the Political Economy
and he's concerned one
about in his paper
to harvest a lot
by engaging
to say are gullible,
to provide certain information which
if they knew it was a fake account
So he was quite worried about that.
We'll talk about that
And the final group,
So Facebook, for example, have
and it's kind of this constant battle
and the fake account creators are
the Facebook immune system,
And I also have
a pretty good job if you look at spam,
in 2011 they've done
of throttling that down.
So these are
and obviously they've been applying
So we set
to actually go in and have a look at this.
So let me go over some of the some
in this this sphere of research.
So the talk's going
of some of the current research.
Then we'll talk
discuss the findings that we had,
up with some conclusions.
That's how it's going to go
So the only other people who we found
on Twitter provided this working
A social bot is a piece
a user account
and passes itself off as human.
So kind of well, duh, but that
working definition.
You might also hear the term Sybil.
Sybil was coined by a chap
and it looks also I mean that's kind
and it derives its name
like 16 different personalities
of the same name.
Some of you might be going, well,
because we have chap bots on IRC
and you're right as well
because this paper here was published
ones, too.
So I mentioned popularity being
We've certainly had a lot of that
accounts, Justin Bieber's got
of fake followers apparently.
But it does make you look
Who's going to trust
ten followers?
Maybe not that many.
The other thing of course is spam.
So, you know, you get DMs.
I'm wondering, you know,
a subtle hint that maybe I should lose
But, you know,
that stuff is, casa berries,
of what you saw a lot on Twitter
clicks on that stuff, right.
The other thing
that are quite amusing.
So in this one, for example,
tweets to his followers,
had a successful auction yesterday,
the universe promptly replies,
and do your thing.
So one bot that I'm particularly sad
by a gentleman called Neil Codner
So every time you mention
tweet you back and give you a ton
such that you'd want to go
just to receive the grief.
But that no longer does that, which
If Twitter could somehow work
But then there was another chap,
or Australia, who got tired
So he created a bot, the clue is kind
It's a Choring Test was the name
So anytime somebody was talking
or denying climate change,
out bits of research and
and set them straight.
So you could apply that to religion,
like to argue on your behalf
to do, right?
(Laughter.) It's a time saving device.
Thanks for making it this early hour,
Another gentleman, and this
was Project Rio Boy
he was one of the first
at social bots and creating social bots
of more like humans, really.
So well worth checking out.
I'm not going to go
because we'll be short on time.
Then politics.
So I always mispronounce Roelof's
the guys that wrote Martigo,
in 2009 called Virtual Plots Real
the concept here of what if both left
false but credible information
Well, a year later, on Twitter, we saw,
between Martha Coakley
before that vote, there was a lot of,
or dragging Martha's name
and that escalated pretty quickly so that
of an overwhelming amount
about Martha Coakley and the result,
to what happened on Twitter,
There was certainly a lot
on the use of Twitter to sort
and that got a name,
and the name for that is swiftboating.
(Laughter.) That actually went
But (Laughter.) As does everything I do.
So, but that gained its name not
military veterans where you've got this
of thing where you question
and that apparently has some effect
in certain states.
So the other thing we've been seeing
or bloggers or what have you were paid
to write positive blogs
up to his election.
It's not a brilliant example of this,
And that's got a term.
It's called astroturfing
a fake grass roots campaign
to build support for a message,
of agenda essentially.
There was a group
working on a project called Truthy,
of political and other sorts
of means and activity like that online.
That's worth having a look.
They've got some neat videos.
You can put
and look at where it originated from.
So they're actually doing some really
because they feel that what's
a real impact in the real world and
pay some attention to this space.
So really interesting.
Then we've got sockpuppets.
It was HBGary's e mail hack when was
like that kind of switched me
and it mentioned that U.S.
spy operations were
for sort
and there was some discussion
in like the Arab spring context
into a certain, you know, direction.
The interesting thing
is that this was cited
and Kenneth Geers' 2009 paper as well,
all over the world and encompassing
could be programmed
political, military or terrorist agenda.
So there's some incentive there.
Next, at Christmas this year,
in the UK that was sending
to users who were having problems.
Please follow us and DM your mobile
and password, thanks.
That was the legitimate account.
(Laughter.) That got picked
in Australia called Troy Hunt,
on Twitter or read his blogs,
he was tweeting them
from this account, my EE,
it's subtly different.
So he's looking then Troy Hunt
I'm not saying Troy Hunt's susceptible,
saying something like it was nice
an engaging approach to attacking
the MIEKER folks said, no,
But the point is that
on behalf of a brand and get
of users aren't actually going
or the non legitimate one.
Then after our Black Hat talk
up to us and started asking questions
of misdirecting emergency resources
and that made me think, oh, yeah,
at Manchester University,
I may be wrong about that.
I'll add that to the speaker notes.
But he was looking
to the London riots and what they'd
and the rumors were like there's lots
don't go into this High Street,
around Twitter at the, you know,
almost nobody
a riot and no stores were on fire.
So they were looking
to perpetuate and the truth catch up,
which made me chuckle,
a rumor gets halfway
before the truth has a chance
So there's another way you could sort
(Clapping.) I mentioned Tim Hwang.
He's
Social because part of that,
the social bots experiment was really
after that it's morphed majorly,
So he was like, oh, can you distort
maybe stitch two disparate
So that was kind of his position
of research it a lit bit more.
The other thing he was interested
contagion and happiness.
So you take a group
when their happiness levels drop
then you start injecting happy tweets
analysis and turn
So that's one
I'm not sure how far they've got to.
He also mentioned this concept
So spread information
see where they're challenged
the most influential,
and target them.
And this chap here at University
in quite some depth in relation
the social bot steal 250 gigabytes
to the paper from M.I.T.
So he was one of the first
to actually understand some
behind who's engaging
and to that end there are two groups
and Trustworthy Cyberspace Initiative
the Insider Threat Project
So we set about looking
to find people who would talk
So our methodology.
We used the winning bot code
at ArrowFate's bot and we did that
which I'll talk about.
But essentially it was the same
license.
We had 610 participants.
We got their Twitter information,
and their Klout score and linguistic
We divided them into two groups
a bot.
We did that just so that we could get
That was all.
We gave our bot an image
along the lines of I've got my own teeth,
like a young dude in New Zealand.
So we switched some sort
How the bot works
in our case dog fashion,
to WordPress and then tweet.
So you'd get a picture like,
and then you'd get a tweet
like new blog post.
So this is creating some content
Then, next we use
which actually then took let's see.
If the weather went above 20ÃƒÆ’Ã†â€™Ãƒâ€ Ã¢â‚¬â„¢ÃƒÆ’Ã¢â‚¬Â ÃƒÂ¢Ã¢â€šÂ¬Ã¢â€žÂ¢ÃƒÆ’Ã†â€™ÃƒÂ¢Ã¢â€šÂ¬Ã‚Â ÃƒÆ’Ã‚Â¢ÃƒÂ¢Ã¢â‚¬Å¡Ã‚Â¬ÃƒÂ¢Ã¢â‚¬Å¾Ã‚Â¢ÃƒÆ’Ã†â€™Ãƒâ€ Ã¢â‚¬â„¢ÃƒÆ’Ã¢â‚¬Å¡Ãƒâ€šÃ‚Â¢ÃƒÆ’Ã†â€™Ãƒâ€šÃ‚Â¢ÃƒÆ’Ã‚Â¢ÃƒÂ¢Ã¢â‚¬Å¡Ã‚Â¬Ãƒâ€¦Ã‚Â¡ÃƒÆ’Ã¢â‚¬Å¡Ãƒâ€šÃ‚Â¬ÃƒÆ’Ã†â€™ÃƒÂ¢Ã¢â€šÂ¬Ã‚Â¦ÃƒÆ’Ã¢â‚¬Å¡Ãƒâ€šÃ‚Â¡ÃƒÆ’Ã†â€™Ãƒâ€ Ã¢â‚¬â„¢ÃƒÆ’Ã¢â‚¬Â ÃƒÂ¢Ã¢â€šÂ¬Ã¢â€žÂ¢ÃƒÆ’Ã†â€™Ãƒâ€šÃ‚Â¢ÃƒÆ’Ã‚Â¢ÃƒÂ¢Ã¢â€šÂ¬Ã…Â¡Ãƒâ€šÃ‚Â¬ÃƒÆ’Ã¢â‚¬Â¦Ãƒâ€šÃ‚Â¡ÃƒÆ’Ã†â€™Ãƒâ€ Ã¢â‚¬â„¢ÃƒÆ’Ã‚Â¢ÃƒÂ¢Ã¢â‚¬Å¡Ã‚Â¬Ãƒâ€¦Ã‚Â¡ÃƒÆ’Ã†â€™ÃƒÂ¢Ã¢â€šÂ¬Ã…Â¡ÃƒÆ’Ã¢â‚¬Å¡Ãƒâ€šÃ‚Â° C
is supposed to live, then she'd,
a tweet and that would look something
I wonder if I can switch the heating
It's 21ÃƒÆ’Ã†â€™Ãƒâ€ Ã¢â‚¬â„¢ÃƒÆ’Ã¢â‚¬Â ÃƒÂ¢Ã¢â€šÂ¬Ã¢â€žÂ¢ÃƒÆ’Ã†â€™ÃƒÂ¢Ã¢â€šÂ¬Ã‚Â ÃƒÆ’Ã‚Â¢ÃƒÂ¢Ã¢â‚¬Å¡Ã‚Â¬ÃƒÂ¢Ã¢â‚¬Å¾Ã‚Â¢ÃƒÆ’Ã†â€™Ãƒâ€ Ã¢â‚¬â„¢ÃƒÆ’Ã¢â‚¬Å¡Ãƒâ€šÃ‚Â¢ÃƒÆ’Ã†â€™Ãƒâ€šÃ‚Â¢ÃƒÆ’Ã‚Â¢ÃƒÂ¢Ã¢â‚¬Å¡Ã‚Â¬Ãƒâ€¦Ã‚Â¡ÃƒÆ’Ã¢â‚¬Å¡Ãƒâ€šÃ‚Â¬ÃƒÆ’Ã†â€™ÃƒÂ¢Ã¢â€šÂ¬Ã‚Â¦ÃƒÆ’Ã¢â‚¬Å¡Ãƒâ€šÃ‚Â¡ÃƒÆ’Ã†â€™Ãƒâ€ Ã¢â‚¬â„¢ÃƒÆ’Ã¢â‚¬Â ÃƒÂ¢Ã¢â€šÂ¬Ã¢â€žÂ¢ÃƒÆ’Ã†â€™Ãƒâ€šÃ‚Â¢ÃƒÆ’Ã‚Â¢ÃƒÂ¢Ã¢â€šÂ¬Ã…Â¡Ãƒâ€šÃ‚Â¬ÃƒÆ’Ã¢â‚¬Â¦Ãƒâ€šÃ‚Â¡ÃƒÆ’Ã†â€™Ãƒâ€ Ã¢â‚¬â„¢ÃƒÆ’Ã‚Â¢ÃƒÂ¢Ã¢â‚¬Å¡Ã‚Â¬Ãƒâ€¦Ã‚Â¡ÃƒÆ’Ã†â€™ÃƒÂ¢Ã¢â€šÂ¬Ã…Â¡ÃƒÆ’Ã¢â‚¬Å¡Ãƒâ€šÃ‚Â° C.
And so this is creating some content
Then we've got our target users, 305,
If any followed back,
in an interaction CSV file.
That's how technical it got.
No databases, just CSV.
Yeah.
Let's hear it for CSV files.
(Applause.) (Inaudible.) CHRIS
So then we started tweeting,
shit basically.
And this was not what we tweeted,
is what ArrowFate's bot tweeted,
as slightly misogynistic,
felt like we might get into a little bit
like this instead.
(Laughter.) And then, you know,
So these were obvious cues that
the person she's pretending to be
we swapped any references to cats
That's essentially all we did.
So you might think, yes, that's banal,
to keep it as close as possible
So we only switched dog for cats
construed as slightly misogynistic.
So all of the responses to that were,
in an interactions file.
Once we'd targeted all of the users, i.e.
followed them all,
to see if we could generate a response.
This is how ArrowFate's bot actually
So we had questions along the line
What's better, a dog or a cat?
And then we'd look at trying
or maintain a conversation
So ArrowFate's blog would just
of tweets,
an Eliezer engine.
So you'd get something like this.
Hello.
And it would say, how
What I liked about ArrowFate's bot
sometimes, so we wanted
of the random tweets
of the coin as to what you'd get.
You could get a "it's interesting"
(Laughter.) So it could go kind
But we also had to consider
So, you know,
in there so we could actually look
out before she tweeted it out.
And this is why, this is ArrowFate's bot.
Do you have any pets, and if so, what?
No, I don't currently have any pets
a few years ago,
LOL that rules.
(Applause and laughter.).
So props to ArrowFate
after reading that blog post.
So we've got a ton of limitations,
about and you're right.
We use basic measures of personality,
We had a pretty basic and dumb social
So maybe the questions had
As the experiment progressed,
and arguably gave it more credibility
if the people knew it was
playing along.
So, but either way,
pretty much identical
which has already got some research
Although we looked at the dimension
So what did we find?
Well, we got 20% return
the ArrowFate bot had got 40%,
for a number of reasons.
We targeted a diverse range
they targeted people who were liking
We also had an old lady.
They had a dude
so there are a number
But we had 124 interactions,
which could be automated for sure,
So the most
interactions with 1 user,
and there's kind
to why we had 65 people replied just
So the difference between us
if we look at it on a sort
We also had some interesting funny
which ArrowFate had also noticed
So you get we had this interaction
and only nouns to describe yourself,
it rhymes with runt, almost ten times,
with the Eliezer engine,
(Laughter.) Which gave me a chuckle
And that was actually one
of conducting this research
at the interactions.
(Laughter.) I'm laughing.
(Laughter.) Bring it on, essentially,
But we also got spotted as well,
what do you do for a living.
This is a pretty clear, you know,
Write software
And she responds
you're right and when you're right,
(Laughter.) Granny failing showing test
is still far off or something like that.
Brilliant.
Okay.
So looking at personality,
he indicated people might not be
from the '80s.
Well, that's Ferris Bueller
We found that extroverted users were
statistically significantly more likely
and that's using
invented by a professor in Texas.
We found that Klout score was
There were no other personality traits,
at a bunch of them.
Klout score, friends, and followers
relationship, albeit relatively weak.
So what.
Well, in terms of eLearning
most eLearning around phishing
of a one size fits all approach,
about that, that's targeted more
There's some papers that look
and personality, and
a better time
over their learning experience,
So I think there's some mileage
relates in the context
of corporate eLearning experience.
So moving on quickly
This was our baseline performance,
true positives in green.
We want to avoid this bit really
the people that are probably going
try and aim for this, which
We're going to go light on some
And the real aim here for us isn't
is to achieve perfection,
More realistically, what we want
the false positive rate
people and hopefully getting more
to responding.
So for the next five minutes we're going
of sort of the machine learning stuff
up five minutes after that.
And so it's my pleasure to introduce Dr.
Randall Wald, who I met here
in the Q & A room.
(Applause.) RANDALL WALD: Okay.
As Chris was saying,
we wanted to actually work
understand what's really going
us make better predictions
So it's very basic.
This is what data looks like.
You've got instances.
In this case we've got three instances
You're going to see, you know, Alice,
and instance each instance has many
In this case,
the class attribute,
and then also
which are the least information we can
So the concept here
is labeled where we know
whether or not they responded
to use this to help us build
in the future say I have a new person,
to them, should my bot try
Should I try to work with them?
I'm going to use the features,
out about this person to build
whether or not that individual
so they're going to be a good target
So we did two separate experiments
The first was to figure
because the data we've been working
all the users,
properties of them.
For example, the number of friends,
how long they've been on Twitter,
that sort of thing.
We also took the content
out like how many what type
Are they using expletives a lot?
Are they using I and me a lot?
Are they using we?
Different types of words like, that try
is most relevant to the problem
Will they reply with the bot?
So we wanted to figure out, first,
and then once we figured out,
to build our models to use that
whether an individual will interact
So all of our experiments here
the WEKA open source tool.
It is available
I don't know if it works on BSD,
desktop operating systems.
Java.
You can extend it,
adding additional tools to it,
I encourage you all to download
in learning more
Our results.
First, the interactive.
We wanted
likely to interact, so we used a bunch
out with these three properties
if they're going to interact with the bot,
The first is the Klout score and
number of followers.
So those were, as Chris was saying,
the features that were most relevant
We also wanted to look
We found something a little different.
So same concept, all the things,
is number one.
But percent follow Friday
feature here.
So that's basically what percentage
#followFriday, that sort of thing,
because we have these two different
a little differently.
Once we figured out these features,
to build models that could actually
into whether they're going to interact
to reply or not.
So we built a number
features, etc., and found
rate of 61%, true negative of 71%, or
we see here we have true positives,
and false negatives.
Now, you're going to look at this
but your model still gave more false
If the model predicted that someone
it's still going
However, compared to the gray boxes,
to everyone and try
we're able to eliminate
otherwise have gotten the message,
to respond anyway,
of hitting people we care about.
So, remember,
a message, somebody is not going
a chance they're going
So you want to minimize the number
to maximize the effectiveness
And we did related models
That's a little more challenging
with a performance a little worse here.
And we found also that different models
And this data mining is important
we always just use the best model.
What's the challenge
and looking how they perform.
Different data sets are going
the optimal model will vary.
We see also here similar performance
more false positives than true positives.
But we're significantly able
of our model, improve the chances
to hit those people we care about,
to the bot,
a better bot that will be able
before it gets reported on Twitter.
So overall we found
is that we have our a model able
the features that are most important
We also found that the two data sets,
So actually Klout score, which
is important for interaction and reply.
These whether the number
mattered more
followFriday matters more for replies.
And even though our models were not
we still have a large number
to build models that let us build bots
likely to be susceptible to the bot.
So with that, I'm going to give it back
CHRIS SUMNER: Thanks, Randall.
So wrapping
So, again,
the greatest risk
We can look at that in maybe just
And bot masters could use machine
of their bots.
So the key here isn't that
at to improve their bot,
to our particular bot.
When Wagner looked
the sort
because they had a larger majority
larger minority class.
However, so you create a bot,
You use machine learning
and then you target it
of how we think this could be
We're not suggesting people do that.
But marketers have got
into that sort of behavior.
And marketers, if you ever go
conference, are actually sort
to improve engagement and understand
all of that sort of stuff.
They are kind of light years ahead
Propagandists
to find people who are more likely
maybe propagate false messages
mentioned that Wecker's got
Well, you can build that model
so that you get a bunch
the likelihood in a different color here
is going to respond.
So find the most gullible potentially
in your organization and reduce your
The other thing, this
this could be used in terms
So help people understand
or more at risk from a social bot.
We see that with browsers and now
in a social network environment.
So you probably want
(Laughter.) So training.
I mentioned training.
You know, focus
at risk.
So maybe it's your sales team who are
more extroverted naturally.
Then in terms of future research here,
of the big five, specifically
which has many facets.
Specifically, I'm thinking impulsivity.
Impulsivity seems to be related
to phishing messages,
cognitive reflective test to see
So here's an example of one
in the cognitive reflective test.
A bat and a ball cost $1.10 in total.
A bat costs $1 more than the ball.
How much does the ball cost?
So get in that getting that wrong
I think there are three questions
So also maybe
Ours was a one size fits
but you could actually start looking
of the call group you're following
But it's not all negative necessarily.
There's a tall German gentleman
who does a lot of work in this field
of his videos
He mentioned that OK cupid had
they created their own bot.
And what they did when
is they created a replica
talk to other bots and now there's
of OK cupid where you've just got what
on bot (Laughter.) (Applause.)
whether that's true or not,
I have no reason
because he's taller than I am.
(Laughter.) CHRIS SUMNER: So
this gets back to Roelof Temmingh
illustrations
activity of the computer should reveal
But maybe that's going
to need more and more time
or maybe you don't care.
But, you know, the key thing here
learning to improve the performance
but it also shows that that's actually
the security community and folks
about tackling because this sort
on social media could be making users
to respond and perhaps need more
So that, ladies and gentlemen,
We'll take questions
(Applause.) Thanks a lot.
So before you all leave,
You know,
we're doing those pub crawly things,
of alcohol and strippers.
Yeah.
Maybe you shouldn't be bringing
Just saying.
You're an adult.
You can make your own evaluated
But What did they call the party,
Well, they did call it the petting zoo.
But if there's a petting zoo in Vegas
Anywhere.
(Laughter.) Then maybe, just maybe,
about what content
That's all.
Thanks for the updates!
Goodbye!
(Applause.) Could I ask
Chris?
Yeah, if you want to come
Well, I just wanted
is that a WhiteHouse.gov,
Or is that a data mining project
CHRIS SUMNER: I'm afraid I'm not
because the next speaker is coming on.
So, if you want to hop up,
and maybe understand more.
Thank you very much.
Appreciate it.
CHRIS SUMNER: No problem.
***** This is being provided
Communication Access Realtime
in order
accessibility and may not be
of the Proceedings.