Twitter has something of a bot problem. Anyone who uses the platform on even an occasional basis likely could point out automated accounts without much trouble. But detecting bots at scale is a much more complex problem, one that a pair of security researchers decided to tackle by building their own classifier and analyzing the characteristics and behavior of 88 million Twitter accounts.
Using a machine learning model with a set of 20 distinct characteristics such as the number of tweets relative to the age of the account and the speed of replies and retweets, the classifier is able to detect bots with about 98 percent accuracy. The tool outputs a probability that a given account is a bot, with anything above 50 percent likely being a bot. During their research, conducted from May through July, Jordan Wright and Olabode Anise of Duo Security discovered an organized network of more than 15,000 bots that was being used to promote a cryptocurrency scam. The botnet, which is still partially active, spoofs many legitimate accounts and even took over some verified accounts as part of a scheme designed to trick victims into sending small amounts of the cryptocurrency Ethereum to a specific address.
Unlike most botnets, the Ethereum network has a hierarchical structure, with a division of labor among the bots. Usually, each bot in a network performs the same task, whether that’s launching a DDoS attack or mining Bitcoin on a compromised machine. But the Ethereum botnet had clusters of bots with a three-tier organization. Some of the bots published the scam tweets, while others amplified those tweets or served as hub accounts for others to follow. Wright and Anise mapped the social media connections between the various accounts and looked at which accounts followed which others to create a better picture of the network.
“Each bot had its own role and the botnet evolved over time. They started by targeting legitimate cryptocurrency accounts, like the official Bitcoin account, and then moved on from there,” said Wright, an R&D engineer at Duo Labs.
“They changed to targeting celebrities and then posing as legitimate news accounts. We found different clusters that showed how the owner moved them over time.”
"There were times when some accounts were behaving like bots and others when they looked legitimate.”
Anise and Wright will discuss the results of their research during a talk at the Black Hat USA conference on Wednesday and will release their detection tool as an open source project that day, too.
The operator of the botnet changed the appearance of tweets over time, adding or removing whitespace and sometimes using Unicode characters rather than letters, in an effort to make the tweets look different and fool human users. One of the challenges Wright and Anise faced with their research was distinguishing legitimate accounts that may employ some automation from bot accounts. Many legitimate Twitter accounts use automation as a way to interact with other users quickly.
“Automation by itself isn’t bad. Legitimate accounts use it too, a lot of times as a customer service tool to respond to questions,” Anise, a data scientist, said. “We looked at two hundred tweets from each account we studied and there were times when some accounts were behaving like bots and others when they looked legitimate.”
Twitter has more than 336 million active users, and the company has had to deal with the bot problem for many years now and has created tools and strategies to identify and remove bots. Recently, Twitter officials said they had removed more than 70 million such accounts in a two-month period earlier this year. Wright and Anise notified Twitter about the cryptocurrency scam botnet they discovered, and the company has taken steps to address it.
“Twitter is aware of this form of manipulation and is proactively implementing a number of detections to prevent these types of accounts from engaging with others in a deceptive manner. Spam and certain forms of automation are against Twitter's rules. In many cases, spammy content is hidden on Twitter on the basis of automated detections,” a Twitter spokesperson said.
Anise and Wright plan to continue the research in the future by finding a way to identify malicious bots.