FireEye’s Lanstein and Wolf speak at Black Hat
I’ve worked in the security industry for nearly five years, and it was apparent early on that the most successful people in this field bring to their work a passion and a commitment to protecting not only one’s customers, but to providing a certain level of information about security threats to the world at-large, so even your non-customers can help or protect themselves.
It can be hard to know where to stop once you get on a roll. Malware infections frequently lead to unexplored, interesting backwaters on the Internet. And, sometimes, those backwaters are where the criminals run those operations. When I stumble upon a criminal network or a botnet controller, it simply doesn’t feel like I’ve done enough when I merely add signatures which block or remediate infections and communications with a command-and-control server from Webroot customers. If malicious behavior depends on one or more Internet sites that send instructions, my (and many others’) initial reaction is we need to shut that down, permanently. But sometimes, a too-rapid reaction can blow back in your face.
Obviously, that was also the case when Alex Lanstein and Julia Wolf of internet security firm FireEye stumbled upon the Rustock botnet. At one time, before law enforcement in several countries swooped in on the data centers hosting the botnet’s command-and-control (CnC) infrastructure in a coordinated raid earlier this year, the massive network of Rustock-infected computers was responsible for about half of spam flooding the ‘net. The researchers’ instincts to engineer a takedown of the botnet sounded very familiar, but their initial attempts to do so backfired, and may have even spurred the malware developers to change their game, and may have made it more difficult, eventually, to eliminate the CnC altogether.
Wolf, incidentally, was also instrumental in helping law enforcement shut down the Srizbi spam botnet, which in 2008 held the notorious distinction of being the largest source of spam by volume of any botnet at the time. Lanstein also helped shut down Srizbi and another botnet called Mega-D. So they’re not exactly green recruits to this business.
The FireEye researchers studied Rustock, learning its secrets — which were not well hidden initially. In fact, the botnet’s behavior and activity was performed in plain sight, something we at Webroot also noticed when we started analyzing Rustock samples, and created our first anti-Rustock signatures five years ago this month, in the summer of 2006.
Infected machines, in the early days of Rustock, contacted their CnC in ways that humans could easily detect. Researchers initially called the botnet Grum, in fact, because one command the bot used to pull down payloads to an infected machine was get_grum.php.
When the bots needed to pull down a new list of message bodies to spam out, or addresses of recipients, the bots contacted a URL and requested a script in the /spm/ directory on the Web server. Note to malware creators: It’s not especially stealthy to name your spam bot’s command directory something that looks pretty much like the word “spam.”
But stealth didn’t come into play until the researchers sent alert notices to the upstream ISP — the company that provides Internet service to the ISP hosting the CnC servers. At the time, most botherders had been paying for so-called bulletproof hosting servers, which would ignore or deflect these kinds of abuse alerts and permit the criminals to continue operating as long as possible. But Rustock’s operators didn’t use these well-known bad ISPs. Instead, they used small ISPs based in the United States to host their CnC infrastructure.
Sending those notices to the ISPs simply alerted the botnet operators that their operation was under investigation. The takedown notices did not prove to be particularly useful for anyone except the criminals.
Once the botherders got word that the security researchers were on the case, they changed up their tactics: the bot’s CnC URLs were re-engineered so they looked like random Web traffic.
And the bots also began sending their CnC inquiries as User-Agent strings, which are not as easily noticed, unless you know what to look for. User-Agent strings are sent by the browser with almost every HTTP request, and typically identify the name and version of the browser and operating system of the Web surfer’s computer to the Web server.
The server-side code also began to change when Rustock’s creators realized that researchers were able to replicate their CnC commands in order to retrieve payloads. The servers would provide files to their bots, but not to generic queries, which instead caused the server to display a 404 “file not found” error if you tried to replicate the bot’s command but didn’t use all the bot’s secret sauce.
In 2008, the operators of Rustock also began multi-homing their CnC servers, hosting them with a number of Web hosting ISPs, including the notoroious McColo, which was itself shut down for generating such a volume of malicious traffic. The heterogenous list of hosts meant the botnet was able to resist attempts to shut down parts of its operation, or quickly get back online if a server was taken down.
The CnC servers, which initially delivered part of the Rustock payload as a driver file, also began serving up the payload differently. Payloads would be delivered in a scrambled form with a .rar or .bin extension, and the client would unscramble the code and rename the payload file (usually a .sys driver) with random names. Eventually, the developers of the bots began to use increasingly complex, custom encryption methods to scramble the payload data.
A typical Rustock-delivered spam message, this one for a Nigerian 419-type advance fee fraud scam
The botnet operators also began, towards the end of the botnet’s operational life, to try to mask the destination IP address of the CnC servers by instructing the bots to modify the results of DNS lookups. If the server returned an IP address of 4.3.2.1 the bot would mathematically calculate an IP address that was different, and use that IP to contact its CnC server instead of the one listed in the real DNS record.
In the end it was Microsoft’s use of innovative legal tactics — including invoking the Lanham Act, a trademark protection law — that spurred law enforcement to get involved. The software giant named the operation to take down Rustock was called Operation b107. Law enforcement swept into data centers in multiple locations around the world, virtually simultaneously, and secured the botnet’s CnC servers before Rustock’s operators could send any more commands to the network. Meanwhile, Microsoft seized control of the domain names used to host CnC for the botnet.
Photo of a suspected Rustock operator, Dmitri Sergeev
The identities of the botnet operators, who have never been brought to justice, may never be known, though there have been compelling hints. But it’s far from the only mystery. There’s been a lot of speculation as to why, for example, the Rustock network inexplicably stopped sending all spam on December 25, 2010 and then resumed on January 9, 2011 but nobody really knows why it happened. It remains one of the mysteries about this botnet that may never be revealed.
On March 16, 2011, the botnet takedown day, Rustock sent its last Viagra spam, and you won’t catch me shedding any tears. Despite the early setbacks, the Internet is a safer place today thanks to the efforts of these companies and the dedication of these researchers.
Hi Guys,
Haven’t had any system problem lately so I haven’t need to call support. That said what a great article. I find these shadowy goings on very interesting. Intellectual games of cat and mouse. If it weren’t for the fact these people the spammers wreck peoples computers and lives. They would just be games. Thanks for keeping my 6 computers safe.
Keep up the good work,
Mark Barton