WES BROWN (Chief Architect, ThreatGRID)

PRESENTATION TITLE: Supercomputing, Malware, and Correlation: What A Year in the Life of a MD5 Taught Us

PRESENTATION ABSTRACT: 

For more than two years, ThreatGRID has been building a threat intelligence service where samples and content are cross-indexed and related. This allows for tremendous amounts of derived analysis, building relationships based on timing, behavioral, structural, and communications characteristics. We are able to determine origin, aims, and targets of specific samples via second and third order relationships. We track all artifacts and beheaviors, both host and network, and correlate between any of them.

Content is generated through dynamic and static malware analysis. We do perform de-duplication of samples that are collected in the wild and submitted through various sources. Even though a piece of malware can be identified as belonging to a particular family of rootkit or dropper, their characteristics change and evolve over time. These ephemeral behavioral characteristics are vital to identifying relationships between malware, and this is content that we don’t want to miss. We’ve been submitting and analyzing a sample for about a year now, tracking how its functionality, content and relationships have changed over time. This approach of not deduping submissions leads to some interesting issues related to scaling, storage and infrastructure design.

This talk covers the infrastructure requirements and architectural decisions made to facilitate being able to analyze the entire worldwide output of malware samples multiple times; we have built our own in-house supercomputing cluster, with petabyte scalable storage, and a 40gbps interconnect. We will also show the value of such correlation, and why everyone should be building these relationships between content.

ABOUT WES BROWN