One of the foundational blocks of current state-of-the-art code obfuscation are Mixed Boolean-Arithmetic (MBA) expressions: those combining both integer arithmetic and bitwise operators. Such expressions can be leveraged to arbitrarily increase the data-flow complexity of targeted code by iteratively applying rewrite rules and function identities which mess the syntax while preserving its semantic behavior. They can also be leveraged to conceal sensitive data that must be accessible through the program in runtime: cryptographic keys, known constants for hashing algorithms, etc. The use of such expressions is motivated by the fact that combinations of operators from these different fields do not interact well together: we have no rules (distributivity, factorization…) or general theory to deal with this mixing of operators.
Through the course of this 2 hour session, we will explore how to apply MBA transformations to build robust obfuscation primitives from a practical standpoint: ranging from opaque predicates to VM-handlers of a virtualization based obfuscation scheme.