Patent ID: 8621626

Claim:
A non-transitory computer readable storage device comprising instructions stored thereon to cause one or more processors to: determine where one or more candidate areas exist within an arbitrary file; search at least one nearby area in front of or after each of the one or more candidate areas within the arbitrary file for an instruction candidate; disassemble instructions starting at a found offset for the instruction candidate to create a disassembled instruction set, the found offset reflecting a location of the instruction candidate within the arbitrary file; normalize at least a portion of the disassembled instruction set to create a normalized instruction set; scan the normalized instruction set to determine if the normalized instruction set reflects that the disassembled instruction set has a probability of containing shellcode; calculate a statistical probability that the instruction candidate is associated with shellcode for normalized instruction sets associated with disassembled instruction sets that were determined to reflect a probability of containing shellcode, wherein statistical probabilities are not calculated if the normalized instruction set reflects that the disassembled instruction set has no probability of containing shellcode; for a given stream of instructions starting at the found offset, map an instruction-to-shellcode probability to each instruction in the given stream of instructions; and sum the mapped instruction-to-shellcode probability for each instruction using Bayes' formula to generate an overall probability.